Back to search

Cornell University

Plenoptic Imaging and Vision using Angle Sensitive Pixels

Abstract

dc:description.abstract

Computational cameras with sensor hardware co-designed with computer vision and graphics algorithms are an exciting recent trend in visual computing. In particular, most of these new cameras capture the plenoptic function of light, a multidimensional function of radiance for light rays in a scene. Such plenoptic information can be used for a variety of tasks including depth estimation, novel view synthesis, and inferring physical properties of a scene that the light interacts with. In this thesis, we present multimodal plenoptic imaging, the simultaenous capture of multiple plenoptic dimensions, using Angle Sensitive Pixels (ASP), custom CMOS image sensors with embedded per-pixel diffraction gratings. We extend ASP models for plenoptic image capture, and showcase several computer vision and computational imaging applications. First, we show how high resolution 4D light fields can be recovered from ASP images, using both a dictionary-based machine learning method as well as deep learning. We then extend ASP imaging to include the effects of polarization, and use this new information to image stress-induced birefringence and remove specular highlights from light field depth mapping. We explore the potential for ASPs performing time-of-flight imaging, and introduce the depth field, a combined representation of time-of-flight depth with plenoptic spatio-angular coordinates, which is used for applications in robust depth estimation. Finally, we leverage ASP optical edge filtering to be a low power front end for an embedded deep learning imaging system. We also present two technical appendices: a study of using deep learning with energy-efficient binary gradient cameras, and a design flow to enable agile hardware design for computational image sensors in the future.

Degree

thesis:*
Name thesis:degree_name
Ph. D., Electrical and Computer Engineering
Level thesis:degree_level
Doctor of Philosophy
Discipline thesis:degree_discipline
Electrical and Computer Engineering
Grantor
Cornell University
Year dc:date.issued
2017

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Jayasuriya, Suren
Chair dc:contributor.chair
  • Molnar, Alyosha Christopher
Committee members dc:contributor.committeemember
  • Marschner, Stephen Robert
  • Apsel, Alyssa B.

Subjects

dc:subject × 9

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-ShareAlike 4.0 International
Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Dc Identifier Bibid
9906087
Dc Identifier Other
ProQuest Submission ID: 10076
ProQuest Publication ID: 10249017

Chain of custody

source
Harvested from
Cornell University
Base URL
ecommons.cornell.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Jayasuriya, Suren. Plenoptic Imaging and Vision using Angle Sensitive Pixels. Doctor of Philosophy thesis, Cornell University, 2017. https://hdl.handle.net/1813/47840