Imaging systems play an important role in many diverse applications. Requirements for these applications, however, can lead to complex or sub-optimal designs. Traditionally, imaging systems are designed to yield a visually pleasing representation, or "pretty picture", of the scene or object. Often this is because a human operator is viewing the acquired image to perform a specific task. With digital computers increasingly being used for automation, a large number of algorithms have been designed to accept as input a pretty picture. This isomorphic representation however is neither necessary nor optimal for tasks such as data compression, transmission, pattern recognition or classification. This disconnect between optical measurement and post processing for the final system outcome has motivated an interest in computational imaging (CI). In a CI system the optical sub-system and post-processing sub-system is jointly designed to optimize system performance for a specific task. In these hybrid imagers, the measured image may no longer be a pretty picture but rather an intermediate non-traditional measurement. In this work, applications of non-traditional measurements are considered for computational imaging. Two systems for an image reconstruction task are studied and one system for a detection task is investigated. First, a CI system to extend the field of view is analyzed and an experimental prototype demonstrated. This prototype validates the simulation study and is designed to yield a 3x field of view improvement relative to a conventional imager. Second, a CI system to acquire time-varying natural scenes, i.e. video, is developed. A candidate system using 8x8x16 spatiotemporal blocks yields about 292x compression compared to a conventional imager. Candidate electro-optical architectures, including charge-domain processing, to implement this approach are also discussed. Lastly, a CI system with x-ray pencil beam illumination is investigated for a detection task where system performance is quantified using an information-theoretic metric.

Quantitative three dimensional maps of cellular structure, activity and function provide the key to answering many prevalent questions in modern biological research. Fluorescence microscopy has emerged as an indispensable tool in generating such maps, but common techniques are limited by fundamental physical constraints which render them incapable of simultaneously achieving high spatial and temporal resolution. This thesis will describe the development of novel microscopy techniques and complementary computational tools capable of addressing some of the aforementioned limitations of fluorescence microscopy and further outline their application in providing novel biological insights. The first section details the design of a light sheet microscope capable of high-throughput imaging of cleared, macroscopic samples with cellular resolution. In light sheet microscopy, the combination of spatially confined illumination with widefield detection enables multi-megapixel acquisition in a single camera exposure. The corresponding increase in acquisition speed enables systems level biological studies to be performed. The ability of this microscope to perform rapid, high-resolution imaging of intact samples is demonstrated by its application in a project which established a niche and hierarchy for stem cells in the adult nervous system. Light sheet microscopy achieves fast volumetric imaging rates, but the two dimensional nature of each measurement results in an inevitable lag between acquisition of the initial and final planes. The second section of this thesis describes the development and optimization of a light field microscope which captures volumetric information in a snapshot. Light field microscopy is a computational technique and images are reconstructed from raw data. Both the fidelity of computed volumes and the efficiency of the algorithms are strongly dependent on the quality of the rectification. A highly accurate, automated procedure is presented in this section. Light field reconstruction techniques are investigated and compared and the results are used to inform the re-design of the microscope. The new optical configuration is demonstrated to minimize the long-object problem. In the final section of the thesis, the spatial resolution limits of light field microscopy are explored using a combination of simulations and experiments. It is shown that light field microscopy is capable of localizing point sources over a large depth of field with high axial and lateral precision. Notably, this work paves the way towards frame rate limited super resolution localization microscopy with a depth of field larger than the thickness of a typical mammalian cell.

Non-line-of-sight (NLOS) imaging is the inference of the properties of objects or scenes outside of the direct line-of-sight of the observer. Such inferences can range from a 2D photograph-like image of a hidden area, to determining the position, motion or number of hidden objects, to 3D reconstructions of a hidden volume. NLOS imaging has many enticing potential applications, such as leveraging the existing hardware in many automobiles to identify hidden pedestrians, vehicles or other hazards and hence plan safer trajectories. Other potential application areas include improving navigation for robots or drones by anticipating occluded hazards, peering past obstructions in medical settings, or in surveying unreachable areas in search-and-rescue operations. Most modern NLOS imaging methods fall into one of two categories: active imaging methods that have some control of the illumination of the hidden area, and passive methods that simply measure light that already exists. This thesis introduces two NLOS imaging methods, one of each category, along with modeling and data processing techniques that are more broadly applicable. The methods are linked by their use of objects (‘occluders’) that reside somewhere between the observer and the hidden scene and block some possible light paths. Computational periscopy, a passive method, can recover the unknown position of an occluding object in the hidden area and then recover an image of the hidden scene behind it. It does so using only a single photograph of a blank relay wall taken by an ordinary digital camera. We develop also a framework using an optimized preconditioning matrix to improve the speed at which these reconstructions can be made and greatly improve the robustness to ambient light. Lastly, we develop tools necessary to demonstrate recovery of scenes at multiple unknown depths – paving the way towards three-dimensional reconstructions. Edge-resolved transient imaging, an active method, enables the formation of 2.5D representations – a plan view plus heights – of large-scale scenes. A pulsed laser illuminates spots along a small semi-circle on the floor, centered on the edge of a vertical wall such as in a doorway. The wall edge occludes some light paths, only allowing the laser light reflecting off of the floor to illuminate certain portions of the hidden area beyond the wall, depending on where along the semi-circle it is illuminating. The time at which photons return following a laser pulse is recorded. The occluding wall edge provides angular resolution, and time-resolved sensing provides radial resolution. This novel acquisition strategy, along with a scene response model and reconstruction algorithm, allow for 180° field of view reconstructions of large-scale scenes unlike other active imaging methods. Lastly, we introduce a sparsity penalty named mutually exclusive group sparsity (MEGS), that can be used as a constraint or regularization in optimization problems to promote solutions in which certain components are mutually exclusive. We explore how this penalty relates to other similar penalties, develop fast algorithms to solve MEGS-regularized problems, and demonstrate how enforcing mutual exclusivity structure can provide great utility in NLOS imaging problems.

<p dir="ltr">We propose a hardware driven solution to high dynamic range (HDR) imaging in the form of a single metasurface lens. Our design consists of a metasurface capable of forming nine low dynamic range (LDR) sub-images of varying intensities scaling by a factor of 2 onto an imaging sensor. After synthetically verifying the functionality of our design, the metasurface is fabricated and a prototype system is constructed for real world experiments. Utilizing the experimental system, the compatibility of our extracted LDR sub- images with pre-existing exposure bracketing solutions for multi-image HDR fusion is demonstrated. The resulting HDR images are highly robust to scene motion due to the instantaneous capture of multi-exposure LDR sub-images allowing for HDR video capabilities.</p>

<p>'Image volumes' refer to realizations of images in other dimensions such as time, spectrum, and focus. Recent advances in scientific, medical, and consumer applications demand improvements in image volume capture. Though image volume acquisition continues to advance, it maintains the same sampling mechanisms that have been used for decades; every voxel must be scanned and is presumed independent of its neighbors. Under these conditions, improving performance comes at the cost of increased system complexity, data rates, and power consumption. </p><p>This dissertation explores systems and methods capable of efficiently improving sensitivity and performance for image volume cameras, and specifically proposes several sampling strategies that utilize temporal coding to improve imaging system performance and enhance our awareness for a variety of dynamic applications. </p><p>Video cameras and camcorders sample the video volume (x,y,t) at fixed intervals to gain understanding of the volume's temporal evolution. Conventionally, one must reduce the spatial resolution to increase the framerate of such cameras. Using temporal coding via physical translation of an optical element known as a coded aperture, the compressive temporal imaging (CACTI) camera emonstrates a method which which to embed the temporal dimension of the video volume into spatial (x,y) measurements, thereby greatly improving temporal resolution with minimal loss of spatial resolution. This technique, which is among a family of compressive sampling strategies developed at Duke University, temporally codes the exposure readout functions at the pixel level.</p><p>Since video cameras nominally integrate the remaining image volume dimensions (e.g. spectrum and focus) at capture time, spectral (x,y,t,\lambda) and focal (x,y,t,z) image volumes are traditionally captured via sequential changes to the spectral and focal state of the system, respectively. The CACTI camera's ability to embed video volumes into images leads to exploration of other information within that video; namely, focal and spectral information. The next part of the thesis demonstrates derivative works of CACTI: compressive extended depth of field and compressive spectral-temporal imaging. These works successfully show the technique's extension of temporal coding to improve sensing performance in these other dimensions.</p><p>Geometrical optics-related tradeoffs, such as the classic challenges of wide-field-of-view and high resolution photography, have motivated the development of mulitscale camera arrays. The advent of such designs less than a decade ago heralds a new era of research- and engineering-related challenges. One significant challenge is that of managing the focal volume (x,y,z) over wide fields of view and resolutions. The fourth chapter shows advances on focus and image quality assessment for a class of multiscale gigapixel cameras developed at Duke.</p><p>Along the same line of work, we have explored methods for dynamic and adaptive addressing of focus via point spread function engineering. We demonstrate another form of temporal coding in the form of physical translation of the image plane from its nominal focal position. We demonstrate this technique's capability to generate arbitrary point spread functions.</p> / Dissertation

We have previously introduced a high throughput multiplexing computational spectral imaging device. The device measures scalar projections of pseudo-arbitrary spectral filters at each spatial pixel. This paper discusses simulation and initial experimental progress in performing computational spectral unmixing by taking advantage of the natural sparsity commonly found in the fractional abundances. The simulation results show a lower unmixing error compared to traditional spectral imaging devices. Initial experimental results demonstrate the ability to directly perform spectral unmixing with less error than multiplexing alone.

Imaging systems often make use of macroscopic lenses to manipulate light. Modern microfabrication techniques, however, have opened up a pathway to the development of novel arrayed imaging systems. In such systems, centimeter-scale areas can contain thousands to millions of micro-scale optical elements, presenting exciting opportunities for new imaging applications. We show two such applications in this thesis: pressure sensing in microfluidics and high throughput fluorescence microscopy for high content screening. Conversely, we show that arrayed elements are not always needed for three dimensional light field imaging. / Engineering and Applied Sciences

<p>Metamaterials extend the design space, flexibility, and control of optical material systems and so yield fundamentally new computational imaging systems. A computational imaging system relies heavily on the design of measurement modes. Metamaterials provide a great deal of control over the generation of the measurement modes of an aperture. On the other side of the coin, computational imaging uses the data that that can be measured by an imaging system, which may limited, in an optimal way thereby producing the best possible image within the physical constraints of a system. The synergy of these two technologies - metamaterials and computational imaging - allows for entirely novel imaging systems. These contributions are realized in the concept of a frequency-diverse metamaterial imaging system that will be presented in this thesis. This 'metaimager' uses the same electromagnetic flexibility that metamaterials have shown in many other contexts to construct an imaging aperture suitable for single-pixel operation that can measure arbitrary measurement modes, constrained only by the size of the aperture and resonant elements. It has no lenses, no moving parts, a small form-factor, and is low-cost.</p><p>In this thesis we present an overview of work done by the author in the area of metamaterial imaging systems. We first discuss novel transformation-optical lenses enabled by metamaterials which demonstrate the electromagnetic flexibility of metamaterials. We then introduce the theory of computational and compressed imaging using the language of Fourier optics, and derive the forward model needed to apply computational imaging to the metaimager system. We describe the details of the metamaterials used to construct the metaimager and their application to metamaterial antennas. The experimental tools needed to characterize the metaimager, including far-field and near-field antenna characterization, are described. We then describe the design, operation, and characterization of a one-dimensional metaimager capable of collecting two-dimensional images, and then a two-dimensional metaimager capable of collecting two-dimensional images. The imaging results for the one-dimensional metaimager are presented including two-dimensional (azimuth and range) images of point scatters, and video-rate imaging. The imaging results for the two-dimensional metaimager are presented including analysis of the system's resolution, signal-to-noise sensitivity, acquisition rate, human targets, and integration of optical and structured-light sensors. Finally, we discuss explorations into methods of tuning metamaterial radiators which could be employed to significantly increase the capabilities of such a metaimaging system, and describe several systems that have been designed for the integration of tuning into metamaterial imaging systems.</p> / Dissertation

Optical coherence tomography (OCT) is a biomedical imaging technique with many current applications. A limitation of the technique is its shallow depth of imaging. A major factor limiting imaging depth in OCT is multiple-scattering of light. This thesis proposes an integrated computational imgaging approach to improve depth of imaging in OCT. In this approach ultrasound patterns are used to modulate the refractive index of tissue. Simulations of the impact of ultrasound on the refractive index are performed, and the results are shown in this thesis. Simulations of the impact of the modulated refractive index on the propagation of light in tissue are needed. But there is no suitable simulator available. Thus, we implemented a Monte Carlo method to solve integral equations that could be used to perform these simulations. Results for integral equations in 1-D and 2-D are shown.

Optical coherence tomography (OCT) is a biomedical imaging technique with many current applications. A limitation of the technique is its shallow depth of imaging. A major factor limiting imaging depth in OCT is multiple-scattering of light. This thesis proposes an integrated computational imgaging approach to improve depth of imaging in OCT. In this approach ultrasound patterns are used to modulate the refractive index of tissue. Simulations of the impact of ultrasound on the refractive index are performed, and the results are shown in this thesis. Simulations of the impact of the modulated refractive index on the propagation of light in tissue are needed. But there is no suitable simulator available. Thus, we implemented a Monte Carlo method to solve integral equations that could be used to perform these simulations. Results for integral equations in 1-D and 2-D are shown.