Computational sensing has demonstrated the ability to ameliorate or eliminate many trade-offs in traditional sensors. Rather than attempting to form a perfect image, then sampling at the Nyquist rate, and reconstructing the signal of interest prior to post-processing, the computational sensor attempts to utilize a priori knowledge, active or passive coding of the signal-of-interest combined with a variety of algorithms to overcome the trade-offs or to improve various task-specific metrics. While it is a powerful approach to radically new sensor architectures, published research tends to focus on architecture concepts and positive results. Little attention is given towards the practical issues when faced with implementing computational sensing prototypes. I will discuss the various practical challenges that I encountered while developing three separate applications of computational sensors. The first is a compressive sensing based object tracking camera, the SCOUT, which exploits the sparsity of motion between consecutive frames while using no moving parts to create a psuedo-random shift variant point-spread function. The second is a spectral imaging camera, the AFSSI-C, which uses a modified version of Principal Component Analysis with a Bayesian strategy to adaptively design spectral filters for direct spectral classification using a digital micro-mirror device (DMD) based architecture. The third demonstrates two separate architectures to perform spectral unmixing by using an adaptive algorithm or a hybrid techniques of using Maximum Noise Fraction and random filter selection from a liquid crystal on silicon based computational spectral imager, the LCSI. All of these applications demonstrate a variety of challenges that have been addressed or continue to challenge the computational sensing community. One issue is calibration, since many computational sensors require an inversion step and in the case of compressive sensing, lack of redundancy in the measurement data. Another issue is over multiplexing, as more light is collected per sample, the finite amount of dynamic range and quantization resolution can begin to degrade the recovery of the relevant information. A priori knowledge of the sparsity and or other statistics of the signal or noise is often used by computational sensors to outperform their isomorphic counterparts. This is demonstrated in all three of the sensors I have developed. These challenges and others will be discussed using a case-study approach through these three applications.

Fluorescence microscopy is an indispensable tool for biology to study the spatio-temporal dynamics of cells, tissues, and developing organisms. Modern imaging modalities, such as light-sheet microscopy, are able to acquire large three- dimensional volumes with high spatio-temporal resolution for many hours or days, thereby routinely generating Terabytes of image data in a single experiment. The quality of these images, however, is limited by the optics of the microscope, the signal-to-noise ratio of acquisitions, the photo-toxic effects of illumination, and the distortion of light by the sample. Additionally, the serial operation mode of most microscopy experiments, where large data sets are first acquired and only afterwards inspected and analyzed, excludes the possibility to optimize image quality during acquisition by automatically adapting the microscope parameters. These limits make certain observations difficult or impossible, forcing trade-offs between imaging speed, spatial resolution, light exposure, and imaging depth. This thesis is concerned with addressing several of these challenges with computational methods. First, I present methods for visualizing and processing the volumetric data from a microscope in real-time, i.e. at the acquisition rate of typical experiments, which is a prerequisite for the development of adaptive microscopes. I propose a low-discrepancy sampling strategy that enables the seamless display of large data sets during acquisition, investigate real-time compatible denoising, convolution, and deconvolution methods, and introduce a low-rank decomposition strategy for common deblurring tasks. Secondly, I propose a computational tractable method to simulate the interaction of light with realistically large biological tissues by combining a GPU-accelerated beam propagation method with a novel multiplexing scheme. I demonstrate that this approach enables to rigorously simulate the wave-optical image formation in light-sheet microscopes, to numerically investigate correlative effects in scattering tissues, and to elucidate the optical properties of the inverted mouse retina. Finally, I propose a data-driven restoration approach for fluorescence microscopy images based on convolutional neural networks (Care) that leverages sample and imaging specific prior knowledge. By demonstrating the superiority of this approach when compared to classical methods on a variety of problems, ranging from restoration of high quality images from low signal-to-noise-ratio acquisitions, to projection of noisy developing surface, isotropic recovery from anisotropic volumes, and to the recovery of diffraction-limited structures from widefield images alone, I show that Care is a flexible and general method to solve fundamental restoration problems in fluorescence microscopy.

No description available.