The aim of the work described in this thesis is to improve the understanding, implementation, and overall capabilities of structured illumination microscopy (SIM). SIM is a superresolution technique that excels in gentle live-cell volumetric imaging tasks. Many modalities of SIM were developed over the last decade that tailored SIM into the versatile and powerful technique that it is today. Nevertheless, the field of SIM continues to evolve and there is plenty of room for novel concepts. Specifically, in this thesis, a generalised framework for a theoretical description of SIM variants is introduced, the constraints of optical components for a flexible SIM system are discussed and the set-up is realised, the important aspect of deconvolution in SIM is highlighted and further developed, and finally novel SIM modalities introduced that improve its time-resolution, gentleness, and volumetric imaging capabilities. Based on the generalised theory, the computational steps for the extraction of superresolution information from SIM raw data are outlined and the essential concept of spatial frequency un-mixing explained for standard SIM as well as for multifocal SIM. Multifocal SIM hereby acts as a parallelised confocal as well as widefield technique and thus serves as link between the two modalities. Using this novel scheme deconvolution methods for SIM are then further developed to allow a holistic reconstruction procedure. Deconvolution is of great important in the SIM reconstruction process, and hence rigorous derivations of advanced deconvolution methods are provided and further developed to enable generalised ‘multi-image’ Richardson-Lucy deconvolution in SIM, called joint Richardson-Lucy deconvolution (jRL). This approach is demonstrated to robustly produce optically sectioned multifocal SIM images and, through the incorporation of a 3D imaging model, also volumetric standard SIM images within the jRL framework. For standard SIM this approach enabled acquisition speed doubling, because the recovery of superresolved images from a reduced number of raw frames through constrained jRL was made possible. The method is validated in silico and in vitro. For the study of yet faster moving samples deconvolution microscopy is found to be the method of choice. To enable optical sectioning, a key feature of SIM, in deconvolution microscopy, a new modality of optical sectioning microscopy is introduced that can be implemented as a single-shot technique. Via polarised excitation and detection in orthogonal directions in conjunction with structured illumination the theoretical framework is rigorously derived and validated.

<p>Optical microscopy plays a crucial role in the biological sciences for its ability to enable visualization of biological samples at sub-cellular levels. Many imaging subdivisions exist under this umbrella of general microscopy, and each are tailored towards specific design, contrast, and visualization constraints. Standard examples that have found widespread use include dark-field, phase-contrast, holographic, and fluorescent microscopies. However, a critical factor that physically limits the optical resolution of general microscopy is diffraction. Unfortunately, this “diffraction-limit” can prevent visualization of significant biologically relevant structures, which in turn can limit biological insights. In response to such a limit, several works have advanced the field of sub-diffraction resolution imaging, which consist of optical imaging techniques that seek to achieve imaging resolutions beyond that which is allowed by the diffraction-limit. This set of techniques can largely be divided into two classes. The first class of sub-diffraction techniques is targeted towards cases where the sample is coherently illuminated and diffracts into the imaging system’s aperture. For such cases, synthetic aperture (SA) is a popular choice and operates by using oblique illuminations to spatiotemporally synthesize a wider frequency support into the image than allowed by the diffraction limit. The second class of sub-diffraction techniques, often referred to as "super-resolution" techniques, typically utilize specialized fluorophores with either photoswitching or depletion capabilities. Photoactivated localization microscopy (PALM) is a super-resolution example that localizes photoswitchable fluorophores to sub-diffraction resolutions per acquisition, before combining into a final super-resolved image. Stimulated emission depletion (STED) is another super-resolution example that spatially modulates its excitation to narrow its optical point-spread-function. Unfortunately, SA and fluorescent super-resolution techniques are generally incompatible for sub-diffraction resolution fluorescent and coherent imaging, respectively – thus, a multimodal sub-diffraction imaging solution compatible with both coherent and fluorescent imaging has remained elusive. </p><p> In this dissertation, we demonstrate that structured illumination (SI) is a sub-diffraction technique compatible with both diffractive and fluorescent imaging. We first develop the theoretical framework that extends SI to coherent imaging and experimentally demonstrate SI’s capabilities for 2D sub-diffraction resolution imaging of coherently diffractive samples. Sub-diffraction resolution imaging based on scattering intensity and transmission-based quantitative-phase (QP) are shown. In addition, we show extend SI to 3D coherent imaging, and show applications of this towards 3D QP and refractive-index (RI) tomography. Finally, we show multimodal applications of SI that allow sub-diffraction resolution fluorescent and coherent imaging, which has great potential utility for the biological sciences.</p> / Dissertation

Thesis (M.S.)--Miami University, Dept. of Physics, 2009. / Title from first page of PDF document. Includes bibliographical references (p. 57-61).

Récemment, de nombreuses techniques de microscopie de fluorescence de super-résolution ont été développées pour permettre d'observer de nombreuses structures biologiques au-delà de la limite de diffraction. La microscopie d'illumination structurée (SIM) est l'une de ces technologies. Le principe de la SIM est basé sur l'utilisation d'une grille de lumière harmonique qui permet de translater les hautes fréquences spatiales de l'échantillon vers la région d’observation du microscope. Les méthodes classiques de reconstruction SIM nécessitent une connaissance parfaite de l'illumination de l’échantillon. Cependant, l’implémentation d’un contrôle parfait de l’illumination harmonique sur le plan de l'échantillon n'est pas facile expérimentalement et il présente un grand défi. L’hypothèse de la connaissance parfaite de l’intensité de la lumière illuminant l’échantillon en SIM peut donc introduire des artefacts sur l’image reconstruite de l'échantillon, à cause des erreurs d’alignement de la grille qui peuvent se présenter lors de l’acquisition expérimentale. Afin de surmonter ce défi, nous avons développé dans cette thèse des stratégies de reconstruction «aveugle» qui sont indépendantes de d'illumination. À l'aide de ces stratégies de reconstruction dites «blind-SIM», nous avons étendu la SIM harmonique pour l’appliquer aux cas de «SIM-speckle» qui utilisent des illuminations aléatoires et inconnues qui contrairement à l’illumination harmonique, ne nécessitent pas de contrôle. / Recently, many superresolution fluorescence microscopy techniques have been developed which allow the observation of many biological structures beyond the diffraction limit. Structured illumination microscopy (SIM) is one of them. The principle of SIM is based on using a harmonic light grid which downmodulates the high spatial frequencies of the sample into the observable region of the microscope. The resolution enhancement is highly dependent on the reconstruction technique, which restores the high spatial frequencies of the sample to their original position. Common SIM reconstructions require the perfect knowledge of the illumination pattern. However, to perfectly control the harmonic illumination patterns on the sample plane is not easy in experimental implementations and this makes the experimental setup very technical. Reconstructing SIM images assuming the perfect knowledge of the illumination intensity patterns may, therefore, introduce artifacts on the estimated sample due to the misalignment of the grid that can occur during experimental acquisitions. To tackle this drawback of SIM, in this thesis, we have developed blind-SIM reconstruction strategies which are independent of the illumination patterns. Using the 3D blind-SIM reconstruction strategies we extended the harmonic SIM to speckle illumination microscopy which uses random unknown speckle patterns that need no control, unlike the harmonic grid patterns.

No description available.

Quantitative phase Imaging(QPI) has emerged as a valuable tool for imaging specimens with weak scattering and absorbing abilities such as cells and tissues. It is complementary to fluorescence microscopy, as such, it can be applied to unlabelled specimens without the need for fluorescent tagging. By quantitatively mapping the phase changes induced in the incident light field by the optical path length delays of the specimen, QPI provides objective measurement of the cellular dynamics and enables imaging the specimen with high contrast. Transport of Intensity Equation(TIE) is a powerful computational tool for QPI because of its experimental and computational simplicity. Using TIE, the phase can be quantitatively retrieved from defocused intensity images. However, the resolution of the phase image computed using TIE is limited by the diffraction limit of the imaging system used to capture the intensity images. In this thesis, we have developed a super resolution phase imaging technique by applying the principles of Structured Illumination Microscopy(SIM) to Transport of Intensity phase retrieval. The modulation from the illumination shifts the high frequency components of the phase object into the system pass-band. This enables phase imaging with resolutions exceeding the diffraction limit. The proposed method is experimentally validated using a custom-made upright microscope. Because of its experimental and computational simplicity, the method in this thesis should find application in biomedical laboratories where super resolution phase imaging is required / Master of Science / Transport of Intensity Equation is a quantitative phase microscopy technique that enables imaging thin transparent specimens with high phase contrast using a through focus intensity stack. It provides speckle free imaging, compatibility with bright field microscopes and valid under partial coherence. However, the Optical Transfer Function(OTF) of the imaging system or the microscope acts a low pass filter, effectively limiting the maximum spatial frequency that can pass through the system. This reduces the spatial resolution of the computed phase image to the spatial diffraction limit. There has been a continuous drive to develop Super resolution techniques that will provide sub-diffraction resolutions because it will provide better insight into the cellular structure, morphology and composition. Structured Illumination Microscopy(SIM) is one such established technique. Existing work in super resolution phase imaging using SIM is exclusively limited to holography and interferometry based techniques. However, such methods require two-beam interference, illumination sources with high coherence, high experimental stability and phase unwrapping in the postprocessing step to retrieve the true object phase. In this work, we demonstrate a single beam propagation based super resolution phase imaging technique by applying structured illumination to Transport of Intensity Equation. It is valid under partial coherence, and does not require interference, simplifying the experimental and computational requirement. We have designed an upright microscope to demonstrate high resolution phase imaging of human cheek cells.

<p>Histopathology is the clinical standard for tissue diagnosis. However, histopathology has several limitations including that it requires tissue processing, which can take 30 minutes or more, and requires a highly trained pathologist to diagnose the tissue. Additionally, the diagnosis is qualitative, and the lack of quantitation leads to possible observer-specific diagnosis. Taken together, it is difficult to diagnose tissue at the point of care using histopathology.</p><p>Several clinical situations could benefit from more rapid and automated histological processing, which could reduce the time and the number of steps required between obtaining a fresh tissue specimen and rendering a diagnosis. For example, there is need for rapid detection of residual cancer on the surface of tumor resection specimens during excisional surgeries, which is known as intraoperative tumor margin assessment. Additionally, rapid assessment of biopsy specimens at the point-of-care could enable clinicians to confirm that a suspicious lesion is successfully sampled, thus preventing an unnecessary repeat biopsy procedure. Rapid and low cost histological processing could also be potentially useful in settings lacking the human resources and equipment necessary to perform standard histologic assessment. Lastly, automated interpretation of tissue samples could potentially reduce inter-observer error, particularly in the diagnosis of borderline lesions. </p><p>To address these needs, high quality microscopic images of the tissue must be obtained in rapid timeframes, in order for a pathologic assessment to be useful for guiding the intervention. Optical microscopy is a powerful technique to obtain high-resolution images of tissue morphology in real-time at the point of care, without the need for tissue processing. In particular, a number of groups have combined fluorescence microscopy with vital fluorescent stains to visualize micro-anatomical features of thick (i.e. unsectioned or unprocessed) tissue. However, robust methods for segmentation and quantitative analysis of heterogeneous images are essential to enable automated diagnosis. Thus, the goal of this work was to obtain high resolution imaging of tissue morphology through employing fluorescence microscopy and vital fluorescent stains and to develop a quantitative strategy to segment and quantify tissue features in heterogeneous images, such as nuclei and the surrounding stroma, which will enable automated diagnosis of thick tissues.</p><p>To achieve these goals, three specific aims were proposed. The first aim was to develop an image processing method that can differentiate nuclei from background tissue heterogeneity and enable automated diagnosis of thick tissue at the point of care. A computational technique called sparse component analysis (SCA) was adapted to isolate features of interest, such as nuclei, from the background. SCA has been used previously in the image processing community for image compression, enhancement, and restoration, but has never been applied to separate distinct tissue types in a heterogeneous image. In combination with a high resolution fluorescence microendoscope (HRME) and a contrast agent acriflavine, the utility of this technique was demonstrated through imaging preclinical sarcoma tumor margins. Acriflavine localizes to the nuclei of cells where it reversibly associates with RNA and DNA. Additionally, acriflavine shows some affinity for collagen and muscle. SCA was adapted to isolate acriflavine positive features or APFs (which correspond to RNA and DNA) from background tissue heterogeneity. The circle transform (CT) was applied to the SCA output to quantify the size and density of overlapping APFs. The sensitivity of the SCA+CT approach to variations in APF size, density and background heterogeneity was demonstrated through simulations. Specifically, SCA+CT achieved the lowest errors for higher contrast ratios and larger APF sizes. When applied to tissue images of excised sarcoma margins, SCA+CT correctly isolated APFs and showed consistently increased density in tumor and tumor + muscle images compared to images containing muscle. Next, variables were quantified from images of resected primary sarcomas and used to optimize a multivariate model. The sensitivity and specificity for differentiating positive from negative ex vivo resected tumor margins was 82% and 75%. The utility of this approach was further tested by imaging the in vivo tumor cavities from 34 mice after resection of a sarcoma with local recurrence as a bench mark. When applied prospectively to images from the tumor cavity, the sensitivity and specificity for differentiating local recurrence was 78% and 82%. The results indicate that SCA+CT can accurately delineate APFs in heterogeneous tissue, which is essential to enable automated and rapid surveillance of tissue pathology. </p><p>Two primary challenges were identified in the work in aim 1. First, while SCA can be used to isolate features, such as APFs, from heterogeneous images, its performance is limited by the contrast between APFs and the background. Second, while it is feasible to create mosaics by scanning a sarcoma tumor bed in a mouse, which is on the order of 3-7 mm in any one dimension, it is not feasible to evaluate an entire human surgical margin. Thus, improvements to the microscopic imaging system were made to (1) improve image contrast through rejecting out-of-focus background fluorescence and to (2) increase the field of view (FOV) while maintaining the sub-cellular resolution needed for delineation of nuclei. To address these challenges, a technique called structured illumination microscopy (SIM) was employed in which the entire FOV is illuminated with a defined spatial pattern rather than scanning a focal spot, such as in confocal microscopy. </p><p>Thus, the second aim was to improve image contrast and increase the FOV through employing wide-field, non-contact structured illumination microscopy and optimize the segmentation algorithm for new imaging modality. Both image contrast and FOV were increased through the development of a wide-field fluorescence SIM system. Clear improvement in image contrast was seen in structured illumination images compared to uniform illumination images. Additionally, the FOV is over 13X larger than the fluorescence microendoscope used in aim 1. Initial segmentation results of SIM images revealed that SCA is unable to segment large numbers of APFs in the tumor images. Because the FOV of the SIM system is over 13X larger than the FOV of the fluorescence microendoscope, dense collections of APFs commonly seen in tumor images could no longer be sparsely represented, and the fundamental sparsity assumption associated with SCA was no longer met. Thus, an algorithm called maximally stable extremal regions (MSER) was investigated as an alternative approach for APF segmentation in SIM images. MSER was able to accurately segment large numbers of APFs in SIM images of tumor tissue. In addition to optimizing MSER for SIM image segmentation, an optimal frequency of the illumination pattern used in SIM was carefully selected because the image signal to noise ratio (SNR) is dependent on the grid frequency. A grid frequency of 31.7 mm-1 led to the highest SNR and lowest percent error associated with MSER segmentation. </p><p>Once MSER was optimized for SIM image segmentation and the optimal grid frequency was selected, a quantitative model was developed to diagnose mouse sarcoma tumor margins that were imaged ex vivo with SIM. Tumor margins were stained with acridine orange (AO) in aim 2 because AO was found to stain the sarcoma tissue more brightly than acriflavine. Both acriflavine and AO are intravital dyes, which have been shown to stain nuclei, skeletal muscle, and collagenous stroma. A tissue-type classification model was developed to differentiate localized regions (75x75 µm) of tumor from skeletal muscle and adipose tissue based on the MSER segmentation output. Specifically, a logistic regression model was used to classify each localized region. The logistic regression model yielded an output in terms of probability (0-100%) that tumor was located within each 75x75 µm region. The model performance was tested using a receiver operator characteristic (ROC) curve analysis that revealed 77% sensitivity and 81% specificity. For margin classification, the whole margin image was divided into localized regions and this tissue-type classification model was applied. In a subset of 6 margins (3 negative, 3 positive), it was shown that with a tumor probability threshold of 50%, 8% of all regions from negative margins exceeded this threshold, while over 17% of all regions exceeded the threshold in the positive margins. Thus, 8% of regions in negative margins were considered false positives. These false positive regions are likely due to the high density of APFs present in normal tissues, which clearly demonstrates a challenge in implementing this automatic algorithm based on AO staining alone. </p><p>Thus, the third aim was to improve the specificity of the diagnostic model through leveraging other sources of contrast. Modifications were made to the SIM system to enable fluorescence imaging at a variety of wavelengths. Specifically, the SIM system was modified to enabling imaging of red fluorescent protein (RFP) expressing sarcomas, which were used to delineate the location of tumor cells within each image. Initial analysis of AO stained panels confirmed that there was room for improvement in tumor detection, particularly in regards to false positive regions that were negative for RFP. One approach for improving the specificity of the diagnostic model was to investigate using a fluorophore that was more specific to staining tumor. Specifically, tetracycline was selected because it appeared to specifically stain freshly excised tumor tissue in a matter of minutes, and was non-toxic and stable in solution. Results indicated that tetracycline staining has promise for increasing the specificity of tumor detection in SIM images of a preclinical sarcoma model and further investigation is warranted. </p><p>In conclusion, this work presents the development of a combination of tools that is capable of automated segmentation and quantification of micro-anatomical images of thick tissue. When compared to the fluorescence microendoscope, wide-field multispectral fluorescence SIM imaging provided improved image contrast, a larger FOV with comparable resolution, and the ability to image a variety of fluorophores. MSER was an appropriate and rapid approach to segment dense collections of APFs from wide-field SIM images. Variables that reflect the morphology of the tissue, such as the density, size, and shape of nuclei and nucleoli, can be used to automatically diagnose SIM images. The clinical utility of SIM imaging and MSER segmentation to detect microscopic residual disease has been demonstrated by imaging excised preclinical sarcoma margins. Ultimately, this work demonstrates that fluorescence imaging of tissue micro-anatomy combined with a specialized algorithm for delineation and quantification of features is a means for rapid, non-destructive and automated detection of microscopic disease, which could improve cancer management in a variety of clinical scenarios.</p> / Dissertation

Final accepted manuscripts of parts 4 and 5 from Roadmap on Structured Light, authored by Masud Mansuripur, College of Optical Sciences, The University of Arizona.

The understanding of the biological processes at the cellular and subcellular level requires the ability to directly visualize them. Fluorescence microscopy played a key role in biomedical imaging because of its high sensitivity and specificity. However, traditional fluorescence microscopy has a limited resolution due to optical diffraction. In recent years, various approaches have been developed to overcome the diffraction limit. Among these techniques, nonlinear structured illumination microscopy (SIM) has been demonstrated a fast and full field superresolution imaging tool, such as Saturated-SIM and Photoswitching-SIM. In this dissertation, I report a new approach that applies nonlinear structured illumination by combining a uniform excitation field and a patterned stimulated emission depletion (STED) field. The nature of STED effect allows fast switching response, negligible stochastic noise during switching, low shot noise and theoretical unlimited resolution, which predicts STED-SIM to be a better nonlinear SIM. After the algorithm development and the feasibility study by simulation, the STED-SIM microscope was tested on fluorescent beads samples and achieved full field imaging over 1 x 10 micron square at the speed of 2s/frame with 4-fold improved resolution. Our STED-SIM technique has been applied on biological samples and superresolution images with tubulin of U2OS cells and granules of neuron cells have been obtained. In this dissertation, an effort to apply a field enhancement mechanism, surface plasmon resonance (SPR), to nonlinear STED-SIM has been made and around 8 time enhancement on STED quenching effect was achieved. Based on this enhancement on STED, 1D SPR enhanced STED-SIM was built and 50 nm resolution of fluorescence beads sample was achieved. Algorithm improvement is required to achieve full field superresolution imaging with SPR enhanced STED-SIM. The application of nonlinear structured illumination in two photon light-sheet microscopy is also studied in this dissertation. Fluorescent cellular imaging of deep internal organs is highly challenging because of the tissue scattering. By combining two photon Bessel beam light-sheet microscopy and nonlinear SIM, 3D live sample imaging at cellular resolution in depth beyond 200 microns has been achieved on live zebrafish. Two-color imaging of pronephric glomeruli and vasculature of zebrafish kidney, whose cellular structures located at the center of the fish body are revealed in high clarity.

A method of lensless, single pixel imaging is presented. This method, referred to as MAS-IPSII, is theoretically capable of resolutions as small as one quarter of the wavelength of the imaging light. The resolution is not limited by the aperture of any optic, making high resolutions (including subwavelength) feasible even at very large (greater than a meter) distances. Imaging requires only flat optics and a coherent source, making it a good candidate for imaging with extreme wavelengths in the UV and x-ray regimes. The method is demonstrated by the imaging of various test targets. Both real and complex imaging (i.e. holography) is demonstrated.