Dynamics between organic molecules and surface enhanced Raman scattering (SERS) hot spots are extracted from far-field optical images by two experimental methods presented in this thesis: orientational imaging and super-resolution optical imaging. We introduce SERS orientational imaging as an all-optical technique able to determine the three-dimensional orientations of SERS-active Ag nanoparticle dimers. This is accomplished by observing lobe positions in SERS emission patterns formed by the directional polarization of SERS emission along the longitudinal axis of the dimer. We further extend this technique to discriminate nanoparticle dimers from higher order aggregates by observing the wavelength-dependence of SERS emission patterns, which are unchanged in nanoparticle dimers, but show differences in higher order aggregates involving two or more nanoparticle junctions. Dynamic fluctuations in the SERS emission pattern lobes are observed in aggregates labeled with low dye concentrations, as molecules diffuse into regions of higher electromagnetic enhancement in multiple nanoparticle junctions. In order to investigate these dynamic interactions between single organic molecules and nanoparticle hot spots we present the first super-resolution optical images of single-molecule SERS (SM-SERS), introducing super-resolution imaging as a powerful new tool for SM-SERS studies. Mapping the dynamic movement of SM-SERS centroid positions with +/- 5 nm resolution reveals the position-dependent SERS intensity as the centroid samples different positions in space. We have proposed that the diffusion of the SERS centroid is due to diffusion of a single molecule on the surface of the nanoparticle, which leads to changes in coupling between the scattering dipole and the optical near field of the nanoparticle. Finally, we combine an isotope-edited bi-analyte SERS spectral approach with super-resolution optical imaging and atomic force microscopy (AFM) structural analysis for a more complete picture of molecular dynamics in SERS hot spots. We demonstrate the ability to observe multiple molecule dynamics in a single hot spot and show that in addition to the single-molecule regime, a "few" molecule regime is able to report on position-dependent SERS intensities in a hot spot. Furthermore, we are able to identify multiple local hot spots in single nanoparticle aggregates. / text

In recent years, localization-based super-resolution imaging has been developed to overcome the diffraction limit of far-field fluorescence microscopy. Photoswitchable probes are a hallmark of this technique. Their fluorescence can be modulated between an emissive and dark state whereby the sequential, nanoscale measurement of individual fluorophore positions can be used to reconstruct an image at higher spatial resolution. Despite the importance of photoswitchable probes for localization-based super-resolution imaging, both a mechanistic and quantitative understanding of the essential photoswitching properties is lacking for most fluorophores. In this thesis, we begin to address this need. Furthermore, we demonstrate the development of new probes and methodologies for both multicolor and live-cell super-resolution imaging. Chapter 2 describes our mechanistic insights into the photoswitching of a common class of dyes called carbocyanines. Red carbocyanines, such as Cy5, enter a long-lived dark state upon illumination with red light in the presence of a primary thiol. We show that the dark state is a covalent conjugate between the thiol and dye and that this dark state recovers by illumination with ultraviolet light. We also speculate on possible reactivation mechanisms. Our mechanistic studies may ultimately lead to the creation of new probes with improved photoswitching properties. Chapter 3 details our quantitative characterization of the photoswitching properties of 26 organic dyes, including carbocyanines and several other structural classes. We define the essential properties of photoswitchable probes, including photons per switching event, on/off duty cycle, photostability, and number of switching cycles, and demonstrate how these properties dictate super-resolution image quality. This rigorous evaluation will enable more effective use of probes. In Chapters 4 and 5, we focus on expanding the super-resolution toolbox with novel strategies for multicolor and live-cell imaging. Chapter 4 discusses two approaches we have developed for multicolor super-resolution imaging, which distinguish probes based on either the color of activation or emission light. These tools allow multiple cellular targets to be resolved with high spatial resolution. Lastly, Chapter 5 introduces a method for targeted cellular labeling with photoswitchable probes using a small peptide tag, as well as a new sulfonate-protection strategy for intracellular delivery of high performing photoswitchable dyes.

The main purpose of the project was to create an initial lab set-up for a dig-ital in-line holographic microscope and a reconstruction algorithm. Different parameters including: light source, pin-hole size and distances pinhole-object and object-camera had to be optimized. The lab set-up is to be developed further by a master student at the University of Nairobi and then be used for malaria detection in blood samples. To acquire good enough resolution for malaria detection it has been found necessary to purchase a gray scale camera with smaller pixel size. Two dierent approaches, in this report called the on-sensor approach and the object-magnication approach, were investigated. A reconstruction algorithm anda phase recovery algorithm was implemented as well as a super resolution algorithm to improve resolution of the holograms. The on-sensor approach proved easier and cheaper to use with approximately the same results as the object-magnication method. Necessary further research and development of experimental set-up was thoroughly discussed. / Projketet har gått ut på att bygga en billigare och enklare metod för att identifiera malaria i blodprover. Malaria är ett stort problem i en mängd områden i världen. Flera av dessa är fattiga och kan i nuläget inte tillhandahålla den här tjänsten till sin befolkning. Förutom att dyr apparatur krävs måste även utbildad personal lägga ner mycket tid för att kolla en stor mängd blodprover för att statistiskt säkerställa om en person har malaria eller inte. Vårt mål var att bygga en labbuppställning för "Digital in line holographic microscopy" och en rekonstruktionsalgoritm som en masterstudent vid Nairobi universitet ska fortsätta utveckla. Vi kom också fram till vilken upplösning som krävdes för att kunna urskilja malaria i blodproverna. Digital in line holographic microscopy går till så att man har en ljuskälla som riktas genom ett pinnhål, ljuset som går genom pinnhålet ljuser upp det prov, blodproverna i vårt fall, man vill undersöka och det resulterande ljuset fångas på en kamera. Med kunskap om fourieroptik går det att rekonstruera den digitala bilden man fångat på kameran, innan rekonstruktion är den ett hologram vilken är svårtydd. Labbuppställningen byggdes delvis med en 3D printer. För att förbättra resultaten implementerades flera algoritmer vilka lade ihop en mängd förskjutna bilder till en bättre bild, så kallad super resolution. Vi lyckades inte komma till den upplösning som krävdes för att urskilja malaria men gjorde en grundlig förstudie och en utförlig beskrivning av det arbete som väntar den student som fortsätter med projektet. Framför allt beskrevs värden på parametrar och vilken typ av kamera som ska användas för att optimera uppställningen.

The optics and photonics development is currently driven towards nanometer scales. However, diffraction imposes challenges for this development because it prevents confinement of light below a physical limit, commonly known as the diffraction limit. Several implications of the diffraction limit include that conventional optical microscopes are unable to resolve objects smaller than 250nm, and photonic circuits have a physical dimension on the order of the wavelength. Metals at optical frequencies display collective electron oscillations when excited by photon energy, giving rise to the surface plasmon modes with subdiffractional modal profile at metal-dielectric interfaces. Therefore, metallo-dielectric structures are promising candidates for alleviating the obstacles due to diffraction. This thesis investigates a particular branch of plasmonic structures, namely plasmonic antennas, for the purpose of optical imaging and sensing applications. Plasmonic antennas are known for their ability of dramatic near-field enhancement, as well as effective coupling of free-space radiation with localized energy. Such properties are demonstrated in this thesis through two particular applications. The first one is to utilize the interference of evanescent waves from an array of antennas to achieve near-field subdiffraction focusing, also known as superfocusing, in both one and two dimensions. Such designs could alleviate the tradeoffs in the current near-field scanning optical microscopy by improving the signal throughput and extending the imaging distance. The second application is to achieve more efficient radiation from single-emitters through coupling to a highly directive leaky-wave antenna. In this case, the leaky-wave antenna demonstrates the ability of enhancing the directivity over a very wide spectrum.

The optics and photonics development is currently driven towards nanometer scales. However, diffraction imposes challenges for this development because it prevents confinement of light below a physical limit, commonly known as the diffraction limit. Several implications of the diffraction limit include that conventional optical microscopes are unable to resolve objects smaller than 250nm, and photonic circuits have a physical dimension on the order of the wavelength. Metals at optical frequencies display collective electron oscillations when excited by photon energy, giving rise to the surface plasmon modes with subdiffractional modal profile at metal-dielectric interfaces. Therefore, metallo-dielectric structures are promising candidates for alleviating the obstacles due to diffraction. This thesis investigates a particular branch of plasmonic structures, namely plasmonic antennas, for the purpose of optical imaging and sensing applications. Plasmonic antennas are known for their ability of dramatic near-field enhancement, as well as effective coupling of free-space radiation with localized energy. Such properties are demonstrated in this thesis through two particular applications. The first one is to utilize the interference of evanescent waves from an array of antennas to achieve near-field subdiffraction focusing, also known as superfocusing, in both one and two dimensions. Such designs could alleviate the tradeoffs in the current near-field scanning optical microscopy by improving the signal throughput and extending the imaging distance. The second application is to achieve more efficient radiation from single-emitters through coupling to a highly directive leaky-wave antenna. In this case, the leaky-wave antenna demonstrates the ability of enhancing the directivity over a very wide spectrum.

abstract: In this work, atomic force microscopy (AFM) and time resolved confocal fluorescence microscopy are combined to create a microscopy technique which allows for nanometer resolution topographic and fluorescence imaging. This technique can be applied to any sample which can be immobilized on a surface and which can be observed by fluorescence microscopy. Biological problems include small molecular systems, such as membrane receptor clusters, where very high optical resolutions need to be achieved. In materials science, fluorescent nanoparticles or other optically active nanostructures can be investigated using this technique. In the past decades, multiple techniques have been developed that yield high resolution optical images. Multiple far-field techniques have overcome the diffraction limit and allow fluorescence imaging with resolutions of few tens of nanometers. On the other hand, near-field microscopy, that makes use of optically active structures much smaller than the diffraction limit can give resolutions around ten nanometers with the possibility to collect topographic information from flat samples. The technique presented in this work reaches resolutions in the nanometer range along with topographic information from the sample. DNA origami with fluorophores attached to it was used to show this high resolution. The fluorophores with 21 nm distance could be resolved and their position on the origami determined within 10 nm. Not only did this work reach a new record in optical resolution in near-field microscopy (5 nm resolution in air and in water), it also gave an insight into the physics that happens between a fluorescent molecule and a dielectric nanostructure, which the AFM tip is. The experiments with silicon tips made a detailed comparison with models possible on the single molecule level, highly resolved in space and time. On the other hand, using silicon nitride and quartz as tip materials showed that effects beyond the established models play a role when the molecule is directly under the AFM tip, where quenching of up to 5 times more efficient than predicted by the model was found. / Dissertation/Thesis / Ph.D. Physics 2012

It is well known that a translating mask can optically encode low-resolution measurements from which higher resolution images can be computationally reconstructed. We experimentally demonstrate that this principle can be used to achieve substantial increase in image resolution compared to the size of the focal plane array (FPA). Specifically, we describe a scalable architecture with a translating mask (also referred to as a coded aperture) that achieves eightfold resolution improvement (or 64: 1 increase in the number of pixels compared to the number of focal plane detector elements). The imaging architecture is described in terms of general design parameters (such as field of view and angular resolution, dimensions of the mask, and the detector and FPA sizes), and some of the underlying design trades are discussed. Experiments conducted with different mask patterns and reconstruction algorithms illustrate how these parameters affect the resolution of the reconstructed image. Initial experimental results also demonstrate that the architecture can directly support task-specific information sensing for detection and tracking, and that moving objects can be reconstructed separately from the stationary background using motion priors. (C) 2017 Society of Photo-Optical Instrumentation Engineers (SPIE)

Realtime Magnetic Resonance Imaging (MRI) is a method used for human anatomical study. MRIs give exceptionally detailed information about soft-tissue structures, such as tongues, that other current imaging techniques cannot achieve. However, the process requires special equipment and is expensive. Hence, it is not quite suitable for all patients. Speech diagrams show the side view positions of organs like the tongue, throat, and lip of a speaking or singing person. The process of making a speech diagram is like the semantic segmentation of an MRI, which focuses on the selected edge structure. Speech diagrams are easy to understand with a clear speech diagram of the tongue and inside mouth structure. However, it often requires manual annotation on the MRI machine by an expert in the field. By using machine learning methods, we achieved transferring images between MRI and speech diagrams in two directions. We first matched videos of speech diagram and tongue MRIs. Then we used various image processing methods and data augmentation methods to make the paired images easy to train. We built our network model inspired by different cross-domain image transfer methods and applied reference-based super-resolution methods—to generate high-resolution images. Thus, we can do the transferring work through our network instead of manually. Also, generated speech diagram can work as an intermediary part to be transferred to other medical images like computerized tomography (CT), since it is simpler in structure compared to an MRI. We conducted experiments using both the data from our database and other MRI video sources. We use multiple methods to do the evaluation and comparisons with several related methods show the superiority of our approach.

Contexte: La microscopie joue un rôle important en biologie depuis plusieurs siècles, mais sa résolution a longtemps été limitée à environ 250 nm, de sorte que nombre de structures biologiques (virus, vésicules, pores nucléaires, synapses) ne pouvaient être résolues. Au cours de la dernière décennie, plusieurs méthodes de super-résolution ont été développées pour dépasser cette limite. Parmi ces techniques, les plus puissantes et les plus utilisées reposent sur la localisation de molécules uniques (microscopie à localisation de molécule unique, ou SMLM), comme PALM et STORM. En localisant précisément les positions de molécules fluorescentes isolées dans des milliers d'images de basse résolution acquises de manière séquentielle, la SMLM peut atteindre des résolutions de 20 à 50 nm voire mieux. Cependant, cette technique est intrinsèquement lente car elle nécessite l’accumulation d’un très grand nombre d’images et de localisations pour obtenir un échantillonnage super-résolutif des structures fluorescentes. Cette lenteur (typiquement ~ 30 minutes par image super-résolutive) rend difficile l'utilisation de la SMLM pour l'imagerie cellulaire à haut débit ou en cellules vivantes. De nombreuses méthodes ont été proposées pour pallier à ce problème, principalement en améliorant les algorithmes de localisation pour localiser des molécules proches, mais la plupart de ces méthodes compromettent la résolution spatiale et entraînent l’apparition d’artefacts. Méthodes et résultats: Nous avons adopté une stratégie de transformation d’image en image basée sur l'apprentissage profond dans le but de restaurer des images SMLM parcimonieuses et par là d’améliorer la vitesse d’acquisition et la qualité des images super-résolutives. Notre méthode, ANNA-PALM, s’appuie sur des développements récents en apprentissage profond, notamment l’architecture U-net et les modèles génératifs antagonistes (GANs). Nous montrons des validations de la méthode sur des images simulées et des images expérimentales de différentes structures cellulaires (microtubules, pores nucléaires et mitochondries). Ces résultats montrent qu’après un apprentissage sur moins de 10 images de haute qualité, ANNA-PALM permet de réduire le temps d’acquisition d’images SMLM, à qualité comparable, d’un facteur 10 à 100. Nous avons également montré que ANNA-PALM est robuste à des altérations de la structure biologique, ainsi qu’à des changements de paramètres de microscopie. Nous démontrons le potentiel applicatif d’ANNA-PALM pour la microscopie à haut débit en imageant ~ 1000 cellules à haute résolution en environ 3 heures. Enfin, nous avons conçu un outil pour estimer et réduire les artefacts de reconstruction en mesurant la cohérence entre l’image reconstruite et l’image en épi-fluorescence. Notre méthode permet une microscopie super-résolutive plus rapide et plus douce, compatible avec l’imagerie haut débit, et ouvre une nouvelle voie vers l'imagerie super-résolutive des cellules vivantes. La performance des méthodes d'apprentissage profond augmente avec la quantité des données d’entraînement. Le partage d’images au sein de la communauté de microscopie offre en principe un moyen peu coûteux d’augmenter ces données. Cependant, il est souvent difficile d'échanger ou de partager des données de SMLM, car les tables de localisation seules ont souvent une taille de plusieurs gigaoctets et il n'existe pas de plate-forme de visualisation dédiée aux données SMLM. Nous avons développé un format de fichier pour compresser sans perte des tables de localisation, ainsi qu’une plateforme web (https://shareloc.xyz) qui permet de visualiser et de partager facilement des données SMLM 2D ou 3D. A l’avenir, cette plate-forme pourrait grandement améliorer les performances des modèles d'apprentissage en profondeur, accélérer le développement des outils, faciliter la réanalyse des données et promouvoir la recherche reproductible et la science ouverte. / Background: Microscopy plays an important role in biology since several centuries, but its resolution has long been limited to ~250nm due to diffraction, leaving many important biological structures (e.g. viruses, vesicles, nuclear pores, synapses) unresolved. Over the last decade, several super-resolution methods have been developed that break this limit. Among the most powerful and popular super-resolution techniques are those based on single molecular localization (single molecule localization microscopy, or SMLM) such as PALM and STORM. By precisely localizing positions of isolated fluorescent molecules in thousands or more sequentially acquired diffraction limited images, SMLM can achieve resolutions of 20-50 nm or better. However, SMLM is inherently slow due to the necessity to accumulate enough localizations to achieve high resolution sampling of the fluorescent structures. The drawback in acquisition speed (typically ~30 minutes per super-resolution image) makes it difficult to use SMLM in high-throughput and live cell imaging. Many methods have been proposed to address this issue, mostly by improving the localization algorithms to localize overlapping spots, but most of them compromise spatial resolution and cause artifacts.Methods and results: In this work, we applied deep learning based image-to-image translation framework for improving imaging speed and quality by restoring information from rapidly acquired low quality SMLM images. By utilizing recent advances in deep learning including the U-net and Generative Adversarial Networks, we developed our method Artificial Neural Network Accelerated PALM (ANNA-PALM) which is capable of learning structural information from training images and using the trained model to accelerate SMLM imaging by tens to hundreds folds. With experimentally acquired images of different cellular structures (microtubules, nuclear pores and mitochondria), we demonstrated that deep learning can efficiently capture the structural information from less than 10 training samples and reconstruct high quality super-resolution images from sparse, noisy SMLM images obtained with much shorter acquisitions than usual for SMLM. We also showed that ANNA-PALM is robust to possible variations between training and testing conditions, due either to changes in the biological structure or to changes in imaging parameters. Furthermore, we take advantage of the acceleration provided by ANNA-PALM to perform high throughput experiments, showing acquisition of ~1000 cells at high resolution in ~3 hours. Additionally, we designed a tool to estimate and reduce possible artifacts is designed by measuring the consistency between the reconstructed image and the experimental wide-field image. Our method enables faster and gentler imaging which can be applied to high-throughput, and provides a novel avenue towards live cell high resolution imaging. Deep learning methods rely on training data and their performance can be improved even further with more training data. One cheap way to obtain more training data is through data sharing within the microscopy community. However, it often difficult to exchange or share localization microscopy data, because localization tables alone are typically several gigabytes in size, and there is no dedicated platform for localization microscopy data which provide features such as rendering, visualization and filtering. To address these issues, we developed a file format that can losslessly compress localization tables into smaller files, alongside with a web platform called ShareLoc (https://shareloc.xyz) that allows to easily visualize and share 2D or 3D SMLM data. We believe that this platform can greatly improve the performance of deep learning models, accelerate tool development, facilitate data re-analysis and further promote reproducible research and open science.

Automatic target recognition (ATR) is the ability for a computer to discriminate between different objects in a scene. ATR is often performed on point cloud data from a sensor known as a Ladar. Increasing the resolution of this point cloud in order to get a more clear view of the object in a scene would be of significant interest in an ATR application. A technique to increase the resolution of a scene is known as super resolution. This technique requires many low resolution images that can be combined together. In recent years, however, it has become possible to perform super resolution on a single image. This thesis sought to apply Gabor Wavelets and Compressive Sensing to single image super resolution of digital images of natural scenes. The technique applied to images was then extended to allow the super resolution of a point cloud.