Videoaufnahmen aus den vergangenen Jahrzehnten stellen kulturelles Erbe dar. Diese sind jedoch nach heutigen Sehgewohnheiten nicht ohne große Einschränkungen für die Wiederausstrahlung geeignet. Das liegt zum einen an längst vergangenen Standards der Videoaufzeichnung, aber zum anderen auch in großem Maße an unkontrolliert gealterten Speichermedien durch inadäquate Aufbewahrung. Oftmals war es lokalen Fernsehsendern technisch nicht möglich ihre Archivbestände unter optimalen klimatischen Bedingungen langlebig zu lagern. Videoarchivdaten nach der Digitalisierung für die Einbindung in heutige Produktionen manuell zu durchsuchen und entsprechend aufzubereiten, ist ein zeitaufwändiger Prozess, den lokale TV-Sender nicht bewältigen können. In unserem Beitrag möchten wir neuartige Methoden der automatisierten Aufbereitung von archivierten VHS-Digitalisaten für die Wiederausstrahlung vorstellen. Dazu zählen vor allem Verfahren zu den Schwerpunkten der Korrektur von Falschfarben (Recoloring) und zur Steigerung der Auflösung von ehemals PAL zu Full-HD und Ultra-HD (Super-Resolution). Zum Einsatz kommen dabei künstliche neuronale Netze, die anders als klassische Verfahren der Bildverarbeitung, semantische Bildkomponenten erfassen und bei der Bearbeitung berücksichtigen können. Mitunter können so deutliche Qualitätsverbesserungen erzielt werden. In unserem Beitrag möchten wir auf Chancen und aktuelle Beschränkungen dieser Technologien eingehen und anhand von digitalisierten Videoarchivdaten deren Funktionsweise demonstrieren.

<p>Applications in materials and biological imaging are currently limited by the ability to collect high-resolution data over large areas in practical amounts of time. One possible solution to this problem is to collect low-resolution data and apply a super-resolution interpolation algorithm to produce a high-resolution image. However, state-of-the-art super-resolution algorithms are typically designed for natural images, require aligned pairing of high and low-resolution training data for optimal performance, and do not directly incorporate a data-fidelity mechanism.</p><p><br></p><p>We present a Multi-Resolution Data Fusion (MDF) algorithm for accurate interpolation of low-resolution SEM and TEM data by factors of 4x and 8x. This MDF interpolation algorithm achieves these high rates of interpolation by first learning an accurate prior model denoiser for the TEM sample from small quantities of unpaired high-resolution data and then balancing this learned denoiser with a novel mismatched proximal map that maintains fidelity to measured data. The method is based on Multi-Agent Consensus Equilibrium (MACE), a generalization of the Plug-and-Play method, and allows for interpolation at arbitrary resolutions without retraining. We present electron microscopy results at 4x and 8x super resolution that exhibit reduced artifacts relative to existing methods while maintaining fidelity to acquired data and accurately resolving sub-pixel-scale features.</p>

Breaking Rayleigh’s "curse" and resolving infinitely small spatial separations is one motivation for developing super-resolution in imaging systems. It has been shown that an arbitrarily small distance between two incoherent point sources can be resolved through the use of a spatial mode sorter, by treating it as a parameter estimation problem. However, when extending this method to general objects with many point sources, the added complexity of multi-parameter estimation problems makes resolution of general objects quite challenging. In the first part of this thesis, we propose a new approach to deal with this problem by generalizing the Richardson-Lucy (RL) deconvolution algorithm to accept the outputs from a mode sorter. We simulate the application of this algorithm to an incoherent confocal microscope using a Zernike spatial mode sorter rather than the conventional pinhole. Our method can resolve general scenes with arbitrary geometry. For such spatially incoherent objects, we find that the resolution enhancement of the sorter-based microscopy using the generalized RL algorithm is over 30% higher than the conventional confocal approach using the standard RL algorithm. This method is quite simple and potentially can be used for various applications including fluorescence microscopy. It could also be combined with other super-resolution techniques for enhanced results. The second part of this thesis explores the potential for the Herriott cell to be used as an image-preserving delay line. In quantum imaging, entangled photons are often utilized to take advantage of their spatial and temporal correlations. One photon (“the signal”) interacts with the system to be measured and the other (“the herald”) is used to trigger the detection of the signal. However, for a typical high-sensitivity camera, there is a delay on the order of 20 ns between the trigger and the sensor becoming active allowing for the signal to be recorded. An image-preserving delay line (IPDL) serves to store a photon without distorting the spatial structure and losing the spatial and temporal correlations. It is commonly made with a series of 4f systems to repeatedly image the light field. We propose to use the Herriott cell as an image-preserving delay line. We tested 10 of the lower-order HG modes and found it was able to preserve almost all of them with high fidelities (>90%), with the only exceptions being the largest modes (HG03 and HG30) at the longest delay (7.9 m) where the fidelity was still >86%. In addition to these modes, it was also able to store general images. This application of the Herriott cell affords insights into miniaturizing IPDLs, which tend to occupy a significant amount of space. Overall, these two projects offer novel insight and application to the world of quantum imaging.

High resolution transient/3D imaging technology is of high interest in both scientific research and commercial application. Nowadays, all of the transient imaging methods suffer from low resolution or time consuming mechanical scanning. We proposed a new method based on TCSPC and Compressive Sensing to achieve a high resolution transient imaging with a several seconds capturing process. Picosecond laser sends a serious of equal interval pulse while synchronized SPAD camera's detecting gate window has a precise phase delay at each cycle. After capturing enough points, we are able to make up a whole signal. By inserting a DMD device into the system, we are able to modulate all the frames of data using binary random patterns to reconstruct a super resolution transient/3D image later. Because the low fill factor of SPAD sensor will make a compressive sensing scenario ill-conditioned, We designed and fabricated a diffractive microlens array. We proposed a new CS reconstruction algorithm which is able to denoise at the same time for the measurements suffering from Poisson noise. Instead of a single SPAD senor, we chose a SPAD array because it can drastically reduce the requirement for the number of measurements and its reconstruction time. Further more, it not easy to reconstruct a high resolution image with only one single sensor while for an array, it just needs to reconstruct small patches and a few measurements. In this thesis, we evaluated the reconstruction methods using both clean measurements and the version corrupted by Poisson noise. The results show how the integration over the layers influence the image quality and our algorithm works well while the measurements suffer from non-trival Poisson noise. It's a breakthrough in the areas of both transient imaging and compressive sensing.

京都大学 / 0048 / 新制・課程博士 / 博士(工学) / 甲第18561号 / 工博第3922号 / 新制||工||1603(附属図書館) / 31461 / 京都大学大学院工学研究科社会基盤工学専攻 / (主査)教授 田村 正行, 准教授 須﨑 純一, 准教授 田中 賢治 / 学位規則第4条第1項該当 / Doctor of Philosophy (Engineering) / Kyoto University / DFAM

No description available.

The development of super-resolution microscopy made it possible to surpass the diffraction limit of optical microscopy, enabling researchers to gain a nanometer scale understanding of cellular structures. While many applications have benefited from standard super-resolution microscopy, gaps remained making high-speed dynamic imaging in live cells impossible. To address this problem, single-point edge-excitation sub-diffraction (SPEED) microscopy was developed. This methodology enables the nanometer imaging of dynamic cell processes within live cells, the evaluation of subcellular structural information, the capacity to derive three-dimensional information from two-dimensional images within rotationally symmetric structures, and the interrogation of novel questions regarding the transport dynamics of macromolecules in a variety of cellular structures. Here, I have described the theory and method behind the current iteration of SPEED microscopy that we have developed and validated via Monte Carlo simulation. Further, a detailed description of how we have further developed SPEED microscopy to derive structural information within the nuclear pore complex as well as how SPEED has been applied to evaluate the export kinetics of mRNA. / Biology

4D Flow Magnet Resonance Imaging (4D Flow MRI) is a novel non-invasive technique for imaging of cardiovascular blood flow. However, when utilized as a stand-alone analysis method, 4D Flow MRI has certain limitations including limited spatial resolution and noise artefacts, motivating the application of dedicated post-processing tools. Learning based super-resolution (SR) has here emerged as a promising utility for such work, however, more often than not, these efforts have been constricted to a narrowly defined cardiovascular domain. Rather, there has been limited exploration of how learned super-resolution models perform across \emph{multiple} cardiovascular compartments, with the wide range of hemodynamic compartments covered by the cardiovasculature as an apparent challenge. To address this, we investigate the generalizability of 4D Flow MRI super-resolution using ensemble learning. Our investigation employs ensemble learning techniques, specifically bagging and stacking, with a convolutional neural network (4DFlowNet) serving as the framework for all base learners. To assist in training, synthetic training data was extracted from patient-specific, physics-based velocity fields derived from computational fluid dynamic (CFD) simulations conducted in three key compartments: the aorta, brain and the heart. Varying base and ensemble networks were then trained on pairs of high-resolution and low-resolution synthetic data, with performance quantitatively assessed as a function of cardiovascular domain, and specific architecture. To ensure clinical relevance, we also evaluated model performance on clinically acquired MRI data from the very same three compartments.  We find that ensemble models improve performance, as compared to isolated equivalents. Our ensemble model \textit{Stacking Block-3}, improves in-silico error rate by $16.22$ points across the average domain. Additionally, performance on the aorta, brain and heart improves by $2.66$, $5.81$ and $2.00$ points respectively. Employing both qualitative and quantitative evaluation methods on the in-vivo data, we find that ensemble models produce super-resolved velocity fields that are quantitatively coherent with ground truth reference data and visually pleasing. To conclude, ensemble learning has shown potential in generalizing 4D Flow MRI across multiple cardiovascular compartments.

The nuclear pore complex (NPC) is a proteinaceous gateway embedded in the nuclear envelope (NE) that regulates nucleocytoplasmic transport of molecules in eukaryotes. The NPC is formed by hundreds of proteins that are classified into approximately thirty different types of proteins called nucleoporin (Nup), each presents in multiples of eight copies. These nucleoporins are divided into two categories: the scaffold Nups forming the main structure of the NPC and the phenylalanine-glycine (FG) Nups that contain multiple repeats of intrinsically disordered and hydrophobic FG domains. These FG-Nups constitute the selective permeability barrier in the central channel of the NPC, which mediates the nuclear import of proteins into the nucleus, and the nuclear export of mRNA and pre-ribosomal subunits out of the nucleus. However, the precise copies of these Nups and their specific roles in the nucleocytoplasmic transport mechanism remain largely unknown. Moreover, the dysfunctional nuclear transport and the mutations of Nups have been closely associated with numerous human diseases, such as cancer, tumor and liver cirrhosis. We have developed and employed live-cell high-speed single-molecule microscopy to elucidate these critical questions remained in the nuclear transport and provide the fundamental knowledge for developing therapies. In this dissertation, I will present my major findings for the following three research projects: 1) determine the dynamic components of FG-Nups in native NPCs; 2) track the nucleocytoplasmic transport of transcription factor Smad proteins under ligand-activated conditions; and 3) elucidate the relationship between the nuclear export of mRNA and the presence and absence of specific Nups in live cells.Determination of the dynamic components for FG-Nups in native NPCs. Scaffold Nups have been intensively studied with electron microscopy to reveal their spatial positions and architecture in the past decades. However, the spatial organization of FG-Nups remains obscure due to the challenge of probing these disordered and dynamic polypeptides in live NPCs. By employing high-speed single-molecule microscopy and a live cell HaloTag labeling technique, I have mapped the spatial distribution for all eleven known mammalian FG-Nups within individual NPCs. Results show that all FG-Nups within NPCs are distinct in conformations and organized to form a ~300nm long hourglass shaped toroidal channel through the nuclear envelope. Exceptionally, the two remaining Nups (Nup98 and hCG1) almost extend through the entire NPC and largely overlap with all other FG-Nups in their spatial distributions. These results provide a complete map of FG-Nup organization within the NPC and also offer structural and functional insights into nucleocytoplasmic transport models. Tracking of the nucleocytoplasmic transport of Smad proteins under ligand-activated conditions. The inducement of transforming growth factor β1 (TGF-β1) was reported to cause the nuclear accumulation of Smad2/Smad4 heterocomplexes. However, the relationship between nuclear accumulation and the nucleocytoplasmic transport kinetics of Smad proteins in the presence of TGF-β1 remains obscure. By combining a high-speed single-molecule tracking microscopy technique (FRET), I tracked the entire TGF-β1-induced process of Smad2/Smad4 heterocomplex formation, as well as their transport through nuclear pore complex in live cells. The FRET results have revealed that in TGF-β1-treated cells, Smad2/Smad4 heterocomplexes formed in the cytoplasm, imported through the nuclear pore complexes as entireties, and finally dissociated in the nucleus. Moreover, it was found that basal-state Smad2 or Smad4 cannot accumulate in the nucleus without the presence of TGF-β1, mainly because both of them have an approximately twofold higher nuclear export efficiency compared to their nuclear import. Remarkably and reversely, heterocomplexes of Smad2/Smad4 induced by TGF-β1 can rapidly concentrate in the nucleus because of their almost fourfold higher nuclear import rate in comparison with their nuclear export rate. Thus, these single-molecule tracking data elucidate the basic molecular mechanism to understand nuclear transport and accumulation of Smad protein. Elucidation of the relationship between the nuclear export of mRNA and the presence and absence of specific Nups in live cells. In addition to explore the dynamic organization of NPC, in vivo characterization of the exact copy number and the specific function of each nucleoporin in the nuclear pore complex (NPC) remains desirable and challenging. Using live-cell high-speed super-resolution single-molecule microscopy, we first quantify the native copies of nuclear basket FG-Nups (Nup153, Nup50 and Tpr). Second, with same imaging technique and the auxin-inducible degradation strategies, I track the nuclear export of mRNA through native NPCs in absence of these FG-Nups. I found that these FG-Nups proteins possess the stoichiometric ratio of 1:1:1 and play distinct roles in the nuclear export of mRNAs in live cells. Tpr’s absence in the NPC dominantly reduces nuclear mRNA’s probability of entering the NPC for export. Complete depletion of Nup153 causes mRNA’s successful nuclear export efficiency dropped approximately four folds. Remarkably, the relationship between mRNA’s successful export efficiency and the copy number of Nup153 is not linear but instead follows a sigmoid function, in which mRNA can gain its maximum successful export efficiency as Nup153 increased from zero to around half of their full copies in the NPC. Lastly, the absence of Tpr or Nup153 also alters mRNA’s export routes through the NPC, but the removal of only Nup50 has almost no impact upon mRNA export route and kinetics. / Biology

A process for the use of multihypothesis prediction in the reconstruction of images is proposed for use in both compressed-sensing reconstruction as well as single-image super-resolution. Specifically, for compressed-sensing reconstruction of a single still image, multiple predictions for an image block are drawn from spatially surrounding blocks within an initial non-predicted reconstruction. The predictions are used to generate a residual in the domain of the compressed-sensing random projections. This residual being typically more compressible than the original signal leads to improved compressed-sensing reconstruction quality. To appropriately weight the hypothesis predictions, a Tikhonov regularization to an ill-posed least-squares optimization is proposed. An extension of this framework is applied to the compressed-sensing reconstruction of hyperspectral imagery is also studied. Finally, the multihypothesis paradigm is employed for single-image superresolution wherein each patch of a low-resolution image is represented as a linear combination of spatially surrounding hypothesis patches.