The intricate task of segmenting and quantifying cell nuclear pores in high-resolution 3D microscopy data is critical for cellular biology and disease research. This thesis introduces a deep learning pipeline crafted to automate the segmentation and quantification of nuclear pores from high-resolution 3D cell organelle images. Our aim is to refine computational methods capable of handling the data's complexity and size, thus improving accuracy and reducing manual labor in biological image analysis. The developed pipeline incorporates data preprocessing, augmentation strategies, random block sampling, and a three-stage post-processing algorithm. It utilizes a 3D U-Net with a VGG-16 backbone, optimized through cyclical data augmentation and random block sampling to tackle the challenges posed by limited labeled data and the processing of large-scale 3D images. The pipeline has demonstrated its capability to effectively learn and predict nuclear pore structures, achieving improvements in validation metrics compared to baseline models. Our experiments suggest that cyclical augmentation helps prevent overfitting, and random block sampling contributes to managing data imbalance. The post-processing phase successfully automates the quantification of nuclear pores without the need for manual intervention. The proposed pipeline offers an efficient and scalable approach to segmenting and quantifying nuclear pores in 3D microscopy images. Despite the ongoing challenges of computational intensity and data volume, the techniques developed in this study provide insights into the automation of complex biological image analysis tasks, with potential applications extending beyond the detection of nuclear pores. / Master of Science / This thesis outlines a computer program developed to automatically segment and count nuclear pores in 3D cell images, aiding cell and disease research. This program aims to handle large, complex image data more effectively, boost accuracy, and cut down the need for manual labor. We created a system that prepares data, applies a technique called augmentation to enrich it, selects specific image sections, and carries out a three-step final analysis. At the core of our program is a 3D U-Net model, a type of deep learning network, that has been enhanced to address the challenges of scarce labeled data and the processing of very large images. The system developed is capable of learning and identifying the structure of nuclear pores in cell images. Our experiments indicate that using augmentation in a cyclical manner during training can prevent overfitting, which is when a model learns the training data too well, and cannot suitably generalize. Selecting certain parts of the images for processing proves helpful with imbalanced data. Additionally, the program can automatically count nuclear pores in the final step. The proposed program is effective for analyzing and counting nuclear pores in 3D cell images and has the potential for broader applications in cell analysis. Despite the challenges of managing large datasets and the significant computational power required, our methods open new possibilities for automating cell studies, with uses extending beyond just nuclear pores.

Bright field cellular microscopy offers an image capturing method that is both non-invasive and simple to implement. However, the resulting micrographs pose challenges for image segmentation which are compounded when the subject cells are tightly clustered or overlapping. Filamentous cyanobacteria are a type of organism that grow as linearly arranged cells forming chain-like filaments. Existing methods for bright field cell segmentation perform poorly on micrographs of these bacteria, and are incapable of identifying the filaments. Existing filament tracking methods are rudimentary, and cannot reliably account for overlapping or parallel touching filaments. We propose a new approach for identifying filaments in bright field micrographs by combining information about both filaments and cells. This information is used by an evolutionary strategy to iteratively construct a continuous spline representation that tracks the medial line of the filaments. We demonstrate that overlapping and parallel touching filaments are handled appropriately in many difficult cases.

Willfully controlling the focus of an extracellular stimulus remains a significant challenge in the development of neural prosthetics and therapeutic devices. In part, this challenge is due to the vast set of complex interactions between the electric fields induced by the microelectrodes and the complex morphologies and dynamics of the neural tissue. Overcoming such issues to produce methodologies for targeted neural stimulation requires a system that is capable of (1) delivering precise, localized stimuli a function of the stimulating electrodes and (2) recording the locations and magnitudes of the resulting evoked responses a function of the cell geometry and membrane dynamics. In order to improve stimulus delivery, we developed microfabrication technologies that could specify the electrode geometry and electrical properties. Specifically, we developed a closed-loop electroplating strategy to monitor and control the morphology of surface coatings during deposition, and we implemented pulse-plating techniques as a means to produce robust, resilient microelectrodes that could withstand rigorous handling and harsh environments. In order to evaluate the responses evoked by these stimulating electrodes, we developed microscopy techniques and signal processing algorithms that could automatically identify and evaluate the electrical response of each individual neuron. Finally, by applying this simultaneous stimulation and optical recording system to the study of dissociated cortical cultures in multielectode arrays, we could evaluate the efficacy of excitatory and inhibitory waveforms. Although we found that the proximity of the electrode is a poor predictor of individual neural excitation thresholds, we have shown that it is possible to use inhibitory waveforms to globally reduce excitability in the vicinity of the electrode. Thus, the developed system was able to provide very high resolution insight into the complex set of interactions between the stimulating electrodes and populations of individual neurons.

<p>In this dissertation, we document a few of our recent attempts in bridging the gap between the fast evolving deep learning research and the vast industry needs for dealing with computer vision challenges. More specifically, we developed novel deep-learning-based techniques for the following application-driven computer vision challenges: image super-resolution with quality restoration, motion estimation by optical flow, object detection for shape reconstruction, and object segmentation for motion tracking. Those four topics cover the computer vision hierarchy from the low level where digital images are processed to restore missing information for better human perception, to middle level where certain objects of interest are recognized and their motions are analyzed, finally to high level where the scene captured in the video footage will be interpreted for further analysis. In the process of building the whole-package of  ready-to-deploy solutions, we center our efforts on designing and training the most suitable convolutional neural networks for the particular computer vision problem at hand. Complementary procedures for data collection, data annotation,  post-processing of network outputs tailored for specific application needs, and deployment details will also be discussed where necessary. We hope our work demonstrates the applicability and versatility of convolutional neural networks for real-world computer vision tasks on a broad spectrum, from seeing better to understanding better.</p>

Yes / Background and Objective Corneal endothelial cell abnormalities may be associated with a number of corneal and systemic diseases. Damage to the endothelial cells can significantly affect corneal transparency by altering hydration of the corneal stroma, which can lead to irreversible endothelial cell pathology requiring corneal transplantation. To date, quantitative analysis of endothelial cell abnormalities has been manually performed by ophthalmologists using time consuming and highly subjective semi-automatic tools, which require an operator interaction. We developed and applied a fully-automated and real-time system, termed the Corneal Endothelium Analysis System (CEAS) for the segmentation and computation of endothelial cells in images of the human cornea obtained by in vivo corneal confocal microscopy. Methods First, a Fast Fourier Transform (FFT) Band-pass filter is applied to reduce noise and enhance the image quality to make the cells more visible. Secondly, endothelial cell boundaries are detected using watershed transformations and Voronoi tessellations to accurately quantify the morphological parameters of the human corneal endothelial cells. The performance of the automated segmentation system was tested against manually traced ground-truth images based on a database consisting of 40 corneal confocal endothelial cell images in terms of segmentation accuracy and obtained clinical features. In addition, the robustness and efficiency of the proposed CEAS system were compared with manually obtained cell densities using a separate database of 40 images from controls (n = 11), obese subjects (n = 16) and patients with diabetes (n = 13). Results The Pearson correlation coefficient between automated and manual endothelial cell densities is 0.9 (p < 0.0001) and a Bland–Altman plot shows that 95% of the data are between the 2SD agreement lines. Conclusions We demonstrate the effectiveness and robustness of the CEAS system, and the possibility of utilizing it in a real world clinical setting to enable rapid diagnosis and for patient follow-up, with an execution time of only 6 seconds per image.

In this project report a novel pixel-based approach to digital pathology is proposed. The algorithm directly decides the class of single pixels in an image without needing the larger context of neighbouring pixels. This allows researchers to circumvent complications that might arise from using classical cell segmentation methods based around counting cells - which then relies on the cell segmentation being close to perfect. Such issues are avoided by pixel-based approaches by instead directly measuring total area. The algorithm is tested on the BOMI2 Redox dataset consisting of 79 samples of multi-spectral images from lung cancer patients. The results of the algorithm are compared against ground truth data in the form of RNA sequencing data from the same patient cores as the images are taken. The algorithm achieves Spearman correlations in the range of R = [0.4,0.6], thereby serving as an initial testament to the validity of pixel-based methods. Furthermore an automatic method for deciding biomarker threshold values is proposed, based around finding the knee point of the biomarker histogram. The threshold values found by the algorithm on the BOMI2 Redox data set are reasonable. The method opens up for a standardised way of deciding thresholds in digital pathology, allowing easier comparison between research results from different researchers.

Prostate cancer is one of the most common types of cancer occurring in men. Several types of research have been done using deep learning methods for the classification/prediction of cancer grades. In this thesis, the results of prostate cancer risk prediction, based only on the images of cells from the prostate tissues, have been analyzed. Cell images from the prostate tissues were extracted using a deep learning based segmentation model. These cell images were then used in a Multiple Instance Learning model for cancer risk prediction. An attention mechanism was used to visualize the regions in the tissue to which the model paid more attention. The results suggest that the Multiple Instance Learning (MIL) model achieves an Area Under the receiver Operating Characteristics (AUROC) of 0.641 ± 0.013, which is better than a random model for low-risk vs. high-risk cancer prediction. The model’s prediction was made on cell images, with the glandular information destroyed. The MIL model, however, performs worse than a model which gets to see the glandular architecture of the cells in the prostate tissues. / Prostatacancer är en av de vanligaste typerna av cancer som förekommer ho smän. Flera typer av forskning har gjorts med metoder för djupinlärning förklassificering/förutsägelse av cancerns malignitetsgrad. I detta examensarbete harresultaten av prostatacancerriskprediktion, baserad enbart på bilder av celler från prostatavävnaderna, analyserats. Cellbilder från prostatavävnaderna extraherades med hjälp av en djupinlärningsbaserad segmenteringsmodell. Dessa cellbilder användes sedan i en Multiple Instance Learning-modell för förutsägelse av cancerrisk. En uppmärksamhetsmekanism användes för att visualisera de regioner i vävnaden som modellen ägnade mer uppmärksamhet åt. Resultaten tyder på att Multiple Instance Learning-modellen uppnår en AUROC på 0.641 ± 0.013, vilket är bättre än en slumpmässig modell för förutsägelse av lågrisk kontra högrisk cancer. Modellens förutsägelse gjordes på cellbilder, med körtelinformationen förstörd. MIL-modellen presterar dock sämre än en modell som får se körtelarkitekturen hos cellerna i prostatavävnaderna.

Understanding the development, organization, and function of tissues is a central goal in developmental biology. With modern time-lapse microscopy, it is now possible to image entire tissues during development and thereby localize subcellular proteins. A particularly productive area of research is the study of single layer epithelial tissues, which can be simply described as a 2D manifold. For example, the apical band of cell adhesions in epithelial cell layers actually forms a 2D manifold within the tissue and provides a 2D outline of each cell. The Drosophila melanogaster wing has become an important model system, because its 2D cell organization has the potential to reveal mechanisms that create the final fly wing shape. Other examples include structures that naturally localize at the surface of the tissue, such as the ciliary components of planarians. Data from these time-lapse movies typically consists of mosaics of overlapping 3D stacks. This is necessary because the surface of interest exceeds the field of view of todays microscopes. To quantify cellular tissue dynamics, these mosaics need to be processed in three main steps: (a) Extracting, correcting, and stitching individ- ual stacks into a single, seamless 2D projection per time point, (b) obtaining cell characteristics that occur at individual time points, and (c) determine cell dynamics over time. It is therefore necessary that the applied methods are capable of handling large amounts of data efficiently, while still producing accurate results. This task is made especially difficult by the low signal to noise ratios that are typical in live-cell imaging. In this PhD thesis, I develop algorithms that cover all three processing tasks men- tioned above and apply them in the analysis of polarity and tissue dynamics in large epithelial cell layers, namely the Drosophila wing and the planarian epithelium. First, I introduce an efficient pipeline that preprocesses raw image mosaics. This pipeline accurately extracts the stained surface of interest from each raw image stack and projects it onto a single 2D plane. It then corrects uneven illumination, aligns all mosaic planes, and adjusts brightness and contrast before finally stitching the processed images together. This preprocessing does not only significantly reduce the data quantity, but also simplifies downstream data analyses. Here, I apply this pipeline to datasets of the developing fly wing as well as a planarian epithelium. I additionally address the problem of determining cell polarities in chemically fixed samples of planarians. Here, I introduce a method that automatically estimates cell polarities by computing the orientation of rootlets in motile cilia. With this technique one can for the first time routinely measure and visualize how tissue polarities are established and maintained in entire planarian epithelia. Finally, I analyze cell migration patterns in the entire developing wing tissue in Drosophila. At each time point, cells are segmented using a progressive merging ap- proach with merging criteria that take typical cell shape characteristics into account. The method enforces biologically relevant constraints to improve the quality of the resulting segmentations. For cases where a full cell tracking is desired, I introduce a pipeline using a tracking-by-assignment approach. This allows me to link cells over time while considering critical events such as cell divisions or cell death. This work presents a very accurate large-scale cell tracking pipeline and opens up many avenues for further study including several in-vivo perturbation experiments as well as biophysical modeling. The methods introduced in this thesis are examples for computational pipelines that catalyze biological insights by enabling the quantification of tissue scale phenomena and dynamics. I provide not only detailed descriptions of the methods, but also show how they perform on concrete biological research projects.

Understanding the development, organization, and function of tissues is a central goal in developmental biology. With modern time-lapse microscopy, it is now possible to image entire tissues during development and thereby localize subcellular proteins. A particularly productive area of research is the study of single layer epithelial tissues, which can be simply described as a 2D manifold. For example, the apical band of cell adhesions in epithelial cell layers actually forms a 2D manifold within the tissue and provides a 2D outline of each cell. The Drosophila melanogaster wing has become an important model system, because its 2D cell organization has the potential to reveal mechanisms that create the final fly wing shape. Other examples include structures that naturally localize at the surface of the tissue, such as the ciliary components of planarians. Data from these time-lapse movies typically consists of mosaics of overlapping 3D stacks. This is necessary because the surface of interest exceeds the field of view of todays microscopes. To quantify cellular tissue dynamics, these mosaics need to be processed in three main steps: (a) Extracting, correcting, and stitching individ- ual stacks into a single, seamless 2D projection per time point, (b) obtaining cell characteristics that occur at individual time points, and (c) determine cell dynamics over time. It is therefore necessary that the applied methods are capable of handling large amounts of data efficiently, while still producing accurate results. This task is made especially difficult by the low signal to noise ratios that are typical in live-cell imaging. In this PhD thesis, I develop algorithms that cover all three processing tasks men- tioned above and apply them in the analysis of polarity and tissue dynamics in large epithelial cell layers, namely the Drosophila wing and the planarian epithelium. First, I introduce an efficient pipeline that preprocesses raw image mosaics. This pipeline accurately extracts the stained surface of interest from each raw image stack and projects it onto a single 2D plane. It then corrects uneven illumination, aligns all mosaic planes, and adjusts brightness and contrast before finally stitching the processed images together. This preprocessing does not only significantly reduce the data quantity, but also simplifies downstream data analyses. Here, I apply this pipeline to datasets of the developing fly wing as well as a planarian epithelium. I additionally address the problem of determining cell polarities in chemically fixed samples of planarians. Here, I introduce a method that automatically estimates cell polarities by computing the orientation of rootlets in motile cilia. With this technique one can for the first time routinely measure and visualize how tissue polarities are established and maintained in entire planarian epithelia. Finally, I analyze cell migration patterns in the entire developing wing tissue in Drosophila. At each time point, cells are segmented using a progressive merging ap- proach with merging criteria that take typical cell shape characteristics into account. The method enforces biologically relevant constraints to improve the quality of the resulting segmentations. For cases where a full cell tracking is desired, I introduce a pipeline using a tracking-by-assignment approach. This allows me to link cells over time while considering critical events such as cell divisions or cell death. This work presents a very accurate large-scale cell tracking pipeline and opens up many avenues for further study including several in-vivo perturbation experiments as well as biophysical modeling. The methods introduced in this thesis are examples for computational pipelines that catalyze biological insights by enabling the quantification of tissue scale phenomena and dynamics. I provide not only detailed descriptions of the methods, but also show how they perform on concrete biological research projects.

Indiana University-Purdue University Indianapolis (IUPUI) / A crucial task in computer vision and biomedical image applications is to represent images in a numerically compact form for understanding, evaluating and/or mining their content. The fundamental step of this task is the segmentation of images into regions, given some homogeneity criteria, prior appearance and/or shape information criteria. Specifically, segmentation of cells in microscopic images is the first step in analyzing many biomedical applications. This thesis is a part of the project entitled "Construction and profiling of biodegradable cardiac patches for the co-delivery of bFGF and G-CSF growth factors" funded by National Institutes of Health (NIH). We present a method that simultaneously segments the population of cells while partitioning the cell regions into cytoplasm and nucleus in order to evaluate the spatial coordination on the image plane, density and orientation of cells. Having static microscopic images, with no edge information of a cytoplasm boundary and no time sequence constraints, traditional cell segmentation methods would not perform well. The proposed method combines deformable models with a probabilistic framework in a simple graphical model such that it would capture the shape, structure and appearance of a cell. The process aims at the simultaneous cell partitioning into nucleus and cytoplasm. We considered the relative topology of the two distinct cell compartments to derive a better segmentation and compensate for the lack of edge information. The framework is applied to static fluorescent microscopy, where the cultured cells are stained with calcein AM.