No description available.

In this project report a novel pixel-based approach to digital pathology is proposed. The algorithm directly decides the class of single pixels in an image without needing the larger context of neighbouring pixels. This allows researchers to circumvent complications that might arise from using classical cell segmentation methods based around counting cells - which then relies on the cell segmentation being close to perfect. Such issues are avoided by pixel-based approaches by instead directly measuring total area. The algorithm is tested on the BOMI2 Redox dataset consisting of 79 samples of multi-spectral images from lung cancer patients. The results of the algorithm are compared against ground truth data in the form of RNA sequencing data from the same patient cores as the images are taken. The algorithm achieves Spearman correlations in the range of R = [0.4,0.6], thereby serving as an initial testament to the validity of pixel-based methods. Furthermore an automatic method for deciding biomarker threshold values is proposed, based around finding the knee point of the biomarker histogram. The threshold values found by the algorithm on the BOMI2 Redox data set are reasonable. The method opens up for a standardised way of deciding thresholds in digital pathology, allowing easier comparison between research results from different researchers.

Brain and nervous system tumors were responsible for around 250,000 deaths in 2020 worldwide. Correctly identifying different tumors is very important, because treatment options largely depend on the diagnosis. This is an expert task, but recently machine learning, and especially deep learning models have shown huge potential in tumor classification problems, and can provide fast and reliable support for pathologists in the decision making process. This thesis investigates classification of two brain tumors, glioblastoma multiforme and lower grade glioma in high-resolution H&E-stained histology images using deep learning. The dataset is publicly available from TCGA, and 220 whole slide images were used in this study. Ground truth labels were only available on whole slide level, but due to their large size, they could not be processed by convolutional neural networks. Therefore, patches were extracted from the whole slide images in two sizes and fed into separate networks for training. Preprocessing steps ensured that irrelevant information about the background was excluded, and that the images were stain normalized. The patch-level predictions were then combined to slide level, and the classification performance was measured on a test set. Experiments were conducted about the usefulness of pre-trained CNN models and data augmentation techniques, and the best method was selected after statistical comparisons. Following the patch-level training, five slide aggregation approaches were studied, and compared to build a whole slide classifier model. Best performance was achieved when using small patches (336 x 336 pixels), pre-trained CNN model without frozen layers, and mirroring data augmentation. The majority voting slide aggregation method resulted in the best whole slide classifier with 91.7% test accuracy and 100% sensitivity. In many comparisons, however, statistical significance could not be shown because of the relatively small size of the test set.

Challenges within the field of pathology leads to a high workload for pathologists. Machine learning has the ability to assist pathologists in their daily work and has shown good performance in a research setting. Anomaly detection is useful for preventing machine learning models used for classification and segmentation to be applied on data outside of the training distribution of the model. The purpose of this work was to create an optimal anomaly detection pipeline for digital pathology data using a latent diffusion model and various image similarity metrics. An anomaly detection pipeline was created which used a partial diffusion process, a combined similarity metric containing the result of multiple other similarity metrics and a contrast matching strategy for better anomaly detection performance. The anomaly detection pipeline had a good performance in an out-of-distribution detection task with an ROC-AUC score of 0.90. / <p>Examensarbetet är utfört vid Institutionen för teknik och naturvetenskap (ITN) vid Tekniska fakulteten, Linköpings universitet</p>

In digital pathology, hematoxylin &amp; eosin (H&amp;E) is a routine stain which is performed on most clinical cases and it often provides clinicians with sufficient information for diagnosis. However, when making decisions on how to guide breast cancer treatment, immunohistochemical staining of human epidermal growth factor 2 (HER2 staining) is also needed. Over-expression of the HER2 protein plays a significant role in the progression of breast cancer and is therefore important to consider during treatment planning. However, the downside of HER2 staining is that it is both time consuming and rather expensive. This thesis explores the possibility for H&amp;E-to-HER2 stain translation using generative adversarial networks (GANs). If effective, this has the potential to reduce the costs and time spent on tissue processing while still providing clinicians with the images necessary to make a complete diagnosis. To explore this area two supervised (Pix2Pix, PyramidPix2Pix) and one unsupervised (cycleGAN) GAN structure was implemented and trained on digital pathology images from the MIST dataset. These models were trained two times, with 256x256 and 512x512 patches, to see which effect patch size has on stain translation performance as well. In addition, a methodology for evaluating the quality of the generated HER2 patches was also presented and utilized. This methodology consists of structural similarity index (SSIM) and peak signal to noise ratio (PSNR) comparison to the ground truth, and a HER2 status classification protocol. In the latter a classification tool provided by Sectra was used to assign each patch with a HER2 status of No tumor, 1+, 2+ or 3+ and the statuses of the generated patches were then compared to the statuses of the ground truths. The results show that the supervised Pyramid Pix2Pix model trained on 512x512 patches performs the best according to the SSIM and PSNR metrics. However, the unsupervised cycleGAN model shows more promising results when it comes to both visual assessment and the HER2 status classification protocol. Especially when trained on 256x256 patches for 200 epochs which gave an accuracy of 0.655, F1-score of 0.674 and MCC of 0.490. In conclusion the HER2 status classification protocol is deemed as a suitable way to evaluate H&amp;E-to-HER2 stain translation and thereby the unsupervised method is considered to be better than the supervised. Moreover, it is also concluded that a smaller patch size result in worse translation of cellular structure for the supervised methods. Further studies should focus on incorporating HER2 status classification in the cycleGAN loss function and more extensive training runs to further improve the quality of H&amp;E-to-HER2 stain translation.

Histopathology refers to the visual inspection of tissue under the microscope and it is the core part of diagnosis. The process of manual inspection of histopathology slides is very time-consuming for pathologists and error-prone. Furthermore, diagnosis can sometimes differ among specialists. In recent years, convolutional neural networks (CNNs) have demonstrated remarkable performances in the classification of digital histopathology images. However, due to their high resolution, whole-slide images are of immense size, often gigapixels, making it infeasible to train CNNs directly on them. For that, patch-level classification is used instead. In this study, a deep learning approach for patch-level classification of glioblastoma (i.e. brain cancer) tissue is proposed. Four different state-of-the-art models were evaluated (MobileNetV2, ResNet50, ResNet152V2, and VGG16), with MobileNetV2 being the best among them, achieving 80% test accuracy. The study also proposes a scratch-trained CNN architecture, inspired by the popular VGG16 model, which achieved 81% accuracy. Both models scored 87% test accuracy when trained with data augmentation. All models were trained and tested on randomly sampled patches from the Ivy GAP dataset, which consisted of 724 H&amp;E images in total. Finally, as patch-level predictions cannot be used explicitly by pathologists, prediction results from two slides were presented in the form of whole-slide images. Post-processing was also performed on those two predicted WSIs in order to make use of the spatial correlations among the patches and increase the classification accuracy. The models were statistically compared using the Wilcoxon signed-rank test.

Prostate cancer is the second most diagnosed cancer worldwide and its diagnosis is done through visual inspection of biopsy tissue by a pathologist, who assigns a score used by doctors to decide on the treatment. However, the scoring system, the Gleason score, is affected by a high inter and intra-observer variability, lack of standardization, and overestimation. Therefore, there is a need for new solutions that can reduce these issues and provide a more accurate diagnosis. Nowadays, high-resolution digital images of biopsy tissues can be obtained and stored. The availability of such images, called Whole Slide Images (WSI) allows the implementation of Machine and Deep learning models to assist pathologists in diagnosing prostate cancer. Multiple-Instance Learning (MIL) has been shown to reach very promising results in digital pathology and binary classification of prostate cancer slides. However, such models require large datasets to ensure good performances. This project wants to investigate the use of small sets of strongly annotated images to create new large datasets to train a MIL model. To evaluate the performance of this approach, the standard dataset is used to obtain baselines for both binary and multiclass classification tasks. For multiclassification, the International Society of Urological Pathology (ISUP) score is used, which is derived from the Gleason score. The dataset used is the publicly available PANDA. In this project, only the slides from RadboudUniversity Medical Center are used, which consists of 5160 images. The MIL model chosen is the Clustering-constrained Attention Multiple instance learning (CLAM) model, which is publicly available. The standard approach reaches a Cohen’s kappa (κ) of 0.78 and 0.59 for binary and multiclass classification respectively. To evaluate the new approach, large datasets are created starting from different set sizes. Using 500 images, the model reaches a κ of 0.72 and 0.38 respectively. While for the binary the results of the two approaches are comparable, the new approach is not beneficial for multiclass classification tasks.

This thesis develops an Artificial Intelligence (AI) approach intended for accurate patient stratification and precise diagnostics/prognostics in clinical and preclinical applications. The rapid advance in high throughput technologies and bioinformatics tools is still far from linking precisely the genome-phenotype interactions with the biological mechanisms that underlie pathophysiological conditions. In practice, the incomplete knowledge on individual heterogeneity in complex diseases keeps forcing clinicians to settle for surrogate endpoints and therapies based on a generic one-size-fits-all approach. The working hypothesis is that AI can add new tools to elaborate and integrate together in new features or structures the rich information now available from high-throughput omics and bioimaging data, and that such re- structured information can be applied through predictive models for the precision medicine paradigm, thus favoring the creation of safer tailored treatments for specific patient subgroups. The computational techniques in this thesis are based on the combination of dimensionality reduction methods with Deep Learning (DL) architectures to learn meaningful transformations between the input and the predictive endpoint space. The rationale is that such transformations can introduce intermediate spaces offering more succinct representations, where data from different sources are summarized. The research goal was attacked at increasing levels of complexity, starting from single input modalities (omics and bioimaging of different types and scales), to their multimodal integration. The approach also deals with the key challenges for machine learning (ML) on biomedical data, i.e. reproducibility, stability, and interpretability of the models. Along this path, the thesis contribution is thus the development of a set of specialized AI models and a core framework of three tools of general applicability: i. A Data Analysis Plan (DAP) for model selection and evaluation of classifiers on omics and imaging data to avoid selection bias. ii. The histolab Python package that standardizes the reproducible pre-processing of Whole Slide Images (WSIs), supported by automated testing and easily integrable in DL pipelines for Digital Pathology. iii. Unsupervised and dimensionality reduction techniques based on the UMAP and TDA frameworks for patient subtyping. The framework has been successfully applied on public as well as original data in precision oncology and predictive toxicology. In the clinical setting, this thesis has developed1: 1. (DAPPER) A deep learning framework for evaluation of predictive models in Digital Pathology that controls for selection bias through properly designed data partitioning schemes. 2. (RADLER) A unified deep learning framework that combines radiomics fea- tures and imaging on PET-CT images for prognostic biomarker development in head and neck squamous cell carcinoma. The mixed deep learning/radiomics approach is more accurate than using only one feature type. 3. An ML framework for automated quantification tumor infiltrating lymphocytes (TILs) in onco-immunology, validated on original pathology Neuroblastoma data of the Bambino Gesu’ Children’s Hospital, with high agreement with trained pathologists. The network-based INF pipeline, which applies machine learning models over the combination of multiple omics layers, also providing compact biomarker signatures. INF was validated on three TCGA oncogenomic datasets. In the preclinical setting the framework has been applied for: 1. Deep and machine learning algorithms to predict DILI status from gene expression (GE) data derived from cancer cell lines on the CMap Drug Safety dataset. 2. (ML4TOX) Deep Learning and Support Vector Machine models to predict potential endocrine disruption of environmental chemicals on the CERAPP dataset. 3. (PathologAI) A deep learning pipeline combining generative and convolutional models for preclinical digital pathology. Developed as an internal project within the FDA/NCTR AIRForce initiative and applied to predict necrosis on images from the TG-GATEs project, PathologAI aims to improve accuracy and reduce labor in the identification of lesions in predictive toxicology. Furthermore, GE microarray data were integrated with histology features in a unified multi-modal scheme combining imaging and omics data. The solutions were developed in collaboration with domain experts and considered promising for application.

Histopathological evaluation and Gleason grading on Hematoxylin and Eosin(H&amp;E) stained specimens is the clinical standard in grading prostate cancer. Recently, deep learning models have been trained to assist pathologists in detecting prostate cancer. However, these predictions could be improved further regarding variations in morphology, staining and differences across scanners. An approach to tackle such problems is to employ conditional GANs for style transfer. A total of 52 prostatectomies from 48 patients were scanned with two different scanners. Data was split into 40 images for training and 12 images for testing and all images were divided into overlapping 256x256 patches. A segmentation model was trained using images from scanner A, and the model was tested on images from both scanner A and B. Next, GANs were trained to perform style transfer from scanner A to scanner B. The training was performed using unpaired training images and different types of Unsupervised Image to Image Translation GANs (CycleGAN and UNIT). Beside the common CycleGAN architecture, a modified version was also tested, adding Kullback Leibler (KL) divergence in the loss function. Then, the segmentation model was tested on the augmented images from scanner B.The models were evaluated on 2,000 randomly selected patches of 256x256 pixels from 10 prostatectomies. The resulting predictions were evaluated both qualitatively and quantitatively. All proposed methods outperformed in AUC, in the best case the improvement was of 16%. However, only CycleGAN trained on a large dataset demonstrated to be capable to improve the segmentation tool performance, preserving tissue morphology and obtaining higher results in all the evaluation measurements. All the models were analyzed and, finally, the significance of the difference between the segmentation model performance on style transferred images and on untransferred images was assessed, using statistical tests.

L'histologie tridimensionnelle (3D) est un nouvel outil avancé de cancérologie. L'ensemble du profil chimique et des caractéristiques physiologiques d'un tissu est essentiel pour comprendre la logique du développement d'une pathologie. Cependant, il n'existe aucune technique analytique, in vivo ou histologique, capable de découvrir de telles caractéristiques anormales et de fournir une distribution3D à une résolution microscopique. Nous présentons ici une méthode unique de microscopie infrarouge (IR) à haut débit combinant une correction d'image automatisée et une analyse ultérieure des données spectrales pour la reconstruction d'image 3D-IR. Nous avons effectué l'analyse spectrale d'un organe complet pour un petit modèle animal, un cerveau de souris avec une tumeur de gliome implantée. L'image 3D-IR est reconstruite à partir de 370 coupes de tissus consécutives et corrigée à l'aide du tomogramme à rayons X de l'organe pour une analyse quantitative précise du contenu chimique. Une matrice 3D de spectres IR 89 x 106 est générée, ce qui nous permet de séparer la masse tumorale des tissus cérébraux sains en fonction de divers paramètres anatomiques,chimiques et métaboliques. Nous démontrons pour la première fois que des paramètres métaboliques quantitatifs (glucose, glycogène et lactate) peuvent être extraits et reconstruits en 3D à partir des spectres IR pour la caractérisation du métabolisme cérébral / tumoral (évaluation de l'effet de Warburg dans les tumeurs). Notre méthode peut être davantage exploitée en recherchant l'ensemble du profil spectral, en distinguant différents points de repère anatomiques dans le cerveau.Nous le démontrons par la reconstruction du corps calleux et de la région des noyaux gris centraux du cerveau. / Three-dimensional (3D) histology is a new advanced tool for cancerology. The whole chemical profile and physiological characteristics of a tissue is essential to understand the rationale of pathology development. However, there is no analytical technique, in vivo or histological, that is able to discover such abnormal features and provide a 3D distribution at microscopic resolution.Here, we introduce a unique high- throughput infrared (IR) microscopy method that combines automated image correction and subsequent spectral data analysis for 3D-IR image reconstruction. I performed spectral analysis of a complete organ for a small animal model, a mouse brain with animplanted glioma tumor. The 3D-IR image is reconstructed from 370 consecutive tissue sectionsand corrected using the X-ray tomogram of the organ for an accurate quantitative analysis of thechemical content. A 3D matrix of 89 x 106 IR spectra is generated, allowing us to separate the tumor mass from healthy brain tissues based on various anatomical, chemical, and metabolic parameters. I demonstrate for the first time that quantitative metabolic parameters (glucose, glycogen and lactate) can be extracted and reconstructed in 3D from the IR spectra for the characterization of the brain vs. tumor metabolism (assessing the Warburg effect in tumors). Our method can be further exploited by searching for the whole spectral profile, discriminating different anatomical landmarks in the brain. I demonstrate this by the reconstruction of the corpus callosum and basal ganglia region of the brain.