Back to search

Boston University

Flexible multimodal learning for whole slide image analysis

Abstract

dc:description.abstract

Whole slide imaging (WSI) has revolutionized digital pathology, enabling the digitization of entire tissue samples on glass slides into high-resolution images, providing a detailed view of tissue morphology essential for analysis and interpretation. However, practical challenges arise due to the varying availability and cost of different slide staining techniques. Hematoxylin and Eosin (H&E) staining is commonly performed and cost-effective, but it offers limited information compared to more expensive and less frequently conducted immunohistochemical (IHC) staining, which provides specific molecular insights critical for diagnosing conditions like Alzheimer’s disease (AD) and chronic traumatic encephalopathy (CTE). Addressing the need for a comprehensive analysis in the absence of complete modality sets, this research introduces ”MultiStainKD ,” a novel framework designed to handle multimodal WSI data effectively, even when certain modalities, such as IHC, are missing. MultiStainKD employs a combination of computational architectural design and training strategies, optimizing the use of available data to enhance diagnostic accuracy. It utilizes a blend of convolutional and transformer-based components to capture the nuanced interplay of local and global tissue features, thus ensuring robust performance without the necessity for costly feature generation processes traditionally used to compensate for missing modalities. Our extensive testing shows MultiStainKD’s superior capability in managing missing modalities and setting new benchmarks in AD and CTE prediction accuracy. By leveraging the inherent complementary information within accessible stains and slides, MultiStainKD demonstrates its practical value and effectiveness, aligning with the goal of maintaining diagnostic quality despite the unavailability of comprehensive staining modalities. This advancement signifies an important step forward in multimodal learning for digital pathology, potentially enhancing diagnostic accuracy and supporting patient care when full modality data is not available.

Degree

thesis:*
Name thesis:degree_name
Master of Science
Level thesis:degree_level
masters
Discipline thesis:degree_discipline
Computer Science
Grantor
Boston University
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Sharma, Harsh
Advisor dc:contributor.advisor
  • Kolachalama, Vijaya B.
ORCID dc:identifier.orcid

Subjects

dc:subject × 7

Rights

dc:rights
Statement dc:rights
  • Attribution 4.0 International
Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/2144/49902

Chain of custody

source
Harvested from
Boston University
Base URL
open.bu.edu/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Sharma, Harsh. Flexible multimodal learning for whole slide image analysis. masters thesis, Boston University, 2024. https://hdl.handle.net/2144/49902