Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 30 for “"Multimodal representation learning"”.
-
Multimodal Representation Learning for Geospatial Soundscape Mapping
… mapping problem from the perspective of multimodal representation learning and have developed a series of frameworks—GeoCLAP, PSM, and Sat2Sound—to address this task. For each framework, we demonstrate the effectiveness of a shared multimodal representation space in generating soundscape …
-
Self-Supervised Multimodal Representation Learning for Neuroimaging
… have to build computationally and data-efficient multimodal models while ensuring interpretability, robustness, and fairness. We have taken two significant steps toward this goal. First, we introduce a fast, accurate, and multimodal architecture for segmenting brain tissues and atlas regions that …
-
Multimodal representation learning for fine art analysis
This thesis develops multimodal representation learning methods for fine art analysis. Fine art exists within historically situated systems of creation, interpretation, and institutional evaluation in which artworks, artists, movements, and institutions are simultaneously represented as visual …
-
Multimodal Representation Learning Using Self-Supervision and Attention
… to enhance the efficiency of the system. Hence, learning representations simultaneously from multiple modalities is better than learning from a single modality while approaching problems like image/text classification, image-to-text/text-to-image transformation, object detection and segmentation. …
-
Multimodal representation learning with application in sentiment analysis
Multimodal learning is the computational paradigm focused on learning and integrating heterogeneous and interconnected information sources, i.e., modalities. At the core of any multimodal learning system lies multimodal representation learning, which aims to learn representations that both reflect …
-
Multimodal Representation Learning in Medical Using Vision-Language Models
Recent advances in multimodal models such as LLaVA and InstructBLIP highlight the importance of high-quality image encoders, particularly in the biomedical domain where figures and captions are complex. Existing medical vision–language datasets primarily emphasize scale, often overlooking data …
-
Multimodal Representation Learning for Textual Reasoning over Knowledge Graphs
… search. This creates a need for multi-modal representations that capture both the semantic and structural features from the KGs. The primary objective of the proposed work is to extend the accessibility of KGs to non-expert users/institutions by enabling them to utilize non-technical textual …
-
Quantum Cognitively Motivated Context-Aware Multimodal Representation Learning for Human Language Analysis
… systems that can perceive and understand human multimodal language. This requires both the consideration of context in the form of surrounding utterances in a conversation, i.e., <i>context modelling</i>, as well as the impact of different modalities (e.g., linguistic, visual acoustic), i.e., …
-
Multimodal Representation Learning Frameworks for Modeling Progression and Heterogeneity in Alzheimer’s Disease
… computational capabilities, particularly in deep learning, multimodal representation learning frameworks aim to integrate diverse clinical modalities into a cohesive framework, capturing the most significant patterns within each modality. Existing data-driven multimodal representation learning …
-
Multimodal Representation Learning Frameworks for Modeling Progression and Heterogeneity in Alzheimer’s Disease
… computational capabilities, particularly in deep learning, multimodal representation learning frameworks aim to integrate diverse clinical modalities into a cohesive framework, capturing the most significant patterns within each modality. Existing data-driven multimodal representation learning …
-
Multimodal Representation Learning for Face Understanding: From Caption Supervision to Foundation Model Adaptation
… foundation models have greatly improved visual representation learning, their patch-based tokenization and lack of spatial inductive bias limit their ability to capture fine-grained details in low-resolution inputs. This dissertation investigates multimodal representation learning for face …
-
Toward Generalist Ophthalmic Intelligence: From Scalable Multimodal Representation Learning to a Clinical World Model
… moved from hand-specified rules into the learned representation; that doing so increases the range of clinical questions a single model can answer without making the architecture grow with the number of modalities; and that the resulting gain concentrates on tasks requiring integration across …
-
Multimodal representation learning for personalized cardiovascular care : predicting disease progression and treatment from clinical data
… This thesis presents a series of deep learning-based multimodal representation learning frameworks for personalized cardiovascular disease management, using routinely collected and non-invasive clinical data. The thesis first introduces TREAT-Net, an acute coronary syndrome treatment …
-
AN ANALYSIS OF BOTTOM-UP ATTENTION MODELS AND MULTIMODAL REPRESENTATION LEARNING FOR VISUAL QUESTION ANSWERING
… performance (accuracy) of VQA. The first is a representation learning approach (utilizing the state-of-the-art Generative Adversarial Models (GANs) (Goodfellow, et al., 2014)) to improve the image encoding system of VQA. This thesis evaluates four variants of GANs to identify a GAN architecture …
-
Attentive Cross-Modal Connections for Learning Multimodal Representations from Wearable Signals for Affect Recognition
… a new dynamic and effective technique for multimodal representation learning from wearable data. Our solution can be integrated into any stage of the pipeline, i.e., after any convolutional layer or block, to create intermediate connections between individual streams responsible for …
-
REPRESENTATION LEARNING OF DATA WITH MULTIPLE MODALITIES WITH APPLICATIONS TO VISUAL QUESTION ANSWERING
Deep learning has started a new era in Artificial Intelligence research with major breakthroughs in multiple fields. Now, as the field strives to achieve Artificial General Intelligence the focus has shifted to tasks involving data of multiple modalities. With this thesis, I address the challenges …
-
Analysis, Modeling, and Generation of Turn Dynamics in Multiparty Conversation
… behavior. This dissertation presents a unified multimodal framework for analyzing, predicting, and generating conversational behaviors in multiparty interactions. It begins by establishing a data-driven foundation through a large-scale dataset capturing speech, gaze, and full-body motion, and …
-
Audio-Visual Scene-Aware Dialog: A Step Towards Multimodal Conversational Agents
… a challenging task in visual-and-language joint representation learning. It has gained significant attention in recent years, alongside other tasks such as text-based video retrieval and video summarization, with notable progress driven by transformer-based language encoders. However, existing …
-
Learning visual and multimodal representations
Representations lie at the heart of artificial intelligence, enabling machines to perceive, interpret and interact with the world. Visual representations, extracted from images or videos, enable tasks such as image classification, image retrieval, and object detection. Visual-textual …
Page 1 of 2