Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 183 for “"Multimodal learning"”.
-
Sign language understanding using multimodal learning
… language dictionaries in a multiple instance learning framework, (ii) exploiting the attention mechanism of a Transformer trained on a video-to-text sequence prediction task, (iii) pseudo-labelling from a strong sign recognition model, (iv) leveraging in-domain exemplars from previous …
-
DEVELOPING MULTIMODAL LEARNING METHODS FOR VIDEO UNDERSTANDING
In recent years, the field of deep learning, with a particular emphasis on multimodal representation learning, has experienced significant advancements. These advancements are largely attributable to groundbreaking progress in areas such as computer vision, voice recognition, natural language …
-
Video Generation and Understanding with Multimodal Learning
… appearance. However, video content is inherently multimodal, comprising not only static appearance but also motion, geometry, and semantic cues. Motion reflects dynamic changes, geometry defines spatial structure, and semantics add contextual information. Integrating these modalities yields richer …
-
TOWARD HUMAN-CENTRIC VISUAL INTELLIGENCE THROUGH MULTIMODAL LEARNING
… interpretations aligned with human expectations. Multimodal Large Language Models (MLLMs) provide rich semantic priors, yet remain limited in spatial grounding, computational efficiency, and reliable preference alignment. This dissertation addresses these challenges through multimodal methods that …
-
Variational autoencoders for supervision, calibration and multimodal learning
<p>Learning representations of data has long been a desirable goal in machine learning. Constructing such representations enables downstream tasks such as classification or object detection to be preformed efficiently. Furthermore, it is desirable to have these representations be constructed in …
-
Generative and Multimodal Learning for Vision and Language
L'abstract è presente nell'allegato / the abstract is in the attachment
-
Optimising Multimodal Learning with a System-Aware Perspective
Multimodal learning in machine learning (ML) aims to improve model performance by integrating multiple modalities, mirroring human perception. However, there are several challenges when training such multimodal models from scratch. These challenges include (1) modality imbalance, where certain …
-
Multimodal learning with graphs for multiomics cancer classification
… insights into cancer biology. Graph-based multimodal learning has shown remarkable potential in integrating these modalities to unravel cancer complexity, enhance biological predictions, and facilitate biomarker discovery. Despite their promise, these models face three challenges. First, …
-
Flexible multimodal learning for whole slide image analysis
… ,” a novel framework designed to handle multimodal WSI data effectively, even when certain modalities, such as IHC, are missing. MultiStainKD employs a combination of computational architectural design and training strategies, optimizing the use of available data to enhance diagnostic …
-
Multimodal Learning for Wearable Sensor-Based Human Action Recognition
… system. To give machines similar intelligence, multimodal machine learning, which merges data from various sources, has become a popular research area with recent advancements. In this study, we proposed two knowledge distillation (KD) frameworks by distilling knowledge from a larger multimodal …
-
Multimodal Learning for 3D Perception in Autonomous Driving Environments
… of image datasets has enabled self-supervised learning to extract versatile 2D representations from unlabeled data, supporting 2D tasks in label-efficient settings. Additionally, multimodal datasets that combine images with text have propelled the success of Vision-Language Models (VLMs), which …
-
Multimodal Learning and Its Application to Mobile Active Authentication
… application of the more general problem, namely, multimodal classification. The idea of multimodal classification is to utilize multiple sources (modalities) measuring the same instance to improve the overall performance compared to using a single source (modality). Multimodal classification also …
-
Bridging Multimodal Learning and Planning for Intelligent Task Assistance
Task-assistance systems provide adaptive, multimodal guidance for complex, step-based activities such as cooking and DIY projects. A central challenge lies in enabling these systems to interpret real-world scenarios—understanding user intent from verbal, visual, or textual cues and generating …
-
Multimodal learning and its application to modeling Alzheimer's disease
Multimodal learning gains increasing attention in recent years as heterogeneous data modalities are being collected from diverse domains or extracted from various feature extractors and used for learning. Multimodal learning is to integrate predictive information from different modalities to …
-
Protein Multimodal Learning for Variant Effect Prediction and Protein Engineering
… for variant effect prediction (VEP) and motivate learning beyond sequences, we first examined the transferability from the sequence pretraining to VEP by evaluating pLMs’ awareness of sequence semantics and structure properties, in relationship to their sensitivity of variant fitness. We found …
-
Advancing Video Unsupervised Domain Adaptation via Multimodal Learning and Efficient Tokenization
… taking place in each video input. While deep learning has driven substantial progress in this area, a critical vulnerability remains that models trained on specific, labeled datasets (referred as source domain) often fail to generalize to real-world, unseen data (target domain) due to domain …
-
Conditional Neural Language Models for Multimodal Learning and Natural Language Understanding
… recurrent neural networks with applications to multimodal learning and natural language understanding. We first introduce a LSTM encoder for learning visual-semantic embeddings for ranking the relevance of text to images in a joint embedding space. Next we introduce three log-bilinear models for …
Page 1 of 10