Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 99 for “"Multimodal Models"”.
-
Improving commonsense reasoning in multimodal models
… and reasoning. Although large-scale foundation models have achieved strong performance on commonsense benchmarks, they struggle to reason beyond surface-level patterns, particularly in novel and ambiguous settings. This thesis develops methods to evaluate and improve commonsense reasoning in …
-
Synthesizing Missing Data for Multimodal Models
Despite recent advances, multimodal models remain prone to failure in real-world settings. One primary challenge in multimodal model development is the data gap: naturally occurring multimodal data are scarce, distributionally biased, and difficult to obtain at scale. This dissertation investigates …
-
Reasoning about actions with large multimodal models
Large multimodal models have become central for solving sequential decision-making tasks, enabling improved learning in diverse areas such as home robotics and automated software development. However, leveraging these models for sequential decision-making requires robust action reasoning …
-
Grounding Perception and Reasoning in Multimodal Models
Current large multimodal models achieve impressive recognition capabilities that approach human-level performance. Their success is driven largely by massive web-scale pretraining, with continued improvements from scaling model and data size. However, this reliance on web data has not yet …
-
Bridging Perception and Reasoning in Multimodal Models
Most human knowledge is acquired through rich, multimodal cognitive experiences, where both vision and language play essential roles. For a long time, computer vision (CV) has aimed to reconstruct the physical world from sensory data, while natural language processing (NLP) has seeked to achieve …
-
Building the Next Generation of Multimodal Models
… and text. I argue that the next generation of multimodal models will be enabled by a deeper understanding of how to design pretraining datasets, and by techniques that offer better control over models after pretraining. Towards the first goal, I introduce a fully open-source benchmark for …
-
Multimodal Models of Time Series and Text
… in finance, healthcare, and science. However, multimodal NLP research has mostly focused on images and video. Here I enumerate barriers towards such models and describe my work towards mitigating them. I detail new multimodal NLP tasks for reasoning about time series, describe an LLM-powered …
-
Learning, Seeing, and Highlighting with Multimodal Models
Thesis (Ph.D.)--Michigan State University. Computer Science - Doctor of Philosophy, 2025
-
Multimodal models for understanding and assisting skilled activities
… their daily lives. In my dissertation, I develop multimodal models—spanning visual, audio and language modalities—that can enable learning and execution of skilled activities by 1) providing high-quality video understanding capabilities, and 2) offering strong assistance during skilled execution. …
-
Exploring cultural competence in language and multimodal models
… and its role in enhancing language and multimodal models to understand and interpret human behaviour across diverse cultural contexts. In response to the increasing need for culturally inclusive models, this work addresses three major research questions by developing new methodologies, …
-
Leveraging Multimodal Models to Detect Osteoporotic Compression Fractures
… utility. They are also based on unimodal models, or those that leverage a single data modality. Multimodal models that leverage more than one modality have shown improved performance in clinical tasks and also better reflect real-world clinical workflows.In this dissertation, I focus on …
-
Exploring screen summarization with large language and multimodal models
Mobile UIs are inherently multimodal. They can be represented by both visual representations, (screenshots) and structural metadata (view hierarchies). The image modality is rich and can contain information such as images, colors and positional information, while the view-hierarchies represent a …
-
Data-Efficient Learning for Robust Vision and Multimodal Models in Open-World Scenarios
Modern vision and multimodal foundation models are typically trained on massive datasets, requiring substantial computational and annotation resources. However, in many real-world applications, collecting and curating such large-scale data is neither scalable nor practical. Moreover, simply …
-
Adapting Deep Learning Models for Downstream Web Tasks: Multimodal Models, Task Development, Agent Adaptation
… a framework for the development of deep learning models tailored for dynamic web tasks, leveraging generalized pre-trained multimodal transformers. A task generation framework, applied to multiple web datasets, is introduced, facilitating instruction fine-tuning of models for executing multi-step …
-
Multimodal Classification of Adult-Type Diffuse Gliomas using Deep Learning on Whole-Slide Images and MRI
… imaging (MRI). In a first step, foundation models (FMs) were used to extract feature vectors from the images, which were subsequently used as input to the models that performed the final classification. Both unimodal and multimodal models were evaluated, where different multimodal fusion …
-
Captchas in the LMM Era: A Reproducible Framework for Evaluating and Designing Defenses Against LMM-Powered Solvers
The rapid advancement of Large Multimodal Models (LMMs) has challenged the security assumptions underlying traditional CAPTCHA systems, as modern models can interpret complex visual and textual information with increasing accuracy. This thesis investigates whether illusion-based CAPTCHA designs can …
-
Learning representations of visual semantics for out-of-distribution generalization
… Then we propose the hypothesis that learning multimodal knowledge representations which can be recomposed dynamically as needed is a path towards out-of-distribution (OOD) visual semantic understanding. The first focus of this thesis is the behaviour of current VSE systems. We develop a …
-
Learning without Labels - Reducing Supervision in Training, Inference, and Evaluation of Deep Neural Networks
… fixed output vocabularies from Vision Language Models by formalizing the tasks of Vocabulary-free Image Classification and Vocabulary-free Semantic Segmentation and by introducing a family of efficient methods that adapt CLIP to the tasks. We also evaluate Large Multimodal Models under a similar …
-
Self-Supervised Multimodal Representation Learning for Neuroimaging
In the pursuit of developing foundation models to understand neuropsychiatric disorders, we have to build computationally and data-efficient multimodal models while ensuring interpretability, robustness, and fairness. We have taken two significant steps toward this goal. First, we introduce a fast, …
-
Multimodal Learning from Videos: Self-supervised Pre-training, Post-training Alignment, and Benchmarks
The basis of human intelligence is inherently multimodal, relying primarily on vision, language, and audio for perception, learning, and interaction. Consequently, the emergence of human-level artificial intelligence (AI) hinges on its ability to perceive and learn from these fundamental modalities …
Page 1 of 5