Global ETD Search

Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.

Results

Showing 1 to 20 of 49 for “"Multimodal large language models"”.

  1. Towards trustworthy multimodal large language models

    Recent advances in multimodal large language models (MLLMs) have yielded remarkable capabilities in both visual understanding and content generation~\citep{liu2023llava,gpt4v,wu2024janus,openai2024gpt4ocard}. However, these progresses are accompanied by critical challenges in the domains of …

    iastate Repository record for Towards trustworthy multimodal large language models (opens in a new tab)

  2. Extending Multimodal Large Language Models Beyond Single Modalities

    Multimodal large language models (MLLMs) aim to unify diverse sensory inputs—such as text, vision, and audio—within a single reasoning framework, enabling more comprehensive and human-like understanding. However, as the number of supported modalities increases, generalist MLLMs often underperform …

    gatech Repository record for Extending Multimodal Large Language Models Beyond Single Modalities (opens in a new tab)

  3. Hallucinations in Multimodal Large Language Models: Evaluation, Mitigation, and Future Directions

    Multimodal Large Language Models (MLLMs) have achieved impressive performance across a wide array of tasks. Despite these promising developments, MLLMs often generate outputs that are inconsistent with the visual content, a challenge known as hallucination, which poses substantial obstacles to …

    maryland Repository record for Hallucinations in Multimodal Large Language Models: Evaluation, Mitigation, and Future Directions (opens in a new tab)

  4. Advancing efficiency and trustworthiness : from computer vision to multimodal large language models

    … framework for evaluating compressed models' trustworthiness, revealing that quantization generally preserves trustworthiness better than pruning at similar compression rates. Additionally, we demonstrate how pruning can enhance certified robustness verification through stability-based …

    texas Repository record for Advancing efficiency and trustworthiness : from computer vision to multimodal large language models (opens in a new tab)

  5. How Well Do Multimodal Large Language Models Score on Visual Personnel Assessments

    Multimodal Large Language Models (MLLMs) pose new threats to the validity of visual personnel assessments in high stakes selection contexts, as their emergent visual perception and understanding capabilities may facilitate applicant cheating. The study investigated the performance of three popular …

    vt Repository record for How Well Do Multimodal Large Language Models Score on Visual Personnel Assessments (opens in a new tab)

  6. DATA-EFFICIENT TRAINING OF MULTIMODAL LARGE LANGUAGE MODELS THROUGH FINE-GRAINED SELECTION AND MODALITY ANALYSIS

    Instruction tuning is crucial for aligning multimodal large language models (MLLMs) with human intent, but its effectiveness is often hindered by noisy and redundant training data. In this work, we propose ModFilter, a fine-grained, forward-only data selection method designed for MLLMs. Unlike …

    nus Repository record for DATA-EFFICIENT TRAINING OF MULTIMODAL LARGE LANGUAGE MODELS THROUGH FINE-GRAINED SELECTION AND MODALITY ANALYSIS (opens in a new tab) Download file for DATA-EFFICIENT TRAINING OF MULTIMODAL LARGE LANGUAGE MODELS THROUGH FINE-GRAINED SELECTION AND MODALITY ANALYSIS (opens in a new tab)

  7. Personalized and adaptive therapeutic music generation from biosignals using knowledge-guided multimodal large language models

    Multimodal generative models are reshaping digital therapeutics by enabling real-time synthesis of personalized content aligned with a user’s physiological and affective state. However, existing systems remain fragmented across modalities and often lack a unified framework that can jointly …

    utc Repository record for Personalized and adaptive therapeutic music generation from biosignals using knowledge-guided multimodal large language models (opens in a new tab)

  8. Video Understanding with Attention Encoder and Multimodal Large Language Model

    … increasingly significant with the emergence of Multimodal Large Language Models (MLLMs). While MLLMs have demonstrated significant promise, effectively capturing and reasoning about complex temporal dynamics and object-level interactions in videos remains an active area of research. This project …

    auckland-tech Repository record for Video Understanding with Attention Encoder and Multimodal Large Language Model (opens in a new tab)

  9. From Multimodal LLMs to Generalist Embodied Agents

    This thesis investigates how finetuning multimodal large language models (MLLMs) with large-scale embodied experience unlocks their ability to act as generalist embodied agents. While MLLMs provide reasoning abilities and broad world knowledge, finetuning aligns these capabilities with the demands …

    gatech Repository record for From Multimodal LLMs to Generalist Embodied Agents (opens in a new tab)

  10. Low-Resource Ecoacoustic Audio Classification

    … exploring advanced AI, we critically evaluate Multimodal Large Language Models (MLLMs) using mechanistic interpretability, revealing fundamental limitations in audio-text reasoning. Collectively, these contributions offer validated methods, unique datasets, and critical insights for robust …

    cuny-grad Repository record for Low-Resource Ecoacoustic Audio Classification (opens in a new tab)

  11. Multimodal LLM Using Federated Visual Instruction Tuning for Visually Impaired

    Training multimodal large language models (LLMs) is computationally intensive often requiring thousands of GPU hours and poses challenges in resource allocation and efficiency. Additionally, the centralized handling of training data in traditional approaches poses privacy risks, particularly when …

    buffalo Repository record for Multimodal LLM Using Federated Visual Instruction Tuning for Visually Impaired (opens in a new tab)

  12. Interpreting Deep Learning Models and Unlocking New Applications With It

    … across various domains, including natural language processing, computer vision, and speech recognition. These advancements have been driven by innovations in scaling pre-training data, developing new model architectures, integrating distinct modalities (e.g., vision and language, audio and …

    maryland Repository record for Interpreting Deep Learning Models and Unlocking New Applications With It (opens in a new tab)

  13. Exploring the effects of multimodal features on a machine learning knowledge tracker

    … channels of information exchange. Spoken language is the most common, but non-verbal cues such as gestures, body pose, and movements also play a role. These channels carry semantic information but are discrete and harder for machines to detect. Recent advances in multimodal Large Language …

    colostate Repository record for Exploring the effects of multimodal features on a machine learning knowledge tracker (opens in a new tab)

  14. GlitchAgent: Detecting Video Game Glitches from Gameplay Videos

    … The videos are annotated with detailed, natural-language glitch descriptions and precise temporal timestamps, created through a semi-automated pipeline leveraging Multimodal Large Language Models (MLLMs) and human validation. Second, we propose GlitchAgent, a multi-stage framework for open-ended …

    vt Repository record for GlitchAgent: Detecting Video Game Glitches from Gameplay Videos (opens in a new tab)

  15. LLM-Powered Question Answering for Object States in Virtual Reality

    Recent advances in large language models (LLMs) and multimodal large language models (MLLMs) enable natural language–based querying in virtual reality (VR). However, VR environments are highly localized, personalized, and dynamic, making it challenging for general-purpose models to answer …

    kennesaw Repository record for LLM-Powered Question Answering for Object States in Virtual Reality (opens in a new tab)

  16. Towards an AI-Enabled Metaverse: Architecture, Resource Allocation, and Multimodal Benchmarking

    … the scene understanding capabilities of Multimodal Large Language Models (MLLMs). Thirteen representative MLLMs are assessed through a pairwise comparison protocol under an MLLM-as-a-judge paradigm guided by metaverse-specific semantic criteria. Human expert validation confirms the …

    ottawa-retro Repository record for Towards an AI-Enabled Metaverse: Architecture, Resource Allocation, and Multimodal Benchmarking (opens in a new tab)

  17. Grounding in multimodal foundation models: evaluating and improving modality-specific reasoning and generation

    Multimodal foundation models aim to integrate information across vision, audio, and language, yet current systems often fail to ground their outputs in the modality that actually contains the relevant evidence. Instead, they tend to rely on dominant priors, particularly textual cues or vision …

    bu Repository record for Grounding in multimodal foundation models: evaluating and improving modality-specific reasoning and generation (opens in a new tab)

  18. Transferable and Scalable Vision-Language Models for Reasoning and Grounding

    Vision-language pre-trained (VLP) systems have demonstrated strong performance by learning transferable visual representations through language supervision. These models redefine the training paradigm for large visual systems by leveraging weakly paired image- and video-text data, eliminating the …

    jhu Repository record for Transferable and Scalable Vision-Language Models for Reasoning and Grounding (opens in a new tab)

  19. Data-Efficient Learning for Robust Vision and Multimodal Models in Open-World Scenarios

    Modern vision and multimodal foundation models are typically trained on massive datasets, requiring substantial computational and annotation resources. However, in many real-world applications, collecting and curating such large-scale data is neither scalable nor practical. Moreover, simply …

    jhu Repository record for Data-Efficient Learning for Robust Vision and Multimodal Models in Open-World Scenarios (opens in a new tab)

  20. Improving Vulnerable Road Users' Safety Through Computer Vision-Based Crossing Direction Prediction And Multimodal Large Language Model-Based Accident Scene Description

    … prediction and presents VRU-Accident, a large-scale benchmark enabling systematic assessment of multimodal large language models (MLLMs) in understanding accident scenarios involving VRUs. First, the proposed framework leverages multi-modal cues, including 3D pose estimation and …

    ucf Repository record for Improving Vulnerable Road Users' Safety Through Computer Vision-Based Crossing Direction Prediction And Multimodal Large Language Model-Based Accident Scene Description (opens in a new tab)

Page 1 of 3