Global ETD Search

Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.

Results

Showing 1 to 15 of 15 for “"Multimodal LLMs"”.

  1. From Multimodal LLMs to Generalist Embodied Agents

    This thesis investigates how finetuning multimodal large language models (MLLMs) with large-scale embodied experience unlocks their ability to act as generalist embodied agents. While MLLMs provide reasoning abilities and broad world knowledge, finetuning aligns these capabilities with the demands …

    gatech Repository record for From Multimodal LLMs to Generalist Embodied Agents (opens in a new tab)

  2. Conversational Multimodal LLMs for Food Nutritional Information Retrieval: A Systematic Evaluation

    … systematically evaluates eight state-of-the-art multimodal large language models (including GPT-4o, Qwen 2, and Qwen 2.5, DeepSeek, and LLaVA), in a zero-shot setting on two real-world datasets, Nutrition5k and MetaFood3D. To identify which types of information most improve model predictions, a …

    vt Repository record for Conversational Multimodal LLMs for Food Nutritional Information Retrieval: A Systematic Evaluation (opens in a new tab)

  3. Multimodal LLM Using Federated Visual Instruction Tuning for Visually Impaired

    Training multimodal large language models (LLMs) is computationally intensive often requiring thousands of GPU hours and poses challenges in resource allocation and efficiency. Additionally, the centralized handling of training data in traditional approaches poses privacy risks, particularly when …

    buffalo Repository record for Multimodal LLM Using Federated Visual Instruction Tuning for Visually Impaired (opens in a new tab)

  4. Investigating the Use of Physiological and Behavioral Signals to Facilitate Empathic Human-AI Interaction for Daily Stress Management

    … evaluation of Empathic Large Language Models (EmLLMs) for general mental health support. EmLLMs use physiological and behavioral signals to infer users' mental states (affective and cognitive) and accordingly generate empathic messages as adaptive interventions. Three core research goals guided …

    vt Repository record for Investigating the Use of Physiological and Behavioral Signals to Facilitate Empathic Human-AI Interaction for Daily Stress Management (opens in a new tab)

  5. Towards Multimodal Scene Graph Generation Approaches to Video Understanding

    … and the utilization of Large Language Models (LLMs) for scene graph reasoning. To this end, three novel datasets and corresponding approaches are introduced. First, the ASPIRe dataset incorporates interactivity annotations and leverages the Hierarchical Interlacement Graph (HIG) for …

    arkansas Repository record for Towards Multimodal Scene Graph Generation Approaches to Video Understanding (opens in a new tab)

  6. Semantic Ink: Intent-Based Graphics Authoring through Annotation-Driven Generative User Interfaces and Reusable Tools

    … magnitude, spatialization, surface) to enable multimodal LLMs to parse ambiguous annotations into editable parameter spaces. A user study with six experts demonstrates that Semantic Ink improves editing efficiency, reduces cognitive load from mode-switching, and enhances personalization through …

    waterloo Repository record for Semantic Ink: Intent-Based Graphics Authoring through Annotation-Driven Generative User Interfaces and Reusable Tools (opens in a new tab)

  7. M3EC: A Multimodal Multidocument Benchmark for Event Extraction and Coreference Resolution

    … remains largely unexplored. We introduce M3EC (MultiModal Multidocument Event and Coreference), a new benchmark for multimodal, multidocument event extraction and coreference resolution. It is built from tightly related news clusters that combine eventful videos, audio files, and news articles. …

    vt Repository record for M3EC: A Multimodal Multidocument Benchmark for Event Extraction and Coreference Resolution (opens in a new tab)

  8. Towards Robust Control in Visual Generation and Manipulation

    … all conditional image synthesis tasks with multimodal LLMs, tickling the scalability issue of ImagenHub. The second study focuses on the video domain, which introduces AnyV2V, the first framework to treat video editing as an image editing problem. It leverages the editing power from …

    waterloo Repository record for Towards Robust Control in Visual Generation and Manipulation (opens in a new tab)

  9. Bridging Perception and Reasoning in Multimodal Models

    Most human knowledge is acquired through rich, multimodal cognitive experiences, where both vision and language play essential roles. For a long time, computer vision (CV) has aimed to reconstruct the physical world from sensory data, while natural language processing (NLP) has seeked to achieve …

    penn Repository record for Bridging Perception and Reasoning in Multimodal Models (opens in a new tab)

  10. Reliable Medical LLM and Vision-Language RAG through Multi-Agent Orchestration and Single-Step Preference Alignment

    … Retrieval Augmented Generation in the context of Multimodal Medical LLMs has not been given due consideration from the lens of hallucination mitigation. Further, the existing approaches have been limited in their coverage of the medical domains, often limited to X-Ray. Medical Multimodal LLMs, …

    rice Repository record for Reliable Medical LLM and Vision-Language RAG through Multi-Agent Orchestration and Single-Step Preference Alignment (opens in a new tab) Embargoed until 2030-12-01

  11. Extending Multimodal Large Language Models Beyond Single Modalities

    Multimodal large language models (MLLMs) aim to unify diverse sensory inputs—such as text, vision, and audio—within a single reasoning framework, enabling more comprehensive and human-like understanding. However, as the number of supported modalities increases, generalist MLLMs often underperform …

    gatech Repository record for Extending Multimodal Large Language Models Beyond Single Modalities (opens in a new tab)

  12. Artificial Intelligence-empowered bio-medical applications

    … Pre-trained Transformer 4 with Vision) using multimodal multiple-choice questions in the domain of General Pathology. In addition, the proposed methodology enables the extraction of requirements for targeted interventions and fine-tuning of state-of-the-art models to optimize their performance …

    greece Repository record for Artificial Intelligence-empowered bio-medical applications (opens in a new tab)

  13. Grounding language in image, video and audio modalities

    The sub-field of multimodal learning in artificial intelligence involves the problem of jointly learning representations of the audio, language and visual modalities for the purpose of automated applications including virtual assistants, chatbots and robotics. In particular, learning a multimodal …

    bu Repository record for Grounding language in image, video and audio modalities (opens in a new tab)

  14. A Framework for AI-Assisted Generation of HAZOP Studies’ Reports in Process Industries

    … used for similarity computation. Six multimodal large language models (LLMs)—GPT-4o, GPT-4o-mini, Gemini, LlaMa, DeepSeek, and Qwen—were employed to generate node descriptions and assist in generating HAZOP notes. Topic modeling techniques, including BERTopic, Latent Dirichlet …

    concordia Download PDF for A Framework for AI-Assisted Generation of HAZOP Studies’ Reports in Process Industries (opens in a new tab)