Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 33 for “"LLM Inference"”.
-
Prompt enrichment for Geospatial LLM inference
… συμπερασμάτων από Μεγάλα Γλωσσικά Μοντέλα (LLM) για εργασίες με χωρική επίγνωση. Η παρούσα διατριβή αναπτύσσεται ως βελτίωση του πειράματος της δημοσίευσης του Manvi [12] που αξιοποιεί τα LLMs για εξαγωγή συμπερασμάτων σε γεωχωρικά δεδομένα με τη χρήση prompts προσαρμοσμένων για αυτό το …
-
Durable, Distributed LLM Inference on COTS Devices
The advancement of Large Language Models (LLMs) has fundamentally changed the nature of natural language processing. The substantial memory requirements of frontier models creates a significant barrier to entry, centralizing inference. This thesis presents the design and implementation of a …
-
Analyzing Factors Influencing Performance in LLM Inference Systems
Large Language Models (LLMs) lead to numerous innovative applications, including virtual assistants, content generation, and recommendation systems, which made a revolution in daily life. LLM inference is crucial as it allows these models to process and generate human-like text in real-time, making …
-
Profiling and Visualizing CPU-Based LLM Inference in llama.cpp
Large language models (LLMs) are increasingly deployed on local hardware rather than through cloud services, motivated by concerns around data privacy, network dependence, and operational cost. In many local deployments, the CPU serves as the primary or sole compute resource, since dedicated …
-
Towards Green AI: Evaluating Energy Efficiency for Quantized LLM Inference
… that quantization of large language models (LLMs) reduces memory usage via lower-bit weights, studies quantifying the resulting impact on energy usage and carbon intensity are scarce. We present the first unified evaluation of weight-only quantization strategies for LLM inference across …
-
Pushing the limits of long context LLM inference via KV cache compression
Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-19 without embargo terms
-
Domain-Specific Design on FPGA and NPU for scientific computing and On-Device LLM Inference
… scientific simulations and large language model (LLM) workloads push the need for higher performance under tight power and memory constraints. Conventional platforms such as CPUs and GPUs accelerate computation through instruction-driven parallelism, relying on multithreading, SIMD, and SIMT …
-
MULTI-ARMED BANDIT STRATEGIES FOR RISK-AVERSE DECISION-MAKING, NON-STATIONARY ENVIRONMENTS, AND EFFICIENT LLM INFERENCE
… application of the MAB framework to enhance the inference efficiency and throughput of modern Large Language Models LLMs. These problems capture the trade-off between exploration, exploitation, and X, which can be safety, non-stationarity or efficiency.
-
Adaptive KV Cache Management for Efficient Transformer-based LLM Inference - Leveraging Attention Sparsity for Memory Optimization
… in Transformerbased Large Language Models (LLMs) during inference, specifically focusing on the prohibitive memory footprint of the Key-Value (KV) cache. As LLMs scale, the KV cache becomes a significant bottleneck, limiting longer context windows and overall operational efficiency. To …
-
Accelerating LLM Inference via ANN-Based Draft Model Construction - Speculative decoding with dense and FlashHead draft models
Large language model inference is limited by the sequential nature of autoregressive decoding. Speculative decoding reduces this cost by using a smaller draft model to propose several candidate tokens, which are then verified by a larger target model. However, this shifts part of the work to the …
-
High-Performance Computational Kernels for Algorithm-System Co-Design in Machine Learning: Enabling Efficient GCN Training and LLM Inference Serving.
… to transformer-based Large Language Models (LLMs), continues to expose fundamental limitations in memory efficiency, scalability, and system reliability. GCNs are critical for domains such as biomedical modeling, social networks, and recommendation systems, yet their reliance on sparse …
-
Mitigating the Impact of Data Movement in Memory-Intensive Applications
… targets the GPU memory bandwidth bottleneck in LLM inference, taking advantage of the complementary resource profiles of the two phases of LLM inference, prefill and decode. It fuses compute-bound prefill and memory-bound decode so that they execute concurrently—the prefill operations occupy the …
-
BREAKING MEMORY WALLS: EFFICIENT MEMORY MANAGEMENT FOR THE POST-MOORE’S ERA
… applications like large language models (LLMs) continues to rise. This thesis addresses four major “memory walls” limiting modern systems: translation, capacity, bandwidth, and archival. To overcome the memory translation wall in NUMA systems, Hydra introduces scalable page table …
-
Methodologies and Architectures for AI Inference Hardware: From Foundational Networks to Large Language Models
The rapid advancements in large language models (LLMs) have significantly reshaped the artificial intelligence landscape, enabling transformative applications. However, these developments pose profound challenges for hardware architectures, particularly concerning performance, efficiency, and …
-
Compiler and Runtime Systems for Generative AI Models
… engine tailored for large language model (LLM) inference. As attention increasingly dominates computational costs in modern GenAI models, scalable and customizable GPU kernels become essential. FlashInfer supports block-sparse KV-cache layouts, Just-In-Time (JIT) compilation of …
-
Memory Access Characterization of Large Language Models in CPU Environment and Its Potential Impacts
… accordingly. Oftentimes, it is infeasible to run inference with larger models without an accelerator, which may be unavailable in environments that have constraints such as energy consumption, security, or cost. To increase the availability of these models, we aim to improve the LLM inference …
-
Runtime Configuration of GPU Workloads for Energy-efficient Execution
… through experiments with large language model (LLM) inference workloads and the model is shown to closely approximate the energy consumption trends of real-world workloads. This demonstrates that the given rules can serve as lightweight yet effective guidance for improving the energy efficiency …
-
GOFinder-AI: Rapid and Explainable Gene Ontology Term Assignment Using Large Language Models
… pre-query text mining and large language model (LLM) inference. Given a biomedical text, the system identifies candidate GO annotations and produces supporting citations, explanatory reasoning, and linked biological entities. To improve task-specific performance, we fine-tuned multiple …
-
Efficient inference and adaptation in deep learning : from edge systems to large language models
… problem.This thesis investigates efficient inference, training, and adaptation in DL across the model lifecycle from four complementary perspectives: (1) enabling lightweight and energy-efficient inference of DL models on edge devices and distributed Internet of Things (IoT) systems; (2) …
-
Improving Collective Aggregation Strategies for HPC and AI Workloads
… domains: large-scale HPC checkpointing, LLM training, and LLM inference. We first show that collective I/O techniques designed for synchronous checkpointing are poorly suited to asynchronous execution due to their reliance on globally coordinated synchronization. Through an extensive …
Page 1 of 2