Global ETD Search

Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.

Results

Showing 1 to 20 of 29 for “"KV Cache"”.

  1. Dynamic KV cache management for efficient LLM inference

    … memory-intensive. To accelerate generation, LLMs cache previously computed information in a Key-Value (KV) cache. As task complexity and sequence lengths grow, the KV cache becomes the primary memory bottleneck for LLM inference, limiting their ability to scale. In this thesis, we study the …

    texas Repository record for Dynamic KV cache management for efficient LLM inference (opens in a new tab)

  2. End-to-End KV-Cache Materialization and Evaluation

    Key/value (KV) caching is essential for efficient Transformer inference, but its size grows with context length and model depth, creating bottlenecks in memory capacity and data movement. In distributed and multi-tier serving, these overheads directly affect Time-to-First-Token (TTFT), making …

    tamu Repository record for End-to-End KV-Cache Materialization and Evaluation (opens in a new tab) Embargoed until 2028-05-01

  3. Disaggregated LLM Serving with CXL Shared Memory KV Cache at Rack-Scale

    … but introduces a new bottleneck: key/value (KV) tensors generated during prefill must be transferred to decode workers, typically through the network stack. CXL memory offers an attractive alternative by enabling multiple hosts to share a large byte-addressable memory pool, allowing KV blocks …

    vt Repository record for Disaggregated LLM Serving with CXL Shared Memory KV Cache at Rack-Scale (opens in a new tab)

  4. Pushing the limits of long context LLM inference via KV cache compression

    Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-19 without embargo terms

    uiuc Repository record for Pushing the limits of long context LLM inference via KV cache compression (opens in a new tab)

  5. Ghostdecoding: leveraging random-feature kernels for error-aware and training-free KV cache selection

    Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2027-05-01

    uiuc Repository record for Ghostdecoding: leveraging random-feature kernels for error-aware and training-free KV cache selection (opens in a new tab)

  6. Low-rank KV cache compression in LLMs: A second-order output metric and its limits

    The key–value (KV) cache makes autoregressive decoding in large language models fast, but it grows with the length of the context. At long context it becomes the main drain on a GPU’s memory, both in how much it has to store and in how fast it can be read. Compressing the keys and values with a …

    iastate Repository record for Low-rank KV cache compression in LLMs: A second-order output metric and its limits (opens in a new tab)

  7. Adaptive KV Cache Management for Efficient Transformer-based LLM Inference - Leveraging Attention Sparsity for Memory Optimization

    … prohibitive memory footprint of the Key-Value (KV) cache. As LLMs scale, the KV cache becomes a significant bottleneck, limiting longer context windows and overall operational efficiency. To mitigate this issue, we propose and evaluate Adap-KV, a novel adaptive memory management strategy for the …

    chalmers Repository record for Adaptive KV Cache Management for Efficient Transformer-based LLM Inference - Leveraging Attention Sparsity for Memory Optimization (opens in a new tab)

  8. Efficient Long-Context Retrieval Systems via Dynamic Sparse Attention

    … corpus scale and context length, the key-value (KV) cache makes serving them impractical due to its linear growth with sequence length and batch size. Serving systems like RetroInfer use dynamic sparse attention (DSA) to address that bottleneck, offloading the cache and attending at each decode …

    ethz Repository record for Efficient Long-Context Retrieval Systems via Dynamic Sparse Attention (opens in a new tab)

  9. Quantized Lookup Table-Based Representation for Key-Value Cache

    … bottleneck: the transmission of Key-Value (KV) caches between devices to enable cache reuse. As the demand for long-context inference grows, KV caches can become extremely large, making efficient compression essential to reduce communication costs. Interestingly, we observe that KV caches …

    tamu Repository record for Quantized Lookup Table-Based Representation for Key-Value Cache (opens in a new tab) Embargoed until 2028-05-01

  10. Quantized Lookup Table-Based Representation for Key-Value Cache

    … bottleneck: the transmission of Key-Value (KV) caches between devices to enable cache reuse. As the demand for long-context inference grows, KV caches can become extremely large, making efficient compression essential to reduce communication costs. Interestingly, we observe that KV caches …

    tamu Repository record for Quantized Lookup Table-Based Representation for Key-Value Cache (opens in a new tab) Embargoed until 2028-05-01

  11. Quantized Lookup Table-Based Representation for Key-Value Cache

    … bottleneck: the transmission of Key-Value (KV) caches between devices to enable cache reuse. As the demand for long-context inference grows, KV caches can become extremely large, making efficient compression essential to reduce communication costs. Interestingly, we observe that KV caches …

    tamu Repository record for Quantized Lookup Table-Based Representation for Key-Value Cache (opens in a new tab) Embargoed until 2028-05-01

  12. The Lookahead Transformer

    … stage processes all input tokens in parallel and caches them into the KV cache. The decode stage uses the KV cache, processes a single token at a time, and is generally memory-bandwidth constrained. Each decode step generates a token probability distribution, samples from it, appends the token to …

    byu Repository record for The Lookahead Transformer (opens in a new tab) Download PDF for The Lookahead Transformer (opens in a new tab)

  13. Memory-Efficient and Scalable Parallel Approaches for High-Throughput Machine Learning Systems

    … long-sequence decoding, where Key-Value (KV) cache access can account for over 80% of per-token latency. Existing parallelism strategies, including data, tensor, and pipeline parallelism, either expose limited intra-layer parallelism or rely on redundant replication of model weights and KV …

    tamu Repository record for Memory-Efficient and Scalable Parallel Approaches for High-Throughput Machine Learning Systems (opens in a new tab)

  14. Memory-Efficient and Scalable Parallel Approaches for High-Throughput Machine Learning Systems

    … long-sequence decoding, where Key-Value (KV) cache access can account for over 80% of per-token latency. Existing parallelism strategies, including data, tensor, and pipeline parallelism, either expose limited intra-layer parallelism or rely on redundant replication of model weights and KV …

    tamu Repository record for Memory-Efficient and Scalable Parallel Approaches for High-Throughput Machine Learning Systems (opens in a new tab)

  15. Extracting Hidden, Protected, and Deleted User Interaction Contexts from Applications and AI Systems

    … by extracting and realigning batch tokens and KV cache state so investigators can interrogate a live session with recovered context.

    gatech Repository record for Extracting Hidden, Protected, and Deleted User Interaction Contexts from Applications and AI Systems (opens in a new tab)

  16. EFFECTIVE COMPRESSION FOR SCALABLE VISION GENERATIVE MODELS

    … tokenizer capacity, context length and cache size, and diffusion sampling steps. First, SeQ-GAN introduces semantic compression, prioritizing high-level structure and texture over pixel-perfect reconstruction to create a latent space that generative transformers can model more …

    nus Repository record for EFFECTIVE COMPRESSION FOR SCALABLE VISION GENERATIVE MODELS (opens in a new tab) Download file for EFFECTIVE COMPRESSION FOR SCALABLE VISION GENERATIVE MODELS (opens in a new tab)

  17. Energy-Efficient Hardware Acceleration of Transformer-Based Models

    … activations, the proposed Weight-Hessian-aware KV cache quantization applies intra-layer mixed-precision using precomputed sensitivities, eliminating runtime overhead. To further improve hardware efficiency, the proposed Query-Key coupled scheme aligns bit precision within each outer product …

    gatech Repository record for Energy-Efficient Hardware Acceleration of Transformer-Based Models (opens in a new tab)

  18. Compression Algorithms for Efficient Inference and Adaptation of Foundation Models

    … a method for compressing the key/value (KV) cache in transformer models. We observe that KV cache channels exhibit significant redundancy, which existing quantization methods fail to exploit. Coupled Quantization jointly compresses multiple channels by leveraging their interdependence, …

    rice Repository record for Compression Algorithms for Efficient Inference and Adaptation of Foundation Models (opens in a new tab)

  19. Linear recurrent models for robust and interpretable long-context modeling

    … of Transformers during training and linear KV-cache growth at inference makes them computationally prohibitive for such tasks, necessitating the exploration of more efficient architectures. Linear-time recurrent models and hybrid architectures, which combine linear recurrence with local …

    cornell Repository record for Linear recurrent models for robust and interpretable long-context modeling (opens in a new tab)

  20. On the Resource Efficiency of Language Models

    … distributions. LoRC progressively compresses KV cache, substantially reducing GPU memory requirements while preserving performance. These complementary thrusts form a framework for efficient, deployable language models. The findings demonstrate that targeted resource optimization enables …

    gatech Repository record for On the Resource Efficiency of Language Models (opens in a new tab)

Page 1 of 2