Global ETD Search

Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.

Results

Showing 1 to 20 of 33 for “"sparsity patterns"”.

  1. Sparse Attention Mechanisms in Vision Transformers: from Spatiotemporal Transformers to Instance-Dependent Attention Patterns

    … this dissertation we investigated methods using sparsity and conditional computation to meet this compute demand. The two main components of this contribution involve applying structure and dynamic prediction of sparsity patterns by machine learning algorithms. The hypothesis was that structured …

    toronto-retro Repository record for Sparse Attention Mechanisms in Vision Transformers: from Spatiotemporal Transformers to Instance-Dependent Attention Patterns (opens in a new tab)

  2. Structured Sparsity Learning for Coevolution-Based Protein Contact Prediction

    … This is correct but not the whole picture since patterns of predicted 2D contacts also play a significant impact on 3D structure reconstruction. For example, contacts between long-distance residue pairs in general help more than adjacent residue pairs do. Moreover, current methods always get …

    tamu Repository record for Structured Sparsity Learning for Coevolution-Based Protein Contact Prediction (opens in a new tab)

  3. Compiler and Runtime Systems for Generative AI Models

    … variable sequence lengths and irregular sparsity patterns--and (2) they evolve at a rapid pace, with shifting model architectures and changing deployment requirements. This dissertation addresses these challenges through a co-design approach spanning both compiler and runtime layers, …

    washington Repository record for Compiler and Runtime Systems for Generative AI Models (opens in a new tab)

  4. Adaptive KV Cache Management for Efficient Transformer-based LLM Inference - Leveraging Attention Sparsity for Memory Optimization

    … in real-time, leveraging insights from attention sparsity patterns. Our method aims to optimize memory utilization without compromising the performance or quality of LLM inference. Experimental results demonstrate that Adap-KV significantly reduces KV cache memory consumption, thereby enhancing …

    chalmers Repository record for Adaptive KV Cache Management for Efficient Transformer-based LLM Inference - Leveraging Attention Sparsity for Memory Optimization (opens in a new tab)

  5. Recovering Data with Group Sparsity by Alternating Direction Methods

    Group sparsity reveals underlying sparsity patterns and contains rich structural information in data. Hence, exploiting group sparsity will facilitate more efficient techniques for recovering large and complicated data in applications such as compressive sensing, statistics, signal and image …

    rice Repository record for Recovering Data with Group Sparsity by Alternating Direction Methods (opens in a new tab)

  6. Algorithm–Hardware Co-Design Of Digital Compute-In-Memory Architecture Supporting Flexible And Temporal N:M Sparsity

    … for efficient inference, with structured N:M sparsity emerging as a promising method to reduce overhead. However, fixed sparsity configurations across layers and decoding steps can limit model expressivity and degrade accuracy, while supporting multiple configurations introduces challenges in …

    gatech Repository record for Algorithm–Hardware Co-Design Of Digital Compute-In-Memory Architecture Supporting Flexible And Temporal N:M Sparsity (opens in a new tab)

  7. Optimizing sparse computations using union of Z-polyhedra

    … sparse matrix techniques aim to exploit this sparsity to reduce both storage and computational cost, thereby improving scalability and efficiency. The Union of Z-Polyhedra (UZP) sparse format models sparse structures as unions of integer polyhedra intersected with affine lattices, capturing …

    colostate Repository record for Optimizing sparse computations using union of Z-polyhedra (opens in a new tab)

  8. Performance Primitives for Artificial Neural Networks

    … decades ago, and do not necessarily reflect the patterns of computations in neural networks. I propose modifications to common primitives provided by performance libraries to make them better building blocks for artificial neural networks, with a focus on inference, i.e. evaluation of a …

    gatech Repository record for Performance Primitives for Artificial Neural Networks (opens in a new tab)

  9. Automated Implementation of Advanced Electronic Structure Methods

    … to reduced-scaling methods, which exploit the sparsity inherent in electronic correlation. We introduce a novel "tensor-of-tensors" data structure designed to manage the irregular sparsity patterns of Pair Natural Orbital (PNO) formulations. This development enables the first fully automated …

    vt Repository record for Automated Implementation of Advanced Electronic Structure Methods (opens in a new tab)

  10. 3D Scene Reconstruction from A Monocular Image

    … of the system, including robustness to various sparsity patterns and diverse scenes.

    adelaide Repository record for 3D Scene Reconstruction from A Monocular Image (opens in a new tab)

  11. Trustworthy Machine Learning: Specification, Verification, and Explanation

    … By identifying and exploiting chordal sparsity patterns in large constraints, we obtain speedups of several orders of magnitude, making SDP-based methods more practical for modern networks. Finally, we address the pressing need for reliable insights into complex model behavior through …

    penn Repository record for Trustworthy Machine Learning: Specification, Verification, and Explanation (opens in a new tab)

  12. Generalizing smoothed aggregation-based algebraic multigrid

    … coarsening and the choice of interpolation sparsity patterns. Unfortunately, the classic strength measure is only well-founded for M-matrices, leading us to develop a new measure based on local knowledge of both algebraically smooth error and the behavior of interpolation. Another limitation …

    uiuc Repository record for Generalizing smoothed aggregation-based algebraic multigrid (opens in a new tab)

  13. Discovery of low-dimensional structure in high-dimensional inference problems

    … number of samples required to reliably recover sparsity patterns independent of any specific recovery algorithm. In particular, we show that sample complexity can be tightly characterized using a mutual information formula similar to channel coding results. Next, we derive major extensions to …

    bu Repository record for Discovery of low-dimensional structure in high-dimensional inference problems (opens in a new tab)

  14. Structured sparse models with applications

    … domain. From a data modeling point of view, sparsity can be seen as a form of regularization, that is, as a device to restrict or control the set of coefficient values which are allowed in the model to produce an estimate of the data. In this way, flexibility of the model (that is, the …

    umn Repository record for Structured sparse models with applications (opens in a new tab)

  15. Sparse Support Matrix Machines for the Classification of Corrupted Data

    … term which promotes the structural sparsity and shares similar sparsity patterns across multiple predictors. It is a spectral extension of the conventional elastic net that combines the property of low-rank and joint sparsity together, to deal with complex high dimensional noisy …

    uts Repository record for Sparse Support Matrix Machines for the Classification of Corrupted Data (opens in a new tab)

  16. Using reconfigurable computing technology to accelerate matrix decomposition and applications

    … and asymmetric sparse matrices with arbitrary sparsity patterns.</p> <p>• By further extending the proposed hardware solution for SVD, we parallelize a popular text mining tool-Latent Semantic Indexing with an FPGA-based architecture.</p> <p>• We present a configurable architecture to …

    iastate Repository record for Using reconfigurable computing technology to accelerate matrix decomposition and applications (opens in a new tab)

  17. Μελέτη υπολογιστικών μεθόδων για παράλληλη προσομοίωση του βαρυτικού προβλήματος Ν-σωμάτων

    … factorization, utilizing approximate inverse sparsity patterns. Moreover, a new hybrid Two-Level Algebraic Symmetric Factored Approximate Inverse solver (TASFAI), based on the SFASI matrix, is proposed for solving sparse symmetric linear systems. Additionally, an explicit mesh refinement …

    greece Repository record for Μελέτη υπολογιστικών μεθόδων για παράλληλη προσομοίωση του βαρυτικού προβλήματος Ν-σωμάτων (opens in a new tab)

  18. Lower-scaling advanced correlation methods with local Resolution of Identity strategies

    … parallelization by means of à priori known sparsity patterns. In combination with established integral screening techniques, the computational scaling can also be reduced considerably. The potential of this new approach is demonstrated at the example of an in-memory LT-MP2 with a MPI/OpenMP …

    fu-berlin Repository record for Lower-scaling advanced correlation methods with local Resolution of Identity strategies (opens in a new tab)

  19. Novel Regression Models For High-Dimensional Survival Analysis

    … employ the elastic net penalty term for inducing sparsity into the linear model to effectively handle high-dimensional data. As opposed to the existing censored linear models, the parameter estimation of our model does not need any prior estimation of survival times of censored instances. The …

    wayne-thes Repository record for Novel Regression Models For High-Dimensional Survival Analysis (opens in a new tab)

Page 1 of 2