Global ETD Search

Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.

Results

Showing 1 to 17 of 17 for “"efficient attention"”.

  1. Kairosformer: Hybrid Transformer for Time Series Forecasting

    … address these limitations through sparse attention mechanisms and series decomposition, but introduce trade-offs between predictive accuracy and computational efficiency. In this thesis, we propose Kairosformer, a Transformer-based architecture for long-horizon time-series forecasting that …

    mississippi Repository record for Kairosformer: Hybrid Transformer for Time Series Forecasting (opens in a new tab)

  2. Kolmogorov-Arnold Networks for Medical Image Segmentation

    … U-shaped model architecture that integrates an efficient attention mechanism with an adaptive KAN (AdaptKAN) block. The AdaKAN model consists of a convolutional stage and efficient KAN (EffiKAN) stage. The latter is comprised of an efficient attention for global feature extraction across …

    concordia Download PDF for Kolmogorov-Arnold Networks for Medical Image Segmentation (opens in a new tab)

  3. An Examination of the Relationship Between Mental Resources and Performance Strategies

    … channel would reduce subjects' ability to adopt efficient attention management strategies, whereas tasks tapping separate channels would result in more efficient strategy selection. Mixed support was found for the proposed resource channel distinction, while considerable support was found for the …

    odu Repository record for An Examination of the Relationship Between Mental Resources and Performance Strategies (opens in a new tab)

  4. Person Re-identification in Images and Videos

    … these image regions. Furthermore, we propose an efficient attention based model for person re-identifying from videos. Our method generates an attention score for each frame based on frame-level features. The attention scores of all frames in a video are used to produce a weighted feature vector …

    manitoba Repository record for Person Re-identification in Images and Videos (opens in a new tab)

  5. Cost of Computation

    … to understand how the brain balances reducing attention costs against obtaining rewards. To do this, we propose a reinforcement learning-based normative model to determine how to strategically deploy attention, and how it varies with task utility and signal statistics. Our model suggests that …

    rice Repository record for Cost of Computation (opens in a new tab)

  6. On the efficiency and privacy of foundation models

    … resource requirements and low privacy. The attention mechanism used in large language models requires quadratic time with respect to the length of the input sequence, in contrast with the remainder of the network requiring linear time. We propose a Simhash-based attention scheme which …

    ubc Repository record for On the efficiency and privacy of foundation models (opens in a new tab) Download PDF for On the efficiency and privacy of foundation models (opens in a new tab)

  7. Learning Representations for Audio

    … we mitigate this by replacing standard softmax attention with a normalized attention mechanism, reducing the cost of transformer-based encoders while preserving their ability to model long-range dependencies in spectrogram inputs. We evaluate the resulting spatial audio representations on both …

    maryland Repository record for Learning Representations for Audio (opens in a new tab)

  8. Exploring Efficient Methods for Transformer-based Foundation Language Models

    … from the computationally intensive dot-product attention mechanism, which scales quadratically with input length, and the increasing memory footprint associated with large model sizes and extensive vocabularies. These issues hinder their deployment on resource-constrained devices and limit their …

    whiterose Download PDF for Exploring Efficient Methods for Transformer-based Foundation Language Models (opens in a new tab)

  9. Models of Visual Attention in Deep Residual CNNs

    … residual learning methodologies infused with attention mechanisms to observe their effect on different tasks. To this end, we propose 3 architectures across medical image segmentation and 3D point cloud analysis. In FocusNet, we propose an attention based dual branch encoder decoder structure …

    whiterose Download PDF for Models of Visual Attention in Deep Residual CNNs (opens in a new tab)

  10. Towards Efficient and Generalizable Natural Language Processing

    … Can we build neural architectures that are both efficient and accurate? We first focus on the synergy between modern deep learning models and classical linguistic structures. We explore a surrogate gradient method that allows for incorporating discrete structured prediction as intermediate layers …

    washington Repository record for Towards Efficient and Generalizable Natural Language Processing (opens in a new tab)

  11. INTELLIGENT SURVEILLANCE FOR WEAPON THREAT DETECTION: TRANSFORMER-BASED DETECTION AND SEGMENTATION USING THERMAL AND RGB IMAGERY

    … ARMFormer, is introduced by integrating attention-driven feature refinement within a hierarchical encoder–decoder design to balance accuracy and edge efficiency. For thermal imagery, transformer-based baselines (SegFormer, SegNeXt, DeepLabV3+, and Swin Transformer) are systematically …

    siu-theses Repository record for INTELLIGENT SURVEILLANCE FOR WEAPON THREAT DETECTION: TRANSFORMER-BASED DETECTION AND SEGMENTATION USING THERMAL AND RGB IMAGERY (opens in a new tab)

  12. Adaptive Frequency Gating Networks for 3D Human Motion Prediction

    … spatial structure, and remain computationally efficient. This thesis introduces two complementary deep-learning frameworks, namely SimGate and GEAN, which share a common DCT-based embedding of joint trajectories and a gated multilayer perceptron (MLP) for spatial-frequency interaction, yet …

    concordia Download PDF for Adaptive Frequency Gating Networks for 3D Human Motion Prediction (opens in a new tab)

  13. Multicode Vibrotactile Displays to Support Mulitasking Performance in Complex Domains

    … high mental workload and the need for effective attention management, so the ability to effectively divide attention among multiple tasks and sources of task-relevant data is essential. With increasing technological advances, more and more sources of task-relevant data are being introduced in …

    tamu Repository record for Multicode Vibrotactile Displays to Support Mulitasking Performance in Complex Domains (opens in a new tab)

  14. High-Resolution Queries, Low-Resolution Context: Scaling Vision Transformers with Asymmetric Spatial Reduction

    … however, the quadratic complexity of the self-attention mechanism with respect to sequence length limits their scalability to higher resolution inputs. The attention score matrix grows as O(N2) in both compute and memory, where N is the number of patch tokens, making ViTs computationally …

    calpoly Repository record for High-Resolution Queries, Low-Resolution Context: Scaling Vision Transformers with Asymmetric Spatial Reduction (opens in a new tab)

  15. Deep Learning Based Face Image Synthesis

    … to visible face synthesis method based on selfattention generative adversarial network (SAGAN) which allows efficient attention-guided image synthesis. Rather than focusing only on synthesizing visible faces from thermal faces, we also propose to synthesize thermal faces from visible faces. Our …

    jhu Repository record for Deep Learning Based Face Image Synthesis (opens in a new tab)

  16. Efficient and Test-Time Adaptive Visual Object Tracking in the Wild

    … and IoT applications still challenging. Efficient visual trackers address the efficiency aspect of such bottlenecks; however, they tend to overfit to their training distributions and lack generalization abilities, resulting in them performing well on their respective in-distribution (ID) …

    wvu Repository record for Efficient and Test-Time Adaptive Visual Object Tracking in the Wild (opens in a new tab)

  17. Deep Attention Networks for Images and Graphs

    … convolutions and recurrent units. Recently, the attention mechanism has shown promise in serving as a new kind of fundamental building blocks. Deep attention networks (DANs), i.e. DNNs that use the attention mechanism as a fundamental building block, have revolutionized the area of natural …

    tamu Repository record for Deep Attention Networks for Images and Graphs (opens in a new tab)