Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 17 of 17 for “"efficient attention"”.
-
Kairosformer: Hybrid Transformer for Time Series Forecasting
… address these limitations through sparse attention mechanisms and series decomposition, but introduce trade-offs between predictive accuracy and computational efficiency. In this thesis, we propose Kairosformer, a Transformer-based architecture for long-horizon time-series forecasting that …
-
Kolmogorov-Arnold Networks for Medical Image Segmentation
… U-shaped model architecture that integrates an efficient attention mechanism with an adaptive KAN (AdaptKAN) block. The AdaKAN model consists of a convolutional stage and efficient KAN (EffiKAN) stage. The latter is comprised of an efficient attention for global feature extraction across …
-
An Examination of the Relationship Between Mental Resources and Performance Strategies
… channel would reduce subjects' ability to adopt efficient attention management strategies, whereas tasks tapping separate channels would result in more efficient strategy selection. Mixed support was found for the proposed resource channel distinction, while considerable support was found for the …
-
Person Re-identification in Images and Videos
… these image regions. Furthermore, we propose an efficient attention based model for person re-identifying from videos. Our method generates an attention score for each frame based on frame-level features. The attention scores of all frames in a video are used to produce a weighted feature vector …
-
Cost of Computation
… to understand how the brain balances reducing attention costs against obtaining rewards. To do this, we propose a reinforcement learning-based normative model to determine how to strategically deploy attention, and how it varies with task utility and signal statistics. Our model suggests that …
-
On the efficiency and privacy of foundation models
… resource requirements and low privacy. The attention mechanism used in large language models requires quadratic time with respect to the length of the input sequence, in contrast with the remainder of the network requiring linear time. We propose a Simhash-based attention scheme which …
-
Learning Representations for Audio
… we mitigate this by replacing standard softmax attention with a normalized attention mechanism, reducing the cost of transformer-based encoders while preserving their ability to model long-range dependencies in spectrogram inputs. We evaluate the resulting spatial audio representations on both …
-
Exploring Efficient Methods for Transformer-based Foundation Language Models
… from the computationally intensive dot-product attention mechanism, which scales quadratically with input length, and the increasing memory footprint associated with large model sizes and extensive vocabularies. These issues hinder their deployment on resource-constrained devices and limit their …
-
Models of Visual Attention in Deep Residual CNNs
… residual learning methodologies infused with attention mechanisms to observe their effect on different tasks. To this end, we propose 3 architectures across medical image segmentation and 3D point cloud analysis. In FocusNet, we propose an attention based dual branch encoder decoder structure …
-
Towards Efficient and Generalizable Natural Language Processing
… Can we build neural architectures that are both efficient and accurate? We first focus on the synergy between modern deep learning models and classical linguistic structures. We explore a surrogate gradient method that allows for incorporating discrete structured prediction as intermediate layers …
-
INTELLIGENT SURVEILLANCE FOR WEAPON THREAT DETECTION: TRANSFORMER-BASED DETECTION AND SEGMENTATION USING THERMAL AND RGB IMAGERY
… ARMFormer, is introduced by integrating attention-driven feature refinement within a hierarchical encoder–decoder design to balance accuracy and edge efficiency. For thermal imagery, transformer-based baselines (SegFormer, SegNeXt, DeepLabV3+, and Swin Transformer) are systematically …
-
Adaptive Frequency Gating Networks for 3D Human Motion Prediction
… spatial structure, and remain computationally efficient. This thesis introduces two complementary deep-learning frameworks, namely SimGate and GEAN, which share a common DCT-based embedding of joint trajectories and a gated multilayer perceptron (MLP) for spatial-frequency interaction, yet …
-
Multicode Vibrotactile Displays to Support Mulitasking Performance in Complex Domains
… high mental workload and the need for effective attention management, so the ability to effectively divide attention among multiple tasks and sources of task-relevant data is essential. With increasing technological advances, more and more sources of task-relevant data are being introduced in …
-
High-Resolution Queries, Low-Resolution Context: Scaling Vision Transformers with Asymmetric Spatial Reduction
… however, the quadratic complexity of the self-attention mechanism with respect to sequence length limits their scalability to higher resolution inputs. The attention score matrix grows as O(N2) in both compute and memory, where N is the number of patch tokens, making ViTs computationally …
-
Deep Learning Based Face Image Synthesis
… to visible face synthesis method based on selfattention generative adversarial network (SAGAN) which allows efficient attention-guided image synthesis. Rather than focusing only on synthesizing visible faces from thermal faces, we also propose to synthesize thermal faces from visible faces. Our …
-
Efficient and Test-Time Adaptive Visual Object Tracking in the Wild
… and IoT applications still challenging. Efficient visual trackers address the efficiency aspect of such bottlenecks; however, they tend to overfit to their training distributions and lack generalization abilities, resulting in them performing well on their respective in-distribution (ID) …
-
Deep Attention Networks for Images and Graphs
… convolutions and recurrent units. Recently, the attention mechanism has shown promise in serving as a new kind of fundamental building blocks. Deep attention networks (DANs), i.e. DNNs that use the attention mechanism as a fundamental building block, have revolutionized the area of natural …