Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 300 for “"Self-attention"”.
-
Self-attention policy architectures for reinforcement learning under partial observability
… which necessarily constitutes noise. We explore self-attention-based policy architectures as a solution to this problem, demonstrating their robustness under conditions of high partial observability on different rein-forcement learning benchmark tasks, and explore the advantages and disadvantages …
-
Applications of Diversity and the Self-Attention Mechanism in Neural Networks
… the other two cover novel applications of the self-attention mechanism. An important aspect of training a neural network is the choice of objective function. Regression via Classification (RvC) is often used to tackle problems in deep learning where the target variable is continuous, but …
-
The Effects of Expectancy, Task Importance and Self-Attention on Task Persistence
… power of control theory (Carver, 1979) and self-efficacy theory (Bandura, 1977) in predicting behavioral persistence. This study employed a 2 X 2 X 2 X 2 factorial design. One hundred and twelve undergraduate females were exposed to high and low levels of self-attention and task importance …
-
On the Efficiency and Steerability of Self-Attention Mechanism of Large Language Models
… language processing through their powerful self-attention mechanisms. In this thesis, we focus on two crucial aspects of self-attention: efficiency and steerability, and explore innovative prompting techniques to improve contextual understanding and complex reasoning. Mixed attention span …
-
Towards understanding self-attention and batch normalization : optimization and margin dynamics in neural networks
… understanding of transformers and their core, self-attention mechanism remains incomplete, especially with regard to their optimization and generalization behaviors. Previous research has predominantly focused on single-head attention, thereby limiting insights into the impact of …
-
Revolutionizing Time Series Data Preprocessing with a Novel Cycling Layer in Self-Attention Mechanisms
… by incorporating a cycling layer into self-attention mechanisms. Traditional techniques often struggle to capture the cyclical nature of time series data, impacting predictive model accuracy. By integrating a cycling layer, this thesis aims to enhance the ability of models to recognize …
-
On the optimization and generalization of self-attention models : a stability and implicit bias perspective
… a foundational theory. Specifically, we focus on self-attention models, the core mechanism of the Transformer. We first study multi-head attention models in a binary classification setting, developing optimization and generalization bounds that elucidate the finite-time behavior of these models …
-
Hybrid Deep Learning Model: LLM-Enhanced Linear Self-Attention Transformer-CNN for Stock Price Prediction
… pipelines. One pipeline utilizes an optimized attention-based Transformer to process the LLM-generated embeddings, capturing long-range market dependencies. Concurrently, the second pipeline deploys a Convolutional Neural Network (CNN) to analyze visual stock chart patterns, extracting …
-
DSA-BEATS: Multi step, multi-variate dual self-attention N-BEATS model for forecasting COVID-19 hospitalization
… the spread. In this study, we present the Dual Self-Attention N-BEATS (DSA-BEATS) model, a novel approach that effectively combines the self-attention mechanism of transformers with the proficiency of the N-BEATS model in dealing with multi-variate forecasting problems. The model has been …
-
Theoretical Expressivity of Self-Attention as a Model of Binary Sequence-To-Sequence Functions and Other Explorations
The self-attention mechanism lies at the heart of modern sequential modeling with deep neural networks, especially in the context of Large Language Models (LLMs). The basic role of self-attention is to capture pairwise interactions between tokens in a sequence. This allows token embeddings to be …
-
Glioma Segmentation in MRI using a 3D Hybrid U-NET with Adaptive Self-Attention and Multi-Modal Fusion
… modality-specific feature extraction, adaptive self-attention, and late fusion to improve segmentation of ET, TC, and WT. Second, AIMS-Select extends this framework by learning budget-aware modality–slice gates that select informative inputs before expensive 3D CNN and Transformer encoding, …
-
Recall informed self-attention representation learning and generative AI for improved car accident detection in highly unbalanced datasets
This thesis explores the application of self-attention mechanisms in conjunction with recall-informed loss functions and generative models to the domain of traffic safety analysis and car accident predictions. Utilizing a comprehensive dataset compiled from the Colorado Department of …
-
Multi-headed self-attention mechanism-based Transformer model for predicting bus travel times across multiple bus routes using heterogeneous datasets
… Missouri. This study developed a multi-headed self-attention mechanism-based Univariate Transformer neural network to predict the mean vehicle travel times for different hours of the day for multiple stations across multiple routes. In addition, we developed Multivariate GRU and LSTM neural …
-
Self-focused attention, self-analysis, and rumination in everyday life : friend or foe?
… a proliferation of research linking forms of self-attention to psychological distress, especially to depression. The broad conclusion that self-attention is harmful, however, challenges the central premise of self-regulation theory - a theory detailing the role of self-attention as the engine …
-
Sequence Modeling with Linear Complexity
… in machine learning. Today, non-autoregressive attention-based methods that use self-attention are deemed standard for modeling sequences. The major drawback of self-attention is the quadratic time complexity O(n^2) for the processing of a sequence that it requires. This makes self-attention …
-
A Deep Learning Model to Impute Missing Data in Time Series
… we further propose a novel model, called SAITS (Self-Attention-based Imputation for Time Series), to impute missing values in multivariate time series. SAITS uses a joint-optimization training approach to learn missing values from a weighted combination of two diagonally-masked self-attention …
-
Hierarchical Architecture Optimization for Efficient Transformer-based Monte Carlo Denoising
… (SPP) inputs. Re cently, the Joint Self-Attention (JSA) framework, a Transformer-base method uti lizing auxiliary G-buffers, has achieved state-of-the-art visual fidelity. However, the quadratic computational complexity of standard multi-head self-attention disquali fies it from …
Page 1 of 15