Global ETD Search

Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.

Results

Showing 1 to 20 of 300 for “"Self-attention"”.

  1. Self-attention policy architectures for reinforcement learning under partial observability

    … which necessarily constitutes noise. We explore self-attention-based policy architectures as a solution to this problem, demonstrating their robustness under conditions of high partial observability on different rein-forcement learning benchmark tasks, and explore the advantages and disadvantages …

    cape-town Repository record for Self-attention policy architectures for reinforcement learning under partial observability (opens in a new tab)

  2. Applications of Diversity and the Self-Attention Mechanism in Neural Networks

    … the other two cover novel applications of the self-attention mechanism. An important aspect of training a neural network is the choice of objective function. Regression via Classification (RvC) is often used to tackle problems in deep learning where the target variable is continuous, but …

    lund Repository record for Applications of Diversity and the Self-Attention Mechanism in Neural Networks (opens in a new tab) Download PDF for Applications of Diversity and the Self-Attention Mechanism in Neural Networks (opens in a new tab)

  3. The Effects of Expectancy, Task Importance and Self-Attention on Task Persistence

    … power of control theory (Carver, 1979) and self-efficacy theory (Bandura, 1977) in predicting behavioral persistence. This study employed a 2 X 2 X 2 X 2 factorial design. One hundred and twelve undergraduate females were exposed to high and low levels of self-attention and task importance …

    wilfrid-laurier Repository record for The Effects of Expectancy, Task Importance and Self-Attention on Task Persistence (opens in a new tab) Download PDF for The Effects of Expectancy, Task Importance and Self-Attention on Task Persistence (opens in a new tab)

  4. On the Efficiency and Steerability of Self-Attention Mechanism of Large Language Models

    … language processing through their powerful self-attention mechanisms. In this thesis, we focus on two crucial aspects of self-attention: efficiency and steerability, and explore innovative prompting techniques to improve contextual understanding and complex reasoning. Mixed attention span …

    gatech Repository record for On the Efficiency and Steerability of Self-Attention Mechanism of Large Language Models (opens in a new tab)

  5. Towards understanding self-attention and batch normalization : optimization and margin dynamics in neural networks

    … understanding of transformers and their core, self-attention mechanism remains incomplete, especially with regard to their optimization and generalization behaviors. Previous research has predominantly focused on single-head attention, thereby limiting insights into the impact of …

    ubc Repository record for Towards understanding self-attention and batch normalization : optimization and margin dynamics in neural networks (opens in a new tab) Download PDF for Towards understanding self-attention and batch normalization : optimization and margin dynamics in neural networks (opens in a new tab)

  6. Revolutionizing Time Series Data Preprocessing with a Novel Cycling Layer in Self-Attention Mechanisms

    … by incorporating a cycling layer into self-attention mechanisms. Traditional techniques often struggle to capture the cyclical nature of time series data, impacting predictive model accuracy. By integrating a cycling layer, this thesis aims to enhance the ability of models to recognize …

    york Repository record for Revolutionizing Time Series Data Preprocessing with a Novel Cycling Layer in Self-Attention Mechanisms (opens in a new tab)

  7. On the optimization and generalization of self-attention models : a stability and implicit bias perspective

    … a foundational theory. Specifically, we focus on self-attention models, the core mechanism of the Transformer. We first study multi-head attention models in a binary classification setting, developing optimization and generalization bounds that elucidate the finite-time behavior of these models …

    ubc Repository record for On the optimization and generalization of self-attention models : a stability and implicit bias perspective (opens in a new tab) Download PDF for On the optimization and generalization of self-attention models : a stability and implicit bias perspective (opens in a new tab)

  8. Hybrid Deep Learning Model: LLM-Enhanced Linear Self-Attention Transformer-CNN for Stock Price Prediction

    … pipelines. One pipeline utilizes an optimized attention-based Transformer to process the LLM-generated embeddings, capturing long-range market dependencies. Concurrently, the second pipeline deploys a Convolutional Neural Network (CNN) to analyze visual stock chart patterns, extracting …

    auckland-tech Repository record for Hybrid Deep Learning Model: LLM-Enhanced Linear Self-Attention Transformer-CNN for Stock Price Prediction (opens in a new tab)

  9. DSA-BEATS: Multi step, multi-variate dual self-attention N-BEATS model for forecasting COVID-19 hospitalization

    … the spread. In this study, we present the Dual Self-Attention N-BEATS (DSA-BEATS) model, a novel approach that effectively combines the self-attention mechanism of transformers with the proficiency of the N-BEATS model in dealing with multi-variate forecasting problems. The model has been …

    regina Repository record for DSA-BEATS: Multi step, multi-variate dual self-attention N-BEATS model for forecasting COVID-19 hospitalization (opens in a new tab)

  10. Theoretical Expressivity of Self-Attention as a Model of Binary Sequence-To-Sequence Functions and Other Explorations

    The self-attention mechanism lies at the heart of modern sequential modeling with deep neural networks, especially in the context of Large Language Models (LLMs). The basic role of self-attention is to capture pairwise interactions between tokens in a sequence. This allows token embeddings to be …

    byu Repository record for Theoretical Expressivity of Self-Attention as a Model of Binary Sequence-To-Sequence Functions and Other Explorations (opens in a new tab) Download PDF for Theoretical Expressivity of Self-Attention as a Model of Binary Sequence-To-Sequence Functions and Other Explorations (opens in a new tab)

  11. Glioma Segmentation in MRI using a 3D Hybrid U-NET with Adaptive Self-Attention and Multi-Modal Fusion

    … modality-specific feature extraction, adaptive self-attention, and late fusion to improve segmentation of ET, TC, and WT. Second, AIMS-Select extends this framework by learning budget-aware modality–slice gates that select informative inputs before expensive 3D CNN and Transformer encoding, …

    odu Repository record for Glioma Segmentation in MRI using a 3D Hybrid U-NET with Adaptive Self-Attention and Multi-Modal Fusion (opens in a new tab)

  12. Recall informed self-attention representation learning and generative AI for improved car accident detection in highly unbalanced datasets

    This thesis explores the application of self-attention mechanisms in conjunction with recall-informed loss functions and generative models to the domain of traffic safety analysis and car accident predictions. Utilizing a comprehensive dataset compiled from the Colorado Department of …

    colo-mines Repository record for Recall informed self-attention representation learning and generative AI for improved car accident detection in highly unbalanced datasets (opens in a new tab)

  13. Multi-headed self-attention mechanism-based Transformer model for predicting bus travel times across multiple bus routes using heterogeneous datasets

    … Missouri. This study developed a multi-headed self-attention mechanism-based Univariate Transformer neural network to predict the mean vehicle travel times for different hours of the day for multiple stations across multiple routes. In addition, we developed Multivariate GRU and LSTM neural …

    missouri Repository record for Multi-headed self-attention mechanism-based Transformer model for predicting bus travel times across multiple bus routes using heterogeneous datasets (opens in a new tab)

  14. Self-focused attention, self-analysis, and rumination in everyday life : friend or foe?

    … a proliferation of research linking forms of self-attention to psychological distress, especially to depression. The broad conclusion that self-attention is harmful, however, challenges the central premise of self-regulation theory - a theory detailing the role of self-attention as the engine …

    ubc Repository record for Self-focused attention, self-analysis, and rumination in everyday life : friend or foe? (opens in a new tab) Download PDF for Self-focused attention, self-analysis, and rumination in everyday life : friend or foe? (opens in a new tab)

  15. Sequence Modeling with Linear Complexity

    … in machine learning. Today, non-autoregressive attention-based methods that use self-attention are deemed standard for modeling sequences. The major drawback of self-attention is the quadratic time complexity O(n^2) for the processing of a sequence that it requires. This makes self-attention …

    carleton Repository record for Sequence Modeling with Linear Complexity (opens in a new tab)

  16. A Deep Learning Model to Impute Missing Data in Time Series

    … we further propose a novel model, called SAITS (Self-Attention-based Imputation for Time Series), to impute missing values in multivariate time series. SAITS uses a joint-optimization training approach to learn missing values from a weighted combination of two diagonally-masked self-attention …

    concordia Download PDF for A Deep Learning Model to Impute Missing Data in Time Series (opens in a new tab)

  17. Hierarchical Architecture Optimization for Efficient Transformer-based Monte Carlo Denoising

    … (SPP) inputs. Re cently, the Joint Self-Attention (JSA) framework, a Transformer-base method uti lizing auxiliary G-buffers, has achieved state-of-the-art visual fidelity. However, the quadratic computational complexity of standard multi-head self-attention disquali fies it from …

    chalmers Repository record for Hierarchical Architecture Optimization for Efficient Transformer-based Monte Carlo Denoising (opens in a new tab)

Page 1 of 15