Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 184 for “"Transformer architecture"”.
-
Utilizing the Transformer Architecture for Question Answering
… document(s). In this thesis, we utilize the transformer, a state-of-the-art neural architecture to study two QA problems: the answer sentence selection and the answer summary generation. For answer sentence selection, we present two new approaches that rank a list of candidate answers for a …
-
A Multi-Scale Transformer Architecture for Cardano Price Forecasting with Retrieval-Augmented Context
… Recent advances in deep learning, particular Transformer architectures, have shown their superiority in time series forecasting by leveraging self-attention to capture long range dependencies. However, studies have shown that the canonical Transformer model does not consistently outperform …
-
AN EXPLORATION OF TRANSFORMER ARCHITECTURE FOR SHALE SEM IMAGES INTERPRETATION AND FOR TIME-SERIES ANOMALY DETECTION
This dissertation explores the application of Transformer-based architectures for two criticaltasks in the oilfield domain: Scanning Electron Microscopy (SEM) image analysis and anomaly detection in multivariate time-series production data. The study addresses key challenges in both domains, …
-
Localization of diversions in buried high density polyethylene pipes utilizing preprocessed Zadoff-Chu ultrasonic guided-wave reflections and vision transformer architecture
… sensors, signal processing, and a vision transformer (ViT) architecture. The approach specifically addresses the challenge of determining the distance from the measurement point to the diversion site. Zadoff-Chu (ZC) modulated ultrasonic signals are transmitted through the pipe using a …
-
Teksto žinių perkėlimas natūralios kalbos apdorojimo uždaviniams /
This thesis analyses transformer architecture deep neural networks for natural language processing tasks and text knowledge transfer for Lithuanian language using these networks. During the experiment two transformer architecture models were created and trained using knowledge transfer methods. In …
-
From Intent Detection to Recommendations: Leveraging Pretrained Language Models for Enhanced User Experiences
… (NLP) in particular. With the advent of transformer architecture, pretrained language models are at the forefront of driving advancements in machine intelligence. Scaled dot product attention sits at the core of transformer architecture. In scaled dot product attention, each token gets a …
-
Examining Information Flows Within Transformer Models Using Transfer Entropy
… neural networks. This is especially true for transformer architectures and large language models, which are the driving force behind the current surge in artificial intelligence. The complexity of transformer architecture foils many pre-existing methods for studying explainability in deep …
-
TRANSFORMERS ON CONTEXT FREE GRAMMARS
Transformer models have become the backbone of modern large language models (LLMs), excelling across a broad spectrum of natural language tasks. However, training these models demands extensive, high-quality datasets, which can be both costly and resource-intensive to obtain. As an alternative, …
-
Key-Frame Based Motion Representations for Pose Sequences
… for this task and design a key-framing transformer architecture to distill a motion sequence into key-frames and their relative placements in time. We validate our learned key-frame placement approach with a naive uniform placement strategy and further compare key-frame distillation …
-
Crime Detection from Pre-crime Video Analysis
… Networks (RNNs), and a specially developed transformer architecture, the research systematically explores the impact of integrating additional contextual information into video analysis. By augmenting frame-level video data with detailed pose and emotion insights, and focusing on the …
-
Natūralios kalbos modelių taikymas teisiniams dokumentams /
… as text classification or semantic search. The architecture of the transformers is analysed before practical experiments are carried out. During the analysis of the transformer architecture, models are found and analysed which are recognised as one of the most popular models for natural language …
-
Machine Learning for Large-Scale Cyber-Physical Systems
… in systems by designing size-generalizable architectures. Once trained on small-scale examples, such architectures exhibit equivalent or comparable performance on large-scale systems. The first example we consider is a fully convolutional neural network, for which we prove a bound that …
-
Methodologies for the Management, Normalization and Identification of Sexual Predation of Minors in Cyber Chat Logs
Neural networks based on the Transformer architecture have shown great results in tasks such as machine translation and text generation. Our contribution provides a methodology for an AI agent capable of Sexual Predator Identification (SPI) based on the classification capabilities of models built …
-
AN ANALYSIS OF SECURITY RISKS POSED BY TEXT-BASED GENERATIVE AI AND CORPORATE SECURITY WEAKNESSES LEADING TO DATA LEAKS
… based on the deep learning technique known as Transformer architecture, used for performing Natural Language Processing (NLP) tasks. Additionally, a brief explanation of the Transformer architecture, including its main components, the encoder and decoder, has been included. The research …
-
Scalable vision transformers for remote sensing semantic segmentation
… complex patterns within the data. Recently, the Transformer architecture has been extended to computer vision, further enhancing the versatility and scalability of deep learning models. This thesis investigates the application of the Transformer architecture to semantic segmentation using …
-
Architectural and Training Regime Modifications to Transformer Models for low data Applications
… of high-performance LLM models are based on the transformer architecture and the primary factor behind the recent enhancement in the performance of these models is attributed to the increase of layer quantity and dimensionality, resulting in an exponential rise in the total number of learnable …
-
NEURAL ARCHITECTURE DESIGN AND APPLICATIONS
Neural architecture design is crucial in AI development. This thesis first examines the macroscopic architecture of Transformers, challenging the belief that their attention-based token mixer is key. By replacing the attention module with a simple spatial pooling operator, we create PoolFormer, …
-
Enhancing Post Silicon Visibility Using Language Modelling Techniques
… trace. We trained various models using the transformer architecture to identify missing data in the trace. Our models were trained using several different techniques and we provided a detailed analysis of the results and utility of these techniques.
Page 1 of 10