Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 409 for “"pretraining"”.
-
DexV2A : vision pretraining for dexterous manipulation
Achieving human-like dexterity for common daily tasks with a robotic hand is a challenging task for reinforcement learning. To combat the high dimensionality of the state-action space, we propose pre-training policies on low-level vision tasks. Our system, called DexV2A, first trains a neural …
-
Study of Pretraining Bias and Frequencies
… tasks. Recent works have showcased the impact of pretraining data on the in-context performance of language models. In this work, we experiment with numbers having high and low frequencies in the pretraining data to understand the impact of term frequencies on the model's performance. We also …
-
Novel Methods for Natural Language Modeling and Pretraining
… Acoustic and Text Modeling is a novel pretraining method leveraging both the segmentation and alignment features for text-speech sequence modeling. It reconstructs masked acoustic signals with text input and acoustic-text alignment during training. In this way, the pretrained model can …
-
Pretraining Effects on Visual Learning, Scanning and Heart Rate
… experiment was to investigate the effects of pretraining (visual experience) on visual scanning (VS) and visual discrimination learning (VDL) in adults. In addition, a heart rate (HR) model of visual learning has been proposed. In the model, it is predicted that during discrimination learning …
-
Self-supervised Pretraining of Vision Transformers for Earth Observation
… foundation models for Earth Observation by pretraining vision transformers (ViTs) using self-supervised learning. Using masked autoencoding, I develop one of the first ViTs pretrained on RS data—called SatViT. Next, I develop an improved foundation model—called SatViT-V2—and evaluate it on …
-
TOWARD HONEST LARGE LANGUAGE MODELS VIA RETRIEVAL OF PRETRAINING DATA
… knowledge that the LLM has ingested during its pretraining. In this thesis, we propose a more robust evaluation benchmark dataset for LLM honesty by utilizing Pythia, a truly open LLM with publicly available pretraining data. In addition, we also propose a novel method for harnessing the …
-
ADVANCING VISION-LANGUAGE PRETRAINING: UNIFIED ARCHITECTURES AND IMPROVED GROUNDING ABILITIES
The fusion of visual and language pre-training models has significantly advanced AI, particularly in content retrieval, visual reasoning, and multimodal understanding. However, challenges remain in achieving fine-grained semantic alignment and efficient processing of temporal dynamics in video. …
-
Chart Question Answering with an Universal Vision-Language Pretraining Approach
… existing methods for these tasks often rely on pretraining on language or vision-language tasks, neglecting the explicit modeling of chart structures. To address this, we first build a large corpus of charts covering diverse topics and visual styles. We then present UniChart, a pretrained model …
-
On the Knowledge Transfer via Pretraining, Distillation and Federated Learning
<p>Modern machine learning technology based on a revival of deep neural networks has been successfully applied in many pragmatic domains such as computer vision(CV) and natural language processing(NLP). The very standard paradigm is \emph{pre-training}: a large model with billions of parameters is …
-
Meta-learning and synthetic data for automated pretraining and finetuning
… enhance performance and effectively use data in pretraining and finetuning regimes. We empirically show the underestimated importance of data augmentation when using Self-Supervised Learning in the computer vision domain and, based on that, meta-learn advanced data augmentation strategies that …
-
Unsupervised Pretraining of Neural Networks with Multiple Targets using Siamese Architectures
A model's response for a given input pattern depends on the seen patterns in the training data. The larger the amount of training data, the more likely edge cases are covered during training. However, the more complex input patterns are, the larger the model has to be. For very simple use cases, a …
-
Understanding the mechanism of pretraining stabilization heuristics: A variance-oriented perspective
Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-09-16 without embargo terms
-
Understanding the mechanism of pretraining stabilization heuristics: A variance-oriented perspective
Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-09-16 without embargo terms
-
An Empirical Study of Self-Supervised Pretraining for X-ray Security Screening
… reconstruction-based and self-distillation-based pretraining paradigms influence downstream performance on X-ray security screening tasks with distinct semantic and spatial requirements. We conduct a systematic comparison of Masked Autoencoders (MAE) and DINO (Self-Distillation with No Labels) …
-
Retrieval-First HD-Map Maintenance through Semantic Conditioning and Multimodal Predictive Pretraining
… Its semantic-span joint-embedding predictive-pretraining strategy predicts semantically coherent masked-region representations and is providing significant enhancement over I-JEPA geometric multi-block masking pretraining. Under the NavINST dataset evaluation, semantic conditioning improves …
-
Combining Supervised Pretraining and Reinforcement Learning for Scalable Low-Thrust Trajectory Design
… training framework that combines supervised pretraining with deep reinforcement learning to approximate optimal low-thrust control for heliocentric transfer and rendezvous trajectories. The primary test case is a circular two-body transfer problem in which a continuously thrusting spacecraft …
Page 1 of 21