Global ETD Search

Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.

Results

Showing 1 to 20 of 409 for “"pretraining"”.

  1. DexV2A : vision pretraining for dexterous manipulation

    Achieving human-like dexterity for common daily tasks with a robotic hand is a challenging task for reinforcement learning. To combat the high dimensionality of the state-action space, we propose pre-training policies on low-level vision tasks. Our system, called DexV2A, first trains a neural …

    texas Repository record for DexV2A : vision pretraining for dexterous manipulation (opens in a new tab)

  2. Study of Pretraining Bias and Frequencies

    … tasks. Recent works have showcased the impact of pretraining data on the in-context performance of language models. In this work, we experiment with numbers having high and low frequencies in the pretraining data to understand the impact of term frequencies on the model's performance. We also …

    vt Repository record for Study of Pretraining Bias and Frequencies (opens in a new tab)

  3. Novel Methods for Natural Language Modeling and Pretraining

    … Acoustic and Text Modeling is a novel pretraining method leveraging both the segmentation and alignment features for text-speech sequence modeling. It reconstructs masked acoustic signals with text input and acoustic-text alignment during training. In this way, the pretrained model can …

    waterloo Repository record for Novel Methods for Natural Language Modeling and Pretraining (opens in a new tab)

  4. Pretraining Effects on Visual Learning, Scanning and Heart Rate

    … experiment was to investigate the effects of pretraining (visual experience) on visual scanning (VS) and visual discrimination learning (VDL) in adults. In addition, a heart rate (HR) model of visual learning has been proposed. In the model, it is predicted that during discrimination learning …

    mo-state Repository record for Pretraining Effects on Visual Learning, Scanning and Heart Rate (opens in a new tab)

  5. Self-supervised Pretraining of Vision Transformers for Earth Observation

    … foundation models for Earth Observation by pretraining vision transformers (ViTs) using self-supervised learning. Using masked autoencoding, I develop one of the first ViTs pretrained on RS data—called SatViT. Next, I develop an improved foundation model—called SatViT-V2—and evaluate it on …

    carleton Repository record for Self-supervised Pretraining of Vision Transformers for Earth Observation (opens in a new tab)

  6. TOWARD HONEST LARGE LANGUAGE MODELS VIA RETRIEVAL OF PRETRAINING DATA

    … knowledge that the LLM has ingested during its pretraining. In this thesis, we propose a more robust evaluation benchmark dataset for LLM honesty by utilizing Pythia, a truly open LLM with publicly available pretraining data. In addition, we also propose a novel method for harnessing the …

    nus Repository record for TOWARD HONEST LARGE LANGUAGE MODELS VIA RETRIEVAL OF PRETRAINING DATA (opens in a new tab) Download file for TOWARD HONEST LARGE LANGUAGE MODELS VIA RETRIEVAL OF PRETRAINING DATA (opens in a new tab)

  7. ADVANCING VISION-LANGUAGE PRETRAINING: UNIFIED ARCHITECTURES AND IMPROVED GROUNDING ABILITIES

    The fusion of visual and language pre-training models has significantly advanced AI, particularly in content retrieval, visual reasoning, and multimodal understanding. However, challenges remain in achieving fine-grained semantic alignment and efficient processing of temporal dynamics in video. …

    nus Repository record for ADVANCING VISION-LANGUAGE PRETRAINING: UNIFIED ARCHITECTURES AND IMPROVED GROUNDING ABILITIES (opens in a new tab) Download file for ADVANCING VISION-LANGUAGE PRETRAINING: UNIFIED ARCHITECTURES AND IMPROVED GROUNDING ABILITIES (opens in a new tab)

  8. Chart Question Answering with an Universal Vision-Language Pretraining Approach

    … existing methods for these tasks often rely on pretraining on language or vision-language tasks, neglecting the explicit modeling of chart structures. To address this, we first build a large corpus of charts covering diverse topics and visual styles. We then present UniChart, a pretrained model …

    york Repository record for Chart Question Answering with an Universal Vision-Language Pretraining Approach (opens in a new tab)

  9. On the Knowledge Transfer via Pretraining, Distillation and Federated Learning

    <p>Modern machine learning technology based on a revival of deep neural networks has been successfully applied in many pragmatic domains such as computer vision(CV) and natural language processing(NLP). The very standard paradigm is \emph{pre-training}: a large model with billions of parameters is …

    duke Repository record for On the Knowledge Transfer via Pretraining, Distillation and Federated Learning (opens in a new tab)

  10. Meta-learning and synthetic data for automated pretraining and finetuning

    … enhance performance and effectively use data in pretraining and finetuning regimes. We empirically show the underestimated importance of data augmentation when using Self-Supervised Learning in the computer vision domain and, based on that, meta-learn advanced data augmentation strategies that …

    freiburg-diss Repository record for Meta-learning and synthetic data for automated pretraining and finetuning (opens in a new tab)

  11. Unsupervised Pretraining of Neural Networks with Multiple Targets using Siamese Architectures

    A model's response for a given input pattern depends on the seen patterns in the training data. The larger the amount of training data, the more likely edge cases are covered during training. However, the more complex input patterns are, the larger the model has to be. For very simple use cases, a …

    qucosa-diss

  12. Understanding the mechanism of pretraining stabilization heuristics: A variance-oriented perspective

    Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-09-16 without embargo terms

    uiuc Repository record for Understanding the mechanism of pretraining stabilization heuristics: A variance-oriented perspective (opens in a new tab)

  13. Understanding the mechanism of pretraining stabilization heuristics: A variance-oriented perspective

    Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-09-16 without embargo terms

    uiuc Repository record for Understanding the mechanism of pretraining stabilization heuristics: A variance-oriented perspective (opens in a new tab)

  14. An Empirical Study of Self-Supervised Pretraining for X-ray Security Screening

    … reconstruction-based and self-distillation-based pretraining paradigms influence downstream performance on X-ray security screening tasks with distinct semantic and spatial requirements. We conduct a systematic comparison of Masked Autoencoders (MAE) and DINO (Self-Distillation with No Labels) …

    concordia Download PDF for An Empirical Study of Self-Supervised Pretraining for X-ray Security Screening (opens in a new tab)

  15. Retrieval-First HD-Map Maintenance through Semantic Conditioning and Multimodal Predictive Pretraining

    … Its semantic-span joint-embedding predictive-pretraining strategy predicts semantically coherent masked-region representations and is providing significant enhancement over I-JEPA geometric multi-block masking pretraining. Under the NavINST dataset evaluation, semantic conditioning improves …

    queens Repository record for Retrieval-First HD-Map Maintenance through Semantic Conditioning and Multimodal Predictive Pretraining (opens in a new tab)

  16. Combining Supervised Pretraining and Reinforcement Learning for Scalable Low-Thrust Trajectory Design

    … training framework that combines supervised pretraining with deep reinforcement learning to approximate optimal low-thrust control for heliocentric transfer and rendezvous trajectories. The primary test case is a circular two-body transfer problem in which a continuously thrusting spacecraft …

    maryland Repository record for Combining Supervised Pretraining and Reinforcement Learning for Scalable Low-Thrust Trajectory Design (opens in a new tab)

Page 1 of 21