Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 96 for “"Distributed Training"”.
-
Efficient Distributed Training With Subnetworks
Pre-training large neural networks at scale places significant memory and communication demands on modern accelerators. As model sizes continue to grow faster than available device memory, efficiently distributing training across multiple hardware devices has become increasingly challenging. In …
-
Understanding and Optimizing Communication Overhead in Distributed Training
… Deep Learning models on a specific task requires training. With the recent trends of the rapid growth of the size of the Deep Learning models and datasets, training on a single accelerator can take years. To complete the training within a reasonable amount of time, people start using multiple …
-
Network-aware device placement search for distributed training
… models grow in scale and complexity, efficient distributed training requires not only advanced parallelization strategies but also intelligent placement of model components across heterogeneous computing infrastructures. Existing device placement frameworks often assume simplified, uniform …
-
A comparison research model for adult distributed training
… and cost-effectiveness of online adult distributed training compared to a traditional classroom setting. The model was applied to First Connections, an online infant and toddler care provider training program. One hundred and thirteen participants were given a pretest composed of 40 …
-
Algorithms and architectures for distributed training of deep neural networks
L'abstract è presente nell'allegato / the abstract is in the attachment
-
Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithms
… in the study of median-based algorithms for distributed non-convex optimization. Two prominent examples include signSGD with majority vote, an effective approach for communication reduction via 1-bit compression on the local gradients, and medianSGD, an algorithm recently proposed to ensure …
-
Distributed Training for Deep Reinforcement Learning Decoders on the Toric Code
We distribute the training of a deep reinforcement learning-based decoder on the toric code developed by Fitzek et al. [9]. Reinforcement learning agents asynchronously step through multiple environments in parallel and store transitions in a prioritized experience replay buffer. A separate process …
-
Comparison of distributed training architecture for convolutional neural network in cloud
… natural language processing and data mining. The training process of the DNN is a computationally intensive application that can be accelerated by parallel computing devices such as graphic processing units (GPUs) and field programmable gate arrays (FP- GAs). However, sometimes the amount of the …
-
STOCHASTIC OPTIMIZATION FOR DISTRIBUTED TRAINING AND MULTI-AGENT INTERACTION IN LARGE LANGUAGE MODELS
… develops stochastic optimization frameworks for training and coordinating Large Language Models (LLMs) in distributed settings. We study two interconnected problems: (i) controlling the training dynamics of LLMs across distributed datasets, and (ii) modeling and controlling emergent behaviors …
-
A General Method for Efficient Distributed Training and Federated Learning in Synchronous and Asynchronous Scenarios
… the size of machine learning model, the size of training dataset and the training time, while the available computation resources are generally limited due to limited memory size and computation power of computation devices and limited data usage (due to data storage or user privacy). In general, …
-
Robust Machine Learning Against Faults in Micro-Controllers and Stragglers in Distributed Training on the Cloud
… ML models onto smaller devices and efficiently training more powerful ML models in parallel in different distributed system topologies have drawn interests. This thesis studies the robustness of ML models in the two scenarios when deployed on portable micro-controller units and while being …
-
GraphDHT: Scaling Graph Neural Networks' Distributed Training on Edge Devices on a Peer-to-Peer Distributed Hash Table Network
This thesis presents an innovative strategy for distributed Graph Neural Network (GNN) training, leveraging a peer-to-peer network of heterogeneous edge devices interconnected through a Distributed Hash Table (DHT). As GNNs become increasingly vital in analyzing graph-structured data across various …
-
HW-SW Methods for Modeling and Optimizing Communication for Scalable Training of Deep Learning Models
… and optimizing communication for scalable training of deep learning models. DL applications are becoming an integral part of our society due to their vast application in different domains such as vision, language processing, recommendation systems, speech processing, etc. Before being …
-
A TRAINING FRAMEWORK AND ARCHITECTURAL DESIGN OF DISTRIBUTED DEEP LEARNING
… deep learning systems, including stand-alone and distributed training. Second, we have designed and developed a distributed deep learning system, named SINGA, which tackles the usability problem and realizes optimization techniques for distributed training. SINGA provides a flexible system …
-
Behavioral analysis of memory in Caenorhabditis elegans
… that long-term memory for habituation requires a distributed training protocol. I confirmed the generally accepted view that mechanisms of memory consolidation occur after each training trial by showing that protein synthesis inhibition delivered after every training trial effectively blocked …
-
Distributed On-line Training for Object Detection on Embedded Devices.
In this thesis, we develop a scalable distributed approach for object detection model training and inference, using low cost embedded devices. Examples of the usage of such an approach is automatic beach litter collection using mobile robots, IoT based intelligent video surveillance system for …
-
Accelerating distributed neural network training with network-centric approach
Distributed training of Deep Neural Networks (DNN) is an important technique to reduce the training time of large DNNs for a wide range of applications. In existing distributed training approaches, however, the communication time to periodically exchange parameters (i.e., weights) and gradients …
-
Towards More Efficient Communication for Distributed Learning Systems
… accelerators shift the bottleneckof large-scale distributed training tasks from computation to communication. We observesignificant pressure on the communication backends of various mainstream learning systemsin multiple environments when running such tasks. Achieving efficient large scale …
-
Scalable and efficient deep learning for diabetic retinopathy classification on ARM-based architectures
… focusing on performance, energy consumption, and distributed training efficiency. We evaluated 38 CNN architectures to identify a model that balances predictive accuracy with computational and energy efficiency. MobileNet achieved the best results, consuming 77% less energy, training 83% faster, …
Page 1 of 5