Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 19 of 19 for “"Parameter Server"”.
-
Study and optimization of distributed deep learning under the parameter server architecture
… investigates asynchronous training under the parameter server paradigm, focusing on enhancing both performance and stability. First, a thorough comparative analysis demonstrates that specialized distributed architectures deliver substantially higher throughput than general-purpose …
-
U-Net ship detection in satellite optical imagery
… architectures: single node architecture and a parameter server variant whose workers act as a boosting mechanism. To ex-tend this effort, a refining method of offline hard example mining aims to improve the accuracy of the trained models in both the validation and target datasets however it …
-
Simulation and analysis of gradient coding protocols- Leveraging partial stragglers within gradient coding
… expected, known as \emph{stragglers}, force the parameter server to wait indefinitely for gradient computations, severely degrading training efficiency. Gradient coding, a coding-theoretic approach to distributed learning, addresses this problem by introducing redundancy in the assignment of data …
-
Scaling a convolutional neural network based Flower counting application in a distributed GPU cluster
… two different distributed architectures called Parameter Server and Ring-AllReduce. Moreover, a detailed explanation of the proposed CNN's architecture along with its memory footprints and GPU utilization is also organized as an in-depth case study to help trace the model's memory consumption …
-
Distributed On-line Training for Object Detection on Embedded Devices.
… on embedded devices, we adopt the concept of Parameter Server based distributed training, where each embedded de- vice works as node or worker and can communicate through Parameter server(s). We develop a distributed training approach keeping in mind the resource constraints e.g. memory …
-
Elastic Synchronization for Efficient and Effective Distributed Deep Learning
… two efficient and effective models in the parameter server setting based on the limitations of the state-of-the-art distributed models such as staleness synchronous parallel (SSP) and bulk synchronous parallel (BSP). We introduce DynamicSSP model that adds smart dynamic communication to …
-
Privacy and Communication Efficiency in Distributed Computing and Learning
… intensive tasks to powerful distributed servers. However, distributing tasks to servers incurs significant communication overhead. In addition, the heterogeneity in computing power among servers causes the straggler effect, where a few slower machines severely degrade the overall …
-
Stragglers-Robust First-Order Distributed Optimization Algorithms Over Networks Utilizing Gradient Coding
… First, a centralized algorithm is explored for parameter server networks robust to the straggler problem. It is proved that the server nodes' estimates converge to the minimizer of the global objective function with connections vulnerable to an allowed number of stragglers. We then show that …
-
Toward efficient online scheduling for large-scale distributed machine learning system
… scheduling algorithms to allocate workers and parameter servers across different machines to minimize the overall training time. Toward this end, in this paper, we develop an online scheduling algorithm that jointly optimizes resource allocation and locality decisions. Our main contributions …
-
Efficient Optimization Algorithms for Nonconvex Machine Learning Problems
… minimizing a loss function with respect to model parameters based on training samples. To enhance the efficiency of optimization algorithms, distributed learning has emerged as a popular solution for addressing large-scale machine learning tasks. In distributed learning, multiple worker nodes …
-
Privacy-aware Federated Learning with Global Differential Privacy
… However, by analyzing the uploaded client parameters, confidential information can still be revealed. To combat privacy attacks on the FL systems, various attempts have been made to incorporate differential privacy within the framework. In this thesis, we investigate the trade-offs between …
-
Comparison of distributed training architecture for convolutional neural network in cloud
… distributed deep learning, which outperforms the Parameter Server architecture, a many-to-one architecture, on scalability. In addition, system usage information reported from GCP is leveraged to identify the bottleneck of a neural network training on distributed architecture.
-
Bidirectional TopK Sparsification for Distributed Learning
… have adopted topK gradient sparsification to the parameter-server framework by applying topK compression in both the worker-to-server and server-to-worker direction, as opposed to only the worker-to-server direction. Current intuition and analysis suggest that adding extra compression degrades the …
-
Robust and scalable deep learning for cyber-physical systems
… among the agents with the aid of a central parameter server. Recently, decentralized learning algorithms have demonstrated state-of-the-art results, comparable with centralized algorithms which makes them independent from the central parameter server. However, a key requirement to achieve …
-
GRADIENT DESCENT METHODS IN MODERN MACHINE LEARNING PROBLEMS: PROVABLE GUARANTEES
… algorithm is essential for learning the model parameters. Among optimization methods, gradient descent (GD) methods are the simplest ones. Traditionally, GD methods have shown excellent performance in conventional machine learning problems with nice objective functions and simple training …
-
ACCELERATING DNN INFERENCE AND TRAINING IN DISTRIBUTED SYSTEMS
… is offloaded from mobile devices to cloud servers. The cloud/edge offloading approach investigates the collaboration between local and remote computation resources. The collaboration between mobile devices and the cloud server could speed up the DNN inference. We propose to jointly consider …
-
Scheduling and Power Control in Sustainable Federated Learning over Wireless Networks
… ML model through the iterative exchange of model parameters among the devices and a central parameter server (PS). Specifically, each local device performs local training on its own dataset for multiple rounds and transmits model updates to the PS. The PS then aggregates these updates and …
-
Towards Mobile-Device Friendly Federated Learning: Efficiency, Privacy and Personalization
… it unsafe and inefficient to upload and store on servers. These can be partly addressed by the promising technology of edge computing, which reduces the latency by moving computation to the vicinity of the data source. Moreover, federated learning (FL) which is a fast-growing distributed learning …
-
Leveraging coding techniques and redundancy for speeding up distributed computing and robustifying distributed learning
… in one machine; instead, a cluster of servers is utilized. Such setups often suffer from long communication phases that impede the execution of algorithms. Also, these systems are prone to Byzantine behavior where a server may return erroneous results due to an error or malicious …