Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 9 of 9 for “"Gradient compression"”.
-
Towards the Next Generation Highly Scalable Distributed Machine Learning
… being performed in-place with the data owners. Gradient compression is a promising general approach to alleviating the communication bottleneck in data parallel Deep Neural Network (DNN) training by significantly reducing the data volume of gradients for synchronization. The emerging Graph …
-
On Privacy and Efficiency for Wireless Federated Learning
… FL in a single-cell wireless network, in which gradient compression and DP noise are used to protect users’ privacy. Moreover, gradient compression and wireless resource allocation are jointly designed to reduce the training loss. Second, a privacy-aware FL framework is developed for multi-cell …
-
Communication-Efficient Federated Learning for Resource-Constrained Edge Environments
… selection. Third, AWSFL introduces sign-based gradient compression combined with loss-aware adaptive aggregation and error feedback to achieve high communication compression while preserving robust convergence under non-IID data. Finally, FedSyncDrop addresses system-level unreliability by …
-
Scaling Deep Learning through Optimizing Data- and Management-Plane Communications
… on data-plane communications that support gradient synchronization in DDL. We introduce Zen, a system that fully leverages the sparsity in gradient tensors to scale deep learning with a provable optimal scheme for sparse tensor synchronization. We then propose Espresso and Cupcake to …
-
Bidirectional TopK Sparsification for Distributed Learning
… in distributed training is communicating gradients across different nodes. Different gradient compression techniques have been proposed to alleviate the communication bottleneck, including topK gradient sparsification, which truncates the gradient to the top K components before sending it …
-
Understanding and Optimizing Communication Overhead in Distributed Training
… the network is under-utilized, and achieving gradient compression ratios up to hundreds of times is often unnecessary for data center networks. The second part of the thesis optimizes the communication overhead of distributed training without changing the semantics of the training algorithm. …
-
Improving Communication Efficiency And Convergence In Federated Learning
… in the training are ever in- creasing, the model gradient transmission between the remote clients and the central server orchestrating the training process becomes the critical performance bottleneck. In this thesis, we investigate and address the problems related to improving the communication …
-
Methods for Improving Computation and Communication Efficiency in Distributed Computing Algorithms
… large machine learning models with stochastic gradient descent. By exploiting the structure in messages exchanged between compute nodes, we develop compression methods that reduce the payload size, thus leading to more efficient use of network resources. Second, we concentrate on a source …
-
Re-examining Contribution Fairness in Federated Learning
… of contribution-based Top-K sparsified gradient rewards, such that higher-contributing clients receive denser, and, therefore, more useful, gradients than lower-contributing ones. Top-K sparsification, however, was originally devised as a gradient-compression technique intended to …