Back to search

Michigan Technological University

Privacy-Preserving and Resource-Efficient Learning Frameworks for Collaborative Deep Neural Networks

Abstract

dc:description.abstract

Deep Neural Networks (DNNs) are rapidly advancing in capability and scale, increasing reliance on them to analyze sensitive data and, consequently, heightening privacy risks during training and inference, especially in untrusted federated learning (FL) and collaborative inference settings, where adversaries may exploit gradients or intermediate features. Meanwhile, modern networks often exceed a single device’s capacity, and fine-tuning large foundation models remains costly for resource-limited users. Thus, balancing training efficiency with fine-tuning performance is essential for scalable distributed learning. Therefore, my current research focuses on three aspects: (1) reducing privacy risks in collaborative inference, (2) protecting users’ data in federated learning, and (3) efficient fine-tuning of large foundation models with minimal performance loss. Collaborative inference enables resource-limited edge devices to use large DNNs by computing early layers locally and sending intermediate features to the cloud. However, recent work shows that these features allow model inversion attacks (MIAs) to reconstruct sensitive inputs, and existing perturbation- or cryptographic-based defenses struggle to balance MIA robustness, accuracy, and theoretical guarantees. My first work proposes PATROL, a privacy-oriented pruning framework that strengthens task-relevant feature extraction on the client while suppressing privacy-sensitive signals. Using Lipschitz regularization and adversarial reconstruction training, PATROL achieves superior MIA resistance on a real vehicle re-identification task. My second work proposed a framework, CUPR, to establish a theoretical framework linking reconstruction performance to noise-perturbed utility, clarifying the privacy–utility trade-off observed in prior defenses. The theoretical lower bound is validated on MNIST, CIFAR-10, and FaceScrub under four reconstruction attacks, showing favorable trade-offs for both client-side and server-side CUPR: the former excels on low-dimensional data, and both perform similarly on high-dimensional inputs. During training, FL remains vulnerable because model updates leak private information. Although Trust Execution Environments (TEEs) offer hardware-isolated protection, their limited memory prevents deployment of large networks. My third work introduces Marshaled Learning, which partitions models into subnets across TEE-enabled clients and incorporates a dynamic knowledge propagation mechanism to improve performance under heterogeneous data. Marshaled Learning improves accuracy by 2–5%, converges faster than prior FL methods, and incurs only 1–3× overhead on Azure confidential VMs. For large foundation models, memory constraints also limit fine-tuning efficiency. D2FT addresses this by dynamically selecting attention modules during forward/backward passes, reducing unnecessary computation. With multi-knapsack optimization, D2FT reduces training computation by 40% and communication by 50%, with minimal accuracy loss on three datasets. When integrated with LoRA, D2FT achieves 40% computation or 50% communication reduction, with only 4–6% accuracy drop on Stanford Cars.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy in Computer Science (PhD)
Level thesis:degree_level
Campus Access Dissertation
Discipline thesis:degree_discipline
Department of Computer Science

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Ding, Shiwei
Committee members dc:contributor.committeemember
  • Xinyu Lei
  • Kaichen Yang
Contributors dc:contributor
  • Xiaoyong Yuan
  • Zhenlin Wang

Subjects

dc:subject × 3

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
Michigan Technological University
Base URL
digitalcommons.mtu.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Ding, Shiwei. Privacy-Preserving and Resource-Efficient Learning Frameworks for Collaborative Deep Neural Networks. Campus Access Dissertation thesis, https://doi.org/10.37099/mtu.dc.etdr/2028