Old Dominion University
Pushing High-Performance Private Inference Towards Resource-Constrained Edge Clients
Abstract
dc:description.abstractThe widespread adoption of Machine Learning as a Service (MLaaS) has enabled resource constrained edge clients, such as mobile and IoT devices, to leverage powerful deep learning mod els hosted on the cloud. However, this paradigm introduces critical privacy challenges regarding the client’s sensitive input data and the server’s proprietary model parameters. While cryptographic techniques like Homomorphic Encryption (HE) and Multi-Party Computation (MPC) enable Private Inference (PI), existing frameworks impose prohibitive computational and communication overheads that render them impractical for edge deployment. This dissertation introduces three novel frameworks—SPOT, LUTless, and PrivShap—to systematically address the efficiency bottlenecks of PI in edge computing environments. First, to resolve memory-induced computation stalls in Convolution Neural Networks (CNNs), we propose SPOT (Structure Patching and Overlap Tweaking). SPOT replaces traditional channel wise packing with a structure patching scheme that splits inputs into independent patches, enabling parallel pipelining at the server. Additionally, an overlap tweaking mechanism minimizes redun dancy, allowing the use of smaller, faster cryptographic parameters. This approach achieves up to 3x inference speedup and 2x memory utilization improvement on mobile-grade hardware. Second,to optimize model efficiency, we introduce PrivShap, a fine-grained network linearization method. PrivShap utilizes Shapley Values to accurately quantify the importance of ReLU layers and employs a fast tree-trimming algorithm for estimation. This enables aggressive, precision guided pruning that reduces ReLU operations by 2.5x while maintaining or improving accuracy compared to state-of-the-art methods. Finally, to mitigate the communication bottleneck in Transformer models, we present LUTless. LUTless unifies complex non-linear functions (GeLU, Softmax) into a single inverse square root problem using Smoothed Maximum Unit (SMU)transformations. We introduce a communication free "double approximation" protocol that leverages IEEE 754 floating-point properties to estimate initial values locally, securing the process with a novel "share flooding" technique. LUTless re duces communication by up to 3.9x on GLUE benchmarks. Collectively, these works demonstrate that optimizing data flow, cryptographic protocols, and model architecture can make PI viable for resource-constrained edge clients.
Degree
thesis:*- Name thesis:degree_name
- Doctor of Psychology (PsyD)
- Level thesis:degree_level
- Dissertation
- Discipline thesis:degree_discipline
- Computer Science
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Xu, Xiangrui
- Contributors dc:contributor
-
- Rui Ning
- Lusi Li
- Jiang Li
- Hongyi Wu
Subjects
dc:subject × 7Rights
dc:rights- Statement dc:rights
-
- <p>In Copyright. URI: <a href="http://rightsstatements.org/vocab/InC/1.0/" target="_blank">http://rightsstatements.org/vocab/InC/1.0/</a> This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).</p>
Identifiers
dc:identifier.*- Identifier
- 9798197809261