Georgia Institute of Technology
A Perturbation Approach to Differential Privacy for Deep Learning based Speech Processing
Abstract
dc:description.abstractIn the pursuit of high-performance speech applications, deep neural networks (DNNs) trained on speech data have become the cornerstone. As new data regulations, such as GDPR and CCPA, demand enhanced privacy protections, it is crucial to thwart security threats arising from query-based malevolent attacks. Differential privacy (DP), a technique offering a mathematical definition for privacy loss evaluation under these attacks, becomes invaluable. This dissertation embarks on the journey of developing a perturbation-based machine learning framework, ensuring that DNNs maintain speech model performance whilst satisfying DP constraints. We devise mechanisms for vector-space distortion, achieved by introducing bounded noise to speech training data, and subsequently scrutinize the perturbations through statistical measurements and model estimations. Utilizing both Laplace and Gaussian noise, our theoretical explorations guarantee the privacy budget within a bounded max-divergence measurement, using ensemble learning. We exploit the Lipschitz continuity properties of DNNs (e.g., w2v2-RNN-T) to estimate model robustness under varying vector-space perturbations. To substantiate our theoretical underpinnings, we employ speech recognition in isolated speech commands and large vocabulary continuous speech settings. We extend our proposed framework to popular speech and acoustic modeling tasks, probing the delicate balance between model performance and DP budgets. Two training frameworks are introduced to counteract model degradation: (i) teacher-ensemble learning and (ii) sub-sampling-based training. Our discoveries reveal that, despite cutting-edge speaker protection techniques, speaker identity information may still be compromised without DP-based data extraction or training methods. Finally, we assimilate decentralized and federated training pipelines of DNNs into our privacy-preserving speech processing mechanism, examining potential solutions for on-device or cloud-based speech applications catering to end-users.
Degree
thesis:*- Level thesis:degree_level
- Doctoral
- Department dc:contributor.department
- Electrical and Computer Engineering
- Grantor dc:publisher
- Georgia Institute of Technology
- Year dc:date.issued
- 2023
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Yang, Chao-Han Huck
- Advisor dc:contributor.advisor
-
- Lee, Chin-Hui
- Committee members dc:contributor.committeemember
-
- Heck, Larry P.
- Moore, Elliott II
- Chen, Pin-Yu
- Siniscalchi, Sabato Marco
Subjects
dc:subject × 4Rights
- Language dc:language.iso
- en_US
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/1853/75094