University of British Columbia
PANORAMIA : privacy auditing of machine learning models without retraining
Abstract
dc:descriptionWe present PANORAMIA, a privacy leakage measurement framework for machine learning models that relies on membership inference attacks using generated data as non-members. By relying on generated non-member data, PANORAMIA eliminates the common dependency of privacy measurement tools on in-distribution non-member data. As a result, PANORAMIA does not modify the model, training data, or training process, and only requires access to a subset of the training data. We evaluate PANORAMIA on ML models for image and tabular data classification, as well as on large-scale language models. The theory we develop in this paper provides a meaningful step towards addressing privacy measurements in this setting and provides a more rigorous approach to privacy benchmarks for such models. We demonstrate that PANORAMIA’s privacy measurements can also be empirically valuable, for instance for providing improved measurements with more data.
Degree
thesis:*- Name thesis:degree_name
- Master of Science - MSc
- Level thesis:degree_level
- master's
- Discipline thesis:degree_discipline
- Computer Science
- Grantor dc:publisher
- University of British Columbia
- Year dc:date
- 2024
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Kazmi, Mishaal
- Contributors dc:contributor
-
- Lécuyer, Mathias
- Beschastnikh, Ivan
Rights
dc:rights- Statement dc:rights
-
- Attribution-NonCommercial-NoDerivatives 4.0 International
- Language dc:language
- eng
Identifiers
dc:identifier.*- Handle dc:identifier
- http://hdl.handle.net/2429/89301