Back to search

University of Waterloo

Analyzing Threats of Large-Scale Machine Learning Systems

Abstract

dc:description.abstract

Large-scale machine learning systems such as ChatGPT rapidly transform how we interact with and trust digital media. However, the emergence of such a powerful technology faces a dual-use dilemma. While it can have many positive societal impacts in providing equitable access to information, ML systems can also be misused by untrustworthy entities to cause intentional harm. For example, a system could unintentionally disclose private information about its training data and jeopardize the privacy of individuals in the training data. The system's generated content could also be misused for unethical purposes, such as eroding trust in digital media by misrepresenting generating content as authentic. Providing untrustworthy users with these new capabilities could amplify potential negative consequences emerging through this technology, such as a proliferation of deep fakes or disinformation. I analyze these threats from two perspectives: (i) Data leakage, when the model cannot be trusted because it has memorized private information during training, and (ii) Misuse when users cannot be trusted to use the system for its intended purposes. This thesis presents five projects to assess these risks to the privacy and security of ML systems and evaluates the reliability of known countermeasures. To do so, I assess the privacy risks of extracting Personally Identifiable Information from language models trained with differential privacy. As a method of controlling unintended use, I study the effectiveness and robustness of fingerprinting and watermarking methods to detect the provenance of models and their generated content.

Degree

thesis:*
Grantor dc:publisher
University of Waterloo
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Lukas, Nils
Advisor dc:contributor.advisor
  • Kerschbaum, Florian

Subjects

dc:subject × 6

Rights

Language dc:language.iso
en

Identifiers

dc:identifier.*
Handle dc:identifier.uri
http://hdl.handle.net/10012/20355

Chain of custody

source
Harvested from
University of Waterloo
Base URL
dspacemainprd01.lib.uwaterloo.ca/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Lukas, Nils. Analyzing Threats of Large-Scale Machine Learning Systems. University of Waterloo, 2024. http://hdl.handle.net/10012/20355