Back to search

Cornell University

Threats and Countermeasures in Machine Learning Applications

Abstract

dc:description.abstract

Machine learning as a technique of automatically constructing programs from past data for making future predictions has led to its adoption in many diverse areas of applications. Similar to traditional programs written by expert software engineers, these machine learned programs are also subject to safety requirements when applied to sensitive usage scenarios. In hypothetical situations, if a certain degree of knowledge about the internal details of the machine learned models or their training method is exposed to an adversary, the behavior of the model may not be guaranteed; even worse, the adversary may arbitrarily alter the model's behavior to his or her desire. Throughout the training and deployment pipeline, an adversary may leverage vulnerabilities in the training procedure or model architecture in several ways. First, an adversary may exploit the assumption of identical distribution between training and test data. During test time, inputs to the model can be carefully constructed to fall outside of the natural data distribution and deliberately cause malfunctioning of the system. Second, during training, an adversary may inject secret functionalities into the model and release it publicly or sell it to another party. At test time, the injected functionality can serve as a backdoor to allow the adversary manipulation of the deployed system. Third, training data for the model is often memorized in full or in part due to outlier samples or excess capacity. These memorized samples may be extracted from the model parameters or its predictions to allow an adversary unauthorized access to a private training database. Our goal in this thesis is to expose these potential hazards in real world applications, understand the root cause for the existence of these loopholes, and devise or provide insight into potential solutions. We will focus on the following three main topics: (i) black-box attacks against the prediction of machine learning models; (ii) embedding Trojan horse models into a benign transport model; and (iii) provable removal of training data from a model for protecting data privacy. We hope that by public discussion of these possible safety concerns of machine learning, we can raise awareness and advocate ongoing research to improve the security and privacy of learning systems.

Degree

thesis:*
Name thesis:degree_name
Ph. D., Computer Science
Level thesis:degree_level
Doctor of Philosophy
Discipline thesis:degree_discipline
Computer Science
Grantor
Cornell University
Year dc:date.issued
2020

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Guo, Chuan
Chair dc:contributor.chair
  • Weinberger, Kilian
Committee members dc:contributor.committeemember
  • Sridharan, Karthik
  • Joachims, Thorsten

Subjects

dc:subject × 3

Rights

Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Bibid
13254330
Dc Identifier Other
ProQuest Submission ID: 11933
ProQuest Publication ID: 27838105

Chain of custody

source
Harvested from
Cornell University
Base URL
ecommons.cornell.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Guo, Chuan. Threats and Countermeasures in Machine Learning Applications. Doctor of Philosophy thesis, Cornell University, 2020. https://hdl.handle.net/1813/70425