Back to search

Georgia Institute of Technology

Can Language Models Protect Personal Information

Abstract

dc:description.abstract

Large multimodal language models have been shown to memorize and leak pre-training data, raising serious user privacy and information security concerns. While data leaks should be prevented, it is also crucial to examine the trade-off between the privacy protection and model utility of proposed approaches. In this thesis, we introduce PrivQA— a multimodal benchmark to assess the capabilities of language models to follow privacy protection instructions such that a selected group of personal information is protected while remaining useful to respond to other queries. PrivQA consists of both annotated human-AI dialogues as well as repurposed queries from existing question-answering (QA) datasets. We propose an access control instruction technique to iteratively self-moderate language models’ responses, which significantly improves privacy. However, through a series of red-teaming experiments, we find that adversaries can easily circumvent these protections with simple jailbreaking methods through textual and/or image inputs. We believe PRIVQA has the potential to support the development of new models with improved privacy protections, as well as the adversarial robustness of these protections.

Degree

thesis:*
Level thesis:degree_level
Masters
Department dc:contributor.department
Interactive Computing
Grantor dc:publisher
Georgia Institute of Technology
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Mendes, Ethan
Advisor dc:contributor.advisor
  • Ritter, Alan
Committee members dc:contributor.committeemember
  • Das, Sauvik
  • Xu, Wei

Subjects

dc:subject × 5

Rights

Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1853/75327

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Mendes, Ethan. Can Language Models Protect Personal Information. Masters thesis, Georgia Institute of Technology, 2024. https://hdl.handle.net/1853/75327