Back to search

Georgia Institute of Technology

Efficient and Adaptive Machine Learning for Natural Language Processing

Abstract

dc:description.abstract

NLP has recently undergone a transformational shift towards the development and application of Large Language Models (LLMs), which represent a significant leap in performance. However, the extensive computational resources and the vast textual data sets on which they rely create key challenges in scenarios with limited resources, such as limited data, computational power, memory, and specialized expertise. We argue that developing machine learning methods that could efficiently adapt with limited resources is particularly vital in the era of LLMs to make the benefits of the technology sustainable, accessible, and generalizable. In this thesis, we advocate for efficient and adaptive machine learning for NLP, endeavoring to make NLP models benefit real-world applications by diving into three parts. Part I tackles the essential elements of NLP learning, data. We propose to improve the data efficiency for developing NLP systems by (1) augmenting existing training data through hidden space manipulation and linguistically informed perturbation to alleviate the dependence on abundant data in low-resourced settings, (2) synthesizing training data based on human learning strategies to adaptively generate and organize training data for novel and nascent applications, and (3) generating evaluation data based on graphs to dynamically validate the abilities of rapidly evolving models. Part II deals with the information beyond the text in languages, structures. We explore what and how structures could be explicitly incorporated to enhance the NLP models. In particular, we show that by extracting and explicitly utilizing linguistic structures such as conversation views and graphs, the models are learned more efficiently, effectively, and controllably, and are more generalizable to different domains. Part III focuses on the adapting process of NLP models, training. We investigate how we can efficiently adapt NLP models to real-world applications by pioneering two aspects: (1) how we can fine-tune LLMs to specific tasks with fewer parameters and (2) how we can unlearn unwanted information from the learned LLMs. From improving data efficiency and incorporating structures to improving training efficiency, this thesis aims to develop efficient and adaptive machine learning for NLP through the incorporation of structures in languages to create more accessible, adaptable, and sustainable NLP systems, particularly for low-resource and novel application settings.

Degree

thesis:*
Level thesis:degree_level
Doctoral
Department dc:contributor.department
Computer Science
Grantor dc:publisher
Georgia Institute of Technology
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Chen, Jiaao
Advisor dc:contributor.advisor
  • Yang, Diyi
Committee members dc:contributor.committeemember
  • Riedl, Mark
  • Ritter, Alan
  • Kira, Zslot
  • Raffel, Colin

Subjects

dc:subject × 1

Rights

Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1853/77762

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Chen, Jiaao. Efficient and Adaptive Machine Learning for Natural Language Processing. Doctoral thesis, Georgia Institute of Technology, 2024. https://hdl.handle.net/1853/77762