Georgia Institute of Technology
Efficient and Adaptive Machine Learning for Natural Language Processing
Abstract
dc:description.abstractNLP has recently undergone a transformational shift towards the development and application of Large Language Models (LLMs), which represent a significant leap in performance. However, the extensive computational resources and the vast textual data sets on which they rely create key challenges in scenarios with limited resources, such as limited data, computational power, memory, and specialized expertise. We argue that developing machine learning methods that could efficiently adapt with limited resources is particularly vital in the era of LLMs to make the benefits of the technology sustainable, accessible, and generalizable. In this thesis, we advocate for efficient and adaptive machine learning for NLP, endeavoring to make NLP models benefit real-world applications by diving into three parts. Part I tackles the essential elements of NLP learning, data. We propose to improve the data efficiency for developing NLP systems by (1) augmenting existing training data through hidden space manipulation and linguistically informed perturbation to alleviate the dependence on abundant data in low-resourced settings, (2) synthesizing training data based on human learning strategies to adaptively generate and organize training data for novel and nascent applications, and (3) generating evaluation data based on graphs to dynamically validate the abilities of rapidly evolving models. Part II deals with the information beyond the text in languages, structures. We explore what and how structures could be explicitly incorporated to enhance the NLP models. In particular, we show that by extracting and explicitly utilizing linguistic structures such as conversation views and graphs, the models are learned more efficiently, effectively, and controllably, and are more generalizable to different domains. Part III focuses on the adapting process of NLP models, training. We investigate how we can efficiently adapt NLP models to real-world applications by pioneering two aspects: (1) how we can fine-tune LLMs to specific tasks with fewer parameters and (2) how we can unlearn unwanted information from the learned LLMs. From improving data efficiency and incorporating structures to improving training efficiency, this thesis aims to develop efficient and adaptive machine learning for NLP through the incorporation of structures in languages to create more accessible, adaptable, and sustainable NLP systems, particularly for low-resource and novel application settings.
Degree
thesis:*- Level thesis:degree_level
- Doctoral
- Department dc:contributor.department
- Computer Science
- Grantor dc:publisher
- Georgia Institute of Technology
- Year dc:date.issued
- 2024
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Chen, Jiaao
- Advisor dc:contributor.advisor
-
- Yang, Diyi
- Committee members dc:contributor.committeemember
-
- Riedl, Mark
- Ritter, Alan
- Kira, Zslot
- Raffel, Colin
Subjects
dc:subject × 1Rights
- Language dc:language.iso
- en_US
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/1853/77762