Back to search

Georgia Institute of Technology

Towards Efficiently and Reliably Harnessing Pre-trained Language Models: A Data-centric Lens

Abstract

dc:description.abstract

Large Language Models (LLMs) excel across diverse tasks, but their performance depends heavily on high-quality data, which is often scarce and costly. In this talk, I will present three lines of research focused on developing data-centric approaches for enhancing LLM performance: (1) Cost-Efficient Data Collection: I discuss data-selection techniques that enable resource-efficient fine-tuning of LMs to boost performance with minimal data costs. (2) LLM-assisted Synthetic Data Generation: I design synthetic data generation approaches using LLMs to create diverse and unbiased datasets for classical NLP tasks, as well as generate synthetic critiques to improve reward modeling for LLM alignment. (3) Instruction Fine-Tuning for Trustworthy LLMs: I present a data-efficient instruction fine-tuning pipeline RankRAG that fine-tunes a single LM to handle both context ranking and answer generation, improving the efficacy of retrieval-augmented language models. These thrusts collectively form a comprehensive approach to the creation and utilization of high-quality data, holding significant promise for advancing the application of pretrained language models across diverse domains such as news, social media, biomedical, and healthcare. The primary goal of my work is to develop data-centric AI solutions that emphasize efficiency, reliability, and practical impact in the context of large-scale models.

Degree

thesis:*
Level thesis:degree_level
Doctoral
Department dc:contributor.department
Computational Science and Engineering
Grantor dc:publisher
Georgia Institute of Technology
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Yu, Yue
Advisor dc:contributor.advisor
  • Zhang, Chao
Committee members dc:contributor.committeemember
  • Dai, Bo
  • Chau, Polo
  • Mussmann, Stephen
  • Mitchell, Cassie
  • Liu, Jialu

Subjects

dc:subject × 4

Rights

Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1853/76939

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Yu, Yue. Towards Efficiently and Reliably Harnessing Pre-trained Language Models: A Data-centric Lens. Doctoral thesis, Georgia Institute of Technology, 2024. https://hdl.handle.net/1853/76939