Back to search

Georgia Institute of Technology

User-centered Programmatic Data Labeling

Abstract

dc:description.abstract

The lack of labeled training data is a major challenge impeding the practical application of machine learning (ML) techniques. Therefore, ML practitioners have increasingly turned to programmatic supervision methods, in which a larger volume of programmatically generated, but often noisier, labeled examples is used in lieu of hand-labeled examples. In this paradigm, supervision sources are expressed as labeling functions (LFs), and a label model aggregates the output of multiple LFs to produce training labels. However, the current process of developing LF relies on the expertise of the user and can be inaccessible for non-experts, particularly when dealing with video data. In addition, existing label models require hyperparameters and dataset-specific training for each dataset and can yield non-deterministic results, further complicating the process for non-expert users. This dissertation aims to improve the usability of programmatic data labeling through a three-part research approach. First, I explore how to improve usability by specializing programmatic data labeling to the task at hand. I examine a specific task (entity matching) as a case study to develop a specialized Integrated Development Environment, facilitating the development, debugging, aggregation, and management of LFs. Second, I extend the labeling function interface by introducing a visual interface, allowing users to create LFs for video data intuitively without any coding. Specifically, I propose a visual query language for retrieving video clips across datasets, enabling non-expert users to easily develop LFs with mouse drag-and-drop. Third, to obviate user involvement in the label model, I present a hyper label model that requires neither hyperparameters nor dataset-specific training, while producing deterministic results with superior accuracy and efficiency. The proposed method also offers the first analytical optimal solution to the problem.

Degree

thesis:*
Level thesis:degree_level
Doctoral
Department dc:contributor.department
Computer Science
Grantor dc:publisher
Georgia Institute of Technology
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Wu, Renzhi
Advisor dc:contributor.advisor
  • Chu, Xu
Committee members dc:contributor.committeemember
  • Rong, Kexin
  • Arulraj, Joy
  • Navathe, Shamkant
  • He, Yeye

Subjects

dc:subject × 2

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1853/75209

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Wu, Renzhi. User-centered Programmatic Data Labeling. Doctoral thesis, Georgia Institute of Technology, 2024. https://hdl.handle.net/1853/75209