Back to search

Georgia Institute of Technology

ChatHF: Collecting Rich Human Feedback from Real-time Conversations

Abstract

dc:description.abstract

We introduce ChatHF, an interactive annotation framework for chatbot evaluation, which integrates configurable annotation within a chat interface. ChatHF can be flexibly configured to accommodate various chatbot evaluation tasks, for example detecting offensive content, identifying incorrect or misleading information in chatbot responses, and chatbot responses that might compromise privacy. It supports post-editing of chatbot outputs and supports visual inputs, in addition to an optional voice interface. ChatHF is suitable for collection and annotation of NLP datasets, and Human-Computer Interaction studies, as demonstrated in case studies on image geolocation and assisting older adults with daily activities.

Degree

thesis:*
Level thesis:degree_level
Masters
Department dc:contributor.department
Computer Science
Grantor dc:publisher
Georgia Institute of Technology
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Li, Andrew Larry
Advisor dc:contributor.advisor
  • Ritter, Alan
Committee members dc:contributor.committeemember
  • Xu, Wei
  • Wang, Zhenduo

Subjects

dc:subject × 4

Rights

Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1853/76876

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Li, Andrew Larry. ChatHF: Collecting Rich Human Feedback from Real-time Conversations. Masters thesis, Georgia Institute of Technology, 2024. https://hdl.handle.net/1853/76876