Georgia Institute of Technology
Exploring the Potential of Large Language Models as Computational Engines and Virtual Domain Experts for Visual Data Analysis
Abstract
dc:description.abstractAdvances in generative artificial intelligence have led to the development of pre-trained large language models (LLMs) which are widely available and broadly useful. For data visualization researchers, LLMs' vast domain knowledge and computational power have the promise to extend existing research threads in exciting directions. However, well-documented hallucination and inconsistency issues with LLMs can inhibit visualization system performance and erode user trust. We also have limited formal understanding of LLMs’ ability to help analysts with specific tasks. In my dissertation work, I study the potential use of LLMs as “virtual domain experts” during visual data analysis. This includes two main goals: First, to evaluate LLMs at applying their knowledge bases to data- and chart-centric tasks; and second, to study user satisfaction and trust for LLM-powered visualization systems. I address the first goal through an empirical evaluation of the GPT-4V multimodal language model on a suite of visualization literacy tasks, demonstrating LLM performance at reading and understanding visualizations. In subsequent work, I address both goals by assessing LLMs’ domain knowledge and generative capabilities on two specific tasks: question answering and data integration. For each task, I present formative studies, empirical evaluations, and design probes using proof-of-concept visualization systems, exploring both technical and human-centered perspectives on the use of LLMs during visual data analysis.
Degree
thesis:*- Name thesis:degree_name
- Computer Science, PhD
- Department thesis:degree_department
- Interactive Computing
- Grantor
- Georgia Institute of Technology
- Year dc:date.issued
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Bendeck, Alexander E.
- Advisor dc:contributor.advisor
-
- Stasko, John T.
- Committee members dc:contributor.committeemember
-
- Endert, Alex
- Andris, Clio
- Xiong, Cindy
- Maciejewski, Ross
- ORCID person:identifier.orcid
Rights
- Language dc:language.iso
- English
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/1853/81721