Back to search

Georgia Institute of Technology

Detecting and Mitigating Pedagogical Risks in Large Language Models with Visual Analytics

Abstract

dc:description.abstract

The advent of large language models (LLMs) has catalyzed a surge in LLM-powered educational technologies, enabling transformational advances that can empower learner agency, deliver personalized study materials, and promote active learning. Data scientists in education, called learning engineers, are at the forefront of this technological revolution, leading efforts to integrate LLMs into adaptive learning experiences. Yet persistent pedagogical risks, from bias and hallucinations to unfair grading and misalignment with instructional goals, highlight a critical gap. Existing tools for selecting, fine-tuning, and evaluating LLMs are not designed to address the unique challenges of educational contexts, limiting the broader adoption of LLM-powered tools for learning. To address this gap, this thesis explores how human-in-the-loop visual analytics can equip learning engineers to better discover, understand, and address these risks by combining automated analysis with interactive visualizations that support more informed, iterative decision-making throughout the LLM development lifecycle. Specifically, this thesis contributes to visual analytics, human computer interaction, artificial intelligence, and educational technology across several related thrusts. First, KnowledgeVIS enables visual exploration of latent biases in foundation LLMs during model selection, towards uncovering harmful LLM biases and stereotypes. Second, iScore visualizes internal and external model perturbations to diagnosis unintended LLM decision-making criteria during fine-tuning. Third, our novel framework for measuring and visualizing LLM trustworthiness issues makes pedagogical risks interpretable and traceable during LLM evaluation. Fourth, TrustyVis implements modular visual analytics components for reusable integration into real-world workflows, broadening access to visual analytics for learning engineers.

Degree

thesis:*
Name thesis:degree_name
Human-Centered Computing, PhD
Department thesis:degree_department
Interactive Computing
Grantor
Georgia Institute of Technology
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Coscia, Adam Joseph
Advisor dc:contributor.advisor
  • Endert, Alex
ORCID person:identifier.orcid

Rights

Language dc:language.iso
English

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1853/82145

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Coscia, Adam Joseph. Detecting and Mitigating Pedagogical Risks in Large Language Models with Visual Analytics. Georgia Institute of Technology, 2026. https://hdl.handle.net/1853/82145