Back to search

Georgia Institute of Technology

Combining Natural Language and Direct Manipulation for Human-Data Interaction through Visualizations

Abstract

dc:description.abstract

Visualization is an indispensable tool for human-data interaction, enabling people to better understand their data, identify patterns, and discover insights. Interaction plays a critical role in data visualization tools as it allows users to express their data-related goals and questions to the system. Traditionally, interaction in visualization tools is facilitated predominantly via a keyboard and mouse, following the window-menu-icon-pointer (WIMP) metaphor and the direct manipulation paradigm. However, recent advances in hardware and input recognition technology present the opportunity to reimagine interaction and explore new user experiences grounded in naturalistic human ways of interacting with people and objects in the real world. This thesis explores the design of a novel class of multimodal data visualization interfaces that augment current interaction techniques in visualization systems with natural language. I start with an assessment of the role of natural language input in visualization tools, characterizing the goals it can help people accomplish. Subsequently, I describe the design and implementation of a series of multimodal visualization systems that combine natural language and direct manipulation. Through evaluations of these systems, I capture the strengths and challenges of multimodal visualization interfaces, and highlight how they accommodate varying user interaction patterns and preferences during visual analysis. Finally, to assist future research and development, I also contribute a toolkit to help designers/developers prototype natural language-based visualization systems.

Degree

thesis:*
Level thesis:degree_level
Doctoral
Department dc:contributor.department
Computer Science
Grantor dc:publisher
Georgia Institute of Technology
Year dc:date.issued
2020

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Srinivasan, Arjun
Advisor dc:contributor.advisor
  • Stasko, John T.
Committee members dc:contributor.committeemember
  • Endert, Alex
  • Edwards, Keith
  • Lee, Bongshin
  • Setlur, Vidya

Subjects

dc:subject × 2

Rights

Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
http://hdl.handle.net/1853/64091

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Srinivasan, Arjun. Combining Natural Language and Direct Manipulation for Human-Data Interaction through Visualizations. Doctoral thesis, Georgia Institute of Technology, 2020. http://hdl.handle.net/1853/64091