Back to search

University of Washington

Towards More Intelligent, Intuitive, and Inclusive Communication with Computers in Text and Images

Abstract

dc:description.abstract

Communication is fundamental to human experience and our interaction with computers. The efficiency of human communication relies largely on the quality of the medium. Modern computing devices offer mediums such as keyboards to interact with information, yet they are still primitive (e.g., one has to press every character of a phrase) and often fail to support different user needs (e.g., limited emoji support for visually-impaired users). Thus an important question is, how to make the keyboard understand our languages like a human being, so that communication with computers can be intelligent, intuitive and inclusive? In this dissertation, I demonstrate how to design, build and evaluate communication interactions using the power of artificial intelligence. My dissertation addresses the above question in three different strands of work: (1) Intelligent text entry and editing interactions that understand the user intention; (2) Assistive systems that help blind or low vision users to communicate with pictorial information such as emojis and GIFs; (3) Models and metrics that evaluate intelligent input systems on their performance and impact on human behaviors. Together, the work demonstrates the thesis statement: Artificial intelligence can enable and improve advanced text production and accessible interactions with pictures; in addition, new metrics for text entry enable the evaluation of advanced capabilities.

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Zhang, Mingrui
Advisor dc:contributor.advisor
  • Wobbrock, Jacob JW

Subjects

dc:subject × 7

Rights

dc:rights
Statement dc:rights
  • CC BY-ND
Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
http://hdl.handle.net/1773/49049

Chain of custody

source
Harvested from
University of Washington
Base URL
digital.lib.washington.edu/server/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Zhang, Mingrui. Towards More Intelligent, Intuitive, and Inclusive Communication with Computers in Text and Images. 2022. http://hdl.handle.net/1773/49049