Back to search

University of South Florida

Exploring Scalability of Multimodal User Interface Design in Virtual and Augmented Reality

Abstract

dc:description

Use of Extended Reality (XR) technology such as Augmented Reality (AR) and Virtual Reality (VR) has experienced significant growth, with continuous advances in mobile technology and head-mounted display (HMD) headset development. As applications that span more than one type of reality have started to emerge, there is a need for additional research regarding the user interfaces (UIs) developed for these multimodal systems. While some work exists towards the creation of UI design guidelines in AR and in VR, little to no work has been done in providing recommendations for designing interfaces that work successfully across multiple XR modalities. To explore this, three studies were conducted using an existing military system, the Battlespace Visualization and Interaction (BVI) architecture. Interviews were conducted with User Experience (UX) experts to gain insights into key design aspects required across modalities, resulting in a set of five initial UI design recommendations for multimodal applications that provide guidance to researchers and developers in initial stages of XR interface design. Created rapid prototypes were tested in a remote usability study using Army subject-matter experts (SMEs) to determine additional considerations to mission planning requirements and inform iterative interface designs. Implemented UIs in AR and VR modalities were tested using active duty Army soldiers to collect user interaction and usability data. Results suggest the tested UI is effective and usable for military mission planning across both VR and AR modalities, as shown by high self-reported usability metrics and mostly statistically-insignificant differences when comparing modalities.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy (Ph.D.)
Level thesis:degree_level
Dissertation
Discipline thesis:degree_discipline
Computer Science and Engineering
Grantor
University of South Florida
Year dc:date
2023

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Garcia, Sarah M.
Contributors dc:contributor
  • Marvin Andujar, Ph.D.
  • Paul Rosen, Ph.D.
  • Sylvia Thomas, Ph.D.
  • Shaun Canavan, Ph.D.
  • Michael Boyce, Ph.D.

Subjects

dc:subject × 6

Identifiers

dc:identifier.*
Repository record dc:identifier
https://digitalcommons.usf.edu/etd/10436

Chain of custody

source
Harvested from
University of South Florida
Base URL
digitalcommons.usf.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Garcia, Sarah M.. Exploring Scalability of Multimodal User Interface Design in Virtual and Augmented Reality. Dissertation thesis, University of South Florida, 2023. https://digitalcommons.usf.edu/etd/10436