Back to search

University of Sheffield

Automated Testing of User Interactions in Extended Reality Applications

Abstract

dc:description.abstract

Extended Reality (XR) is a rapidly emerging technology that offers immersive user experiences across diverse application domains. Unlike traditional two-dimensional graphical user interfaces, XR features interactions with six degrees of freedom, introducing new paradigms for user interaction and posing unique software engineering challenges. Among these, the validation of XR software through effective testing stands as a crucial problem requiring innovative solutions. Current research in XR software testing has produced approaches that support scene navigation and basic interaction activation. However, there remains a significant gap: existing methods cannot automatically generate or execute realistic spatial user inputs, such as grab and trigger actions via hand-held controllers or hand gestures. Furthermore, present metrics do not robustly capture the diverse coverage of these spatial interactions, and the field still lacks a systematic understanding of key testing methodologies and concerns specific to XR environments. This thesis addresses these challenges with two main objectives: (1) to investigate the landscape of XR software testing and establish foundational knowledge for XR-specific user interaction testing, and (2) to develop and evaluate new techniques that systematically generate and validate realistic user interactions in XR environments. To achieve these goals, five major contributions are presented: (1) the first systematic mapping study of XR software testing, based on an analysis of 34 published papers; (2) a novel test automation library designed around a new taxonomy of XR user interactions; (3) an automated technique for generating and executing XR user interactions, informed by a categorisation from open-source XR projects; (4) a benchmark dataset of seven XR scenes and 367 user interactions as a standard for evaluating XR testing approaches; and (5) a foundational roadmap for the future of XR devices, software, and testing, outlining structured research challenges and mitigation strategies. Collectively, these contributions advance systematic and scalable testing for XR applications, supporting both future research and practical development in this fast-evolving field.

Degree

thesis:*
Name dc:type.qualificationname
Ph.D
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
University of Sheffield
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Gu, Ruizhen
Advisors dc:contributor.advisor
  • Rojas, José Miguel
  • Shin, Donghwan
ORCID person:identifier.orcid

Rights

Language dc:language.dcterms_iso639_2
eng

Chain of custody

source
Harvested from
White Rose University Consortium
Base URL
etheses.whiterose.ac.uk/cgi/oai2
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
related terms
citation

Gu, Ruizhen. Automated Testing of User Interactions in Extended Reality Applications. doctoral thesis, University of Sheffield, 2026.