Back to search

University of British Columbia

Practical ad hoc tangible interactions in augmented reality

Abstract

dc:description

By demonstrating the capability of creating enhanced immersive user experiences, Augmented Reality (AR) has received significant attention in recent years. However, the most common interaction methods with this new technology are still largely limited to in-air gesture, voice and gaze controls. Previous works have proven that involving haptic feedback by interacting with real objects can improve the users’ performance and reduce their workload. But they are still not mature enough to be adopted in commercial products. This thesis delves into this challenge, offering novel solutions to make tangible interactions in AR more practical and effective. We begin by addressing touch interactions on ad-hoc surfaces. The vision is to transform everyday, un-instrumented surfaces into expansive touch screens, enhancing AR's interactive domain. Existing systems, however, grapple with issues of touch detection accuracy and latency. By harnessing dynamic fingertip information, we introduce a cutting-edge machine learning approach that elevates touch accuracy and anticipates user interactions, significantly reducing response time. We also analyze the possibility of transforming real-world objects into passive controllers. Precise tracking of objects in a 3D space—represented by six degrees of freedom (DOF)—amplifies AR's spatial control potential. Yet, current methods either necessitate extraneous tracking markers, imposing undue burdens on users, or are too resource-intensive for real-time execution on devices like head-mounted displays. Addressing this, we present "TangiAR." This system identifies and tracks everyday objects in AR without auxiliary trackers. Its efficacy is evidenced through comprehensive tests, assessing its robustness under various challenges, such as occlusions. Moreover, we showcase its practical application, operating seamlessly on a standard HoloLens 2, underlining the immediacy of its integration potential. By bridging these tangible interaction gaps in AR, this thesis not only contributes to technical advancements but also underscores the broader implications for AR's future, emphasizing a more intuitive, responsive, and enriched user experience.

Degree

thesis:*
Name thesis:degree_name
Master of Science - MSc
Level thesis:degree_level
master's
Discipline thesis:degree_discipline
Computer Science
Grantor dc:publisher
University of British Columbia
Year dc:date
2023

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Fan, Xu (Neil)
Contributors dc:contributor
  • Xiao, Robert

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-NoDerivatives 4.0 International
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/86989

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Fan, Xu (Neil). Practical ad hoc tangible interactions in augmented reality. master's thesis, University of British Columbia, 2023. http://hdl.handle.net/2429/86989