Back to search

University of South Florida

Bridging Virtual Robots and Physical Tasks via Augmented Reality

Abstract

dc:description

Entry to human-robot interaction research, e.g., conducting empirical experiments, faces a significant economic barrier due to the high cost of physical robots, ranging from thousands to tens of thousands. This cost issue also severely limits the field’s ability to replicate user studies and reproduce the results to verify their reliability, thus offering more confidence to incorporate these findings. Although virtual reality (VR) user studies present a potential solution, it is unclear whether we can confidently transfer the findings to physical robots and physical environments because VR isolates both the physical robot and the physical world where robots operate. To address this issue, we leveraged augmented reality (AR) only to simulate a virtual robot but retained the physical environment. Specifically, we implemented three conditions: (1) a physical robot interacted with a physical object, (2) a physical robot interacted with a virtual object, and (3) a virtual robot interacted with a virtual object. We used the Unity 3D engine and the Gazebo simulator to replicate the FetchIt! Mobile Manipulation Challenge environment and developed a communication bridge to synchronize the poses of virtual robots and objects with their physical counterparts in real-time. We implemented the pick-and-place operations for virtual and physical robots using the Robot Operating System (ROS), including designing custom ROS nodes for trajectory planning, motion execution, and arm-gripper publisher and subscriber between ROS and Unity. Moreover, we designed a human-subjects study, approved by the Institutional Review Board (IRB), to collect subjective experience measures, specifically usability, trust, and personal preference, and task performance across physical and virtual robot conditions. Although the user study will be conducted as part of future research, this thesis lays the technical groundwork by implementing all required robotic and AR components and validating their functionality across three experimental conditions.

Degree

thesis:*
Name thesis:degree_name
Master of Science (M.S.)
Level thesis:degree_level
Thesis
Discipline thesis:degree_discipline
Computer Science and Engineering
Grantor
University of South Florida
Year dc:date
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Kong, Xiangfei
Contributors dc:contributor
  • Zhao Han, Ph.D.
  • Jing Wang, Ph.D.
  • Julia Woodward, Ph.D.

Subjects

dc:subject × 2

Identifiers

dc:identifier.*
Repository record dc:identifier
https://digitalcommons.usf.edu/etd/10876

Chain of custody

source
Harvested from
University of South Florida
Base URL
digitalcommons.usf.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Kong, Xiangfei. Bridging Virtual Robots and Physical Tasks via Augmented Reality. Thesis thesis, University of South Florida, 2025. https://digitalcommons.usf.edu/etd/10876