University of South Florida
Bridging Virtual Robots and Physical Tasks via Augmented Reality
Abstract
dc:descriptionEntry to human-robot interaction research, e.g., conducting empirical experiments, faces a significant economic barrier due to the high cost of physical robots, ranging from thousands to tens of thousands. This cost issue also severely limits the field’s ability to replicate user studies and reproduce the results to verify their reliability, thus offering more confidence to incorporate these findings. Although virtual reality (VR) user studies present a potential solution, it is unclear whether we can confidently transfer the findings to physical robots and physical environments because VR isolates both the physical robot and the physical world where robots operate. To address this issue, we leveraged augmented reality (AR) only to simulate a virtual robot but retained the physical environment. Specifically, we implemented three conditions: (1) a physical robot interacted with a physical object, (2) a physical robot interacted with a virtual object, and (3) a virtual robot interacted with a virtual object. We used the Unity 3D engine and the Gazebo simulator to replicate the FetchIt! Mobile Manipulation Challenge environment and developed a communication bridge to synchronize the poses of virtual robots and objects with their physical counterparts in real-time. We implemented the pick-and-place operations for virtual and physical robots using the Robot Operating System (ROS), including designing custom ROS nodes for trajectory planning, motion execution, and arm-gripper publisher and subscriber between ROS and Unity. Moreover, we designed a human-subjects study, approved by the Institutional Review Board (IRB), to collect subjective experience measures, specifically usability, trust, and personal preference, and task performance across physical and virtual robot conditions. Although the user study will be conducted as part of future research, this thesis lays the technical groundwork by implementing all required robotic and AR components and validating their functionality across three experimental conditions.
Degree
thesis:*- Name thesis:degree_name
- Master of Science (M.S.)
- Level thesis:degree_level
- Thesis
- Discipline thesis:degree_discipline
- Computer Science and Engineering
- Grantor
- University of South Florida
- Year dc:date
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Kong, Xiangfei
- Contributors dc:contributor
-
- Zhao Han, Ph.D.
- Jing Wang, Ph.D.
- Julia Woodward, Ph.D.
Subjects
dc:subject × 2Identifiers
dc:identifier.*- Repository record dc:identifier
- https://digitalcommons.usf.edu/etd/10876