Back to search

Cornell University

An Investigation of Underwater Sensor Fusion and Human-Robot Interaction Approaches

Abstract

dc:description.abstract

Imagine a world where scuba divers are never alone in the vast blue expanse of the ocean—where an intelligent robotic companion swims alongside them, understanding their movements, anticipating their needs, and responding to their gestures. This thesis lays the groundwork for such a "scuba buddy" through two complementary research components. First, we address the fundamental challenge of accurately tracking divers in underwater environments, where conventional sensing technologies face significant limitations. We present a novel sensor fusion framework that intelligently combines data from stereo cameras and motion capture systems to reconstruct precise diver trajectories despite the challenges of light refraction, turbidity, and temporal desynchronization. Our approach introduces a robust algorithm that successfully compensates for temporal and spatial offsets in motion capture data, achieving trajectory reconstruction with significantly reduced error margins compared to existing methods. These reconstructed trajectories serve an important purpose, that is providing critical data for solving unknown hydrodynamic coefficients in our diver hydrodynamic model—essentially teaching our robotic system how humans move underwater. Building upon this spatial awareness foundation, the second component of our research focuses on underwater human-robot interaction through visual communication. We have developed a machine learning computer vision model that can recognize diver hand gestures through stereo video inputs, overcoming the visual distortion and variable lighting conditions inherent to underwater environments. This customizable gesture recognition system allows divers to communicate intuitively with the robotic companion using a personalized gesture dictionary tailored to specific mission requirements. The integration of these two systems—precise spatial tracking and gesture-based communication—represents a significant advancement toward autonomous underwater vehicles capable of meaningful collaboration with human divers. By enabling robots to both follow divers through complex underwater environments and understand their communicative intent, this research contributes essential components to the realization of an intelligent scuba buddy that can assist with navigation, tool retrieval, photography, sample collection, and potentially life-saving interventions in the challenging domain of underwater exploration.

Degree

thesis:*
Name thesis:degree_name
M.S., Mechanical Engineering
Level thesis:degree_level
Master of Science
Discipline thesis:degree_discipline
Mechanical Engineering
Grantor
Cornell University
Year dc:date.issued
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • He, Zilong
Chair dc:contributor.chair
  • Ferrari, Silvia
Committee member dc:contributor.committeemember
  • MacMartin, Douglas

Subjects

dc:subject × 3

Rights

dc:rights
Statement dc:rights
  • Attribution-ShareAlike 4.0 International
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Bibid
17422446
Dc Identifier Other
ProQuest Submission ID: 12409
ProQuest Publication ID: 31999786

Chain of custody

source
Harvested from
Cornell University
Base URL
ecommons.cornell.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

He, Zilong. An Investigation of Underwater Sensor Fusion and Human-Robot Interaction Approaches. Master of Science thesis, Cornell University, 2025. https://hdl.handle.net/1813/121246