Back to search

Cal Poly

Real Time Markerless 3D Hand Tracking for Intuitive Robotic Arm Control

Abstract

dc:description.abstract

This thesis presents a real-time, markerless motion-capture system for intuitive control of a multi-degree-of-freedom robotic arm. Using two synchronized RGB cameras and stereo-vision triangulation, the system reconstructs the three-dimensional position of a user’s hand without the need for physical markers or wearable sensors. The reconstructed 3D coordinates are mapped directly to the end-effector position of the Quanser QArm, a four-degree-of-freedom educational manipulator. Developed entirely in Python, the system integrates stereo camera calibration, 3D hand tracking with MediaPipe, coordinate transformation, and inverse kinematics into a unified real-time control pipeline operating at approximately 200 Hz. Calibration is performed through a checkerboard-based stereo method to obtain precise intrinsic and extrinsic camera parameters. The user’s baseline hand position is first recorded and used to define a neutral reference point; subsequent movements are then interpreted as relative displacements within the robot’s workspace. Experimental evaluation includes idle-stability tests, controlled path tracking, high-speed motion trials, latency characterization, and gripper-actuation reliability. Performance was compared across two stereo-camera configurations: Logitech C270HD webcams operating at 720p and 30 fps, and OBSBot Tiny 2 cameras operating at 1080p and 60 fps. The system consistently maintained an idle percent error below five percent, achieved reliable gesture-based gripper activation in more than ninety-five percent of trials, and exhibited a step-response time constant of approximately 1.1 ± 0.1 s. These results confirm that low-cost stereo cameras, coupled with open-source software, can deliver responsive, stable, and intuitive robotic teleoperation suitable for education, research, and remote operation in hazardous environments.

Degree

thesis:*
Name thesis:degree_name
MS in Mechanical Engineering
Discipline thesis:degree_discipline
Mechanical Engineering

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Featherson, Dylan G
Contributors dc:contributor
  • Behnam Ghalamchi
  • Mechanical Engineering
  • College of Engineering

Subjects

dc:subject × 7

Identifiers

dc:identifier.*
Repository record dc:identifier
https://digitalcommons.calpoly.edu/theses/3200

Chain of custody

source
Harvested from
Cal Poly
Base URL
digitalcommons.calpoly.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Featherson, Dylan G. Real Time Markerless 3D Hand Tracking for Intuitive Robotic Arm Control. https://digitalcommons.calpoly.edu/theses/3200