{"id":{"repo_id":"calpoly","oai_identifier":"oai:digitalcommons.calpoly.edu:theses-4855"},"canonical_url":"https://search.ndltd.org/etd/calpoly/oai:digitalcommons.calpoly.edu:theses-4855","repository":{"repo_id":"calpoly","name":"Cal Poly","base_url":"https://digitalcommons.calpoly.edu/do/oai/"},"display":{"title":"Real Time Markerless 3D Hand Tracking for Intuitive Robotic Arm Control","abstract":"This thesis presents a real-time, markerless motion-capture system for intuitive control of a multi-degree-of-freedom robotic arm. Using two synchronized RGB cameras and stereo-vision triangulation, the system reconstructs the three-dimensional position of a user’s hand without the need for physical markers or wearable sensors. The reconstructed 3D coordinates are mapped directly to the end-effector position of the Quanser QArm, a four-degree-of-freedom educational manipulator. Developed entirely in Python, the system integrates stereo camera calibration, 3D hand tracking with MediaPipe, coordinate transformation, and inverse kinematics into a unified real-time control pipeline operating at approximately 200 Hz. Calibration is performed through a checkerboard-based stereo method to obtain precise intrinsic and extrinsic camera parameters. The user’s baseline hand position is first recorded and used to define a neutral reference point; subsequent movements are then interpreted as relative displacements within the robot’s workspace. Experimental evaluation includes idle-stability tests, controlled path tracking, high-speed motion trials, latency characterization, and gripper-actuation reliability. Performance was compared across two stereo-camera configurations: Logitech C270HD webcams operating at 720p and 30 fps, and OBSBot Tiny 2 cameras operating at 1080p and 60 fps. The system consistently maintained an idle percent error below five percent, achieved reliable gesture-based gripper activation in more than ninety-five percent of trials, and exhibited a step-response time constant of approximately 1.1 ± 0.1 s. These results confirm that low-cost stereo cameras, coupled with open-source software, can deliver responsive, stable, and intuitive robotic teleoperation suitable for education, research, and remote operation in hazardous environments.","abstract_html":"This thesis presents a real-time, markerless motion-capture system for intuitive control of a multi-degree-of-freedom robotic arm. Using two synchronized RGB cameras and stereo-vision triangulation, the system reconstructs the three-dimensional position of a user’s hand without the need for physical markers or wearable sensors. The reconstructed 3D coordinates are mapped directly to the end-effector position of the Quanser QArm, a four-degree-of-freedom educational manipulator. Developed entirely in Python, the system integrates stereo camera calibration, 3D hand tracking with MediaPipe, coordinate transformation, and inverse kinematics into a unified real-time control pipeline operating at approximately 200 Hz. Calibration is performed through a checkerboard-based stereo method to obtain precise intrinsic and extrinsic camera parameters. The user’s baseline hand position is first recorded and used to define a neutral reference point; subsequent movements are then interpreted as relative displacements within the robot’s workspace. Experimental evaluation includes idle-stability tests, controlled path tracking, high-speed motion trials, latency characterization, and gripper-actuation reliability. Performance was compared across two stereo-camera configurations: Logitech C270HD webcams operating at 720p and 30 fps, and OBSBot Tiny 2 cameras operating at 1080p and 60 fps. The system consistently maintained an idle percent error below five percent, achieved reliable gesture-based gripper activation in more than ninety-five percent of trials, and exhibited a step-response time constant of approximately 1.1 ± 0.1 s. These results confirm that low-cost stereo cameras, coupled with open-source software, can deliver responsive, stable, and intuitive robotic teleoperation suitable for education, research, and remote operation in hazardous environments.","abstract_has_math":false,"creators":["Featherson, Dylan G"],"institution":null,"degree_name":"MS in Mechanical Engineering","degree_level":null,"degree_discipline":"Mechanical Engineering","degree_department":null,"school":null,"contributors":["Behnam Ghalamchi","Mechanical Engineering","College of Engineering"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":null,"date_issued":"","date_published":null,"updated_at":"2026-11-19T07:12:41Z","subjects":["Markerless motion capture","robotic arm control","stereo vision","3D hand tracking","python","mediapipe","Acoustics, Dynamics, and Controls"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://digitalcommons.calpoly.edu/theses/3200","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://digitalcommons.calpoly.edu/theses/3200","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://digitalcommons.calpoly.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Adigitalcommons.calpoly.edu%3Atheses-4855","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Behnam Ghalamchi","Mechanical Engineering","College of Engineering"]},{"key":"dc:creator","label":"Author","values":["Featherson, Dylan G"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.available","label":"Dc Date Available","values":["2025-12-09T08:00:00Z"]},{"key":"dc:date.created","label":"Dc Date Created","values":["2025-12-01T08:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Mechanical Engineering"]},{"key":"thesis:degree_name","label":"Degree Name","values":["MS in Mechanical Engineering"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Markerless motion capture","robotic arm control","stereo vision","3D hand tracking","python","mediapipe","Acoustics, Dynamics, and Controls"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.calpoly.edu/theses/3200"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["This thesis presents a real-time, markerless motion-capture system for intuitive control of a multi-degree-of-freedom robotic arm. Using two synchronized RGB cameras and stereo-vision triangulation, the system reconstructs the three-dimensional position of a user’s hand without the need for physical markers or wearable sensors. The reconstructed 3D coordinates are mapped directly to the end-effector position of the Quanser QArm, a four-degree-of-freedom educational manipulator. Developed entirely in Python, the system integrates stereo camera calibration, 3D hand tracking with MediaPipe, coordinate transformation, and inverse kinematics into a unified real-time control pipeline operating at approximately 200 Hz. Calibration is performed through a checkerboard-based stereo method to obtain precise intrinsic and extrinsic camera parameters. The user’s baseline hand position is first recorded and used to define a neutral reference point; subsequent movements are then interpreted as relative displacements within the robot’s workspace. Experimental evaluation includes idle-stability tests, controlled path tracking, high-speed motion trials, latency characterization, and gripper-actuation reliability. Performance was compared across two stereo-camera configurations: Logitech C270HD webcams operating at 720p and 30 fps, and OBSBot Tiny 2 cameras operating at 1080p and 60 fps. The system consistently maintained an idle percent error below five percent, achieved reliable gesture-based gripper activation in more than ninety-five percent of trials, and exhibited a step-response time constant of approximately 1.1 ± 0.1 s. These results confirm that low-cost stereo cameras, coupled with open-source software, can deliver responsive, stable, and intuitive robotic teleoperation suitable for education, research, and remote operation in hazardous environments."]},{"key":"dc:title","label":"Title","values":["Real Time Markerless 3D Hand Tracking for Intuitive Robotic Arm Control"]}]}],"canonical_facts":{"dc:contributor":["Behnam Ghalamchi","Mechanical Engineering","College of Engineering"],"dc:creator":["Featherson, Dylan G"],"dc:date.available":["2025-12-09T08:00:00Z"],"dc:date.created":["2025-12-01T08:00:00Z"],"dc:description.abstract":["This thesis presents a real-time, markerless motion-capture system for intuitive control of a multi-degree-of-freedom robotic arm. Using two synchronized RGB cameras and stereo-vision triangulation, the system reconstructs the three-dimensional position of a user’s hand without the need for physical markers or wearable sensors. The reconstructed 3D coordinates are mapped directly to the end-effector position of the Quanser QArm, a four-degree-of-freedom educational manipulator. Developed entirely in Python, the system integrates stereo camera calibration, 3D hand tracking with MediaPipe, coordinate transformation, and inverse kinematics into a unified real-time control pipeline operating at approximately 200 Hz. Calibration is performed through a checkerboard-based stereo method to obtain precise intrinsic and extrinsic camera parameters. The user’s baseline hand position is first recorded and used to define a neutral reference point; subsequent movements are then interpreted as relative displacements within the robot’s workspace. Experimental evaluation includes idle-stability tests, controlled path tracking, high-speed motion trials, latency characterization, and gripper-actuation reliability. Performance was compared across two stereo-camera configurations: Logitech C270HD webcams operating at 720p and 30 fps, and OBSBot Tiny 2 cameras operating at 1080p and 60 fps. The system consistently maintained an idle percent error below five percent, achieved reliable gesture-based gripper activation in more than ninety-five percent of trials, and exhibited a step-response time constant of approximately 1.1 ± 0.1 s. These results confirm that low-cost stereo cameras, coupled with open-source software, can deliver responsive, stable, and intuitive robotic teleoperation suitable for education, research, and remote operation in hazardous environments."],"dc:identifier":["https://digitalcommons.calpoly.edu/theses/3200"],"dc:subject":["Markerless motion capture","robotic arm control","stereo vision","3D hand tracking","python","mediapipe","Acoustics, Dynamics, and Controls"],"dc:title":["Real Time Markerless 3D Hand Tracking for Intuitive Robotic Arm Control"],"thesis:degree_discipline":["Mechanical Engineering"],"thesis:degree_name":["MS in Mechanical Engineering"]},"updated_at":"2026-11-19T07:12:41Z"}