Back to search

University of Leeds

Physics-Based Multi-Object Tracking for Non-Prehensile Robotic Manipulation

Abstract

dc:description.abstract

This thesis presents an algorithm to track objects, particularly designed to address the challenge of “tracking objects in cluttered environments.” The key point is that the robot can continuously track the target objects, even when they are heavily occluded. This method can be used to retrieve an object from inside a fridge or a crowded shelf, where the robot needs to know the 6D pose of each object in the scene to decide how to move obstacles and then locate and pick up the target item. Accurately performing the task requires knowing the 6D pose of each object, which enables motion planning. However, obtaining such information in real-world environments is extremely difficult. Traditional motion planning usually assumes access to full 6D poses, often using tools like OptiTrack[1] with reflective markers attached to objects. Unfortunately, placing markers on every object and setting up an OptiTrack system is not feasible in practical situations. Another option is to use RGB-based 6D pose estimation systems, but these become failures when objects are heavily occluded—something very common in realworld cluttered scenes. This work addresses these challenges by proposing a tracking algorithm that combines physics predictions with vision information based on the particle filtering algorithm. The system uses the physics simulation and the robot’s joint states as the motion model input and camera images as the observation model input. This setup helps recover the object’s pose even when it becomes temporarily occluded. Initially, the method used only RGB images to track a single object. Results showed that the physics-based approach worked well, outperforming baseline methods in tracking accuracy. However, using only RGB input struggled with occlusions and made it hard to handle multiple objects. To overcome this, I proposed a new approach that uses depth images and a visibility score indicator. This enhanced method performs better than baseline systems when tracking multiple objects and dealing with heavy occlusions, while still maintaining good tracking accuracy.

Degree

thesis:*
Name dc:type.qualificationname
Ph.D
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
University of Leeds
Year dc:date.issued
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Xu, Zisong
Advisors dc:contributor.advisor
  • Dogar, Mehmet
  • Wang, He

Rights

Language dc:language.dcterms_iso639_2
eng

Chain of custody

source
Harvested from
White Rose University Consortium
Base URL
etheses.whiterose.ac.uk/cgi/oai2
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
related terms
citation

Xu, Zisong. Physics-Based Multi-Object Tracking for Non-Prehensile Robotic Manipulation. doctoral thesis, University of Leeds, 2025.