University of Leeds
Physics-Based Multi-Object Tracking for Non-Prehensile Robotic Manipulation
Abstract
dc:description.abstractThis thesis presents an algorithm to track objects, particularly designed to address the challenge of “tracking objects in cluttered environments.” The key point is that the robot can continuously track the target objects, even when they are heavily occluded. This method can be used to retrieve an object from inside a fridge or a crowded shelf, where the robot needs to know the 6D pose of each object in the scene to decide how to move obstacles and then locate and pick up the target item. Accurately performing the task requires knowing the 6D pose of each object, which enables motion planning. However, obtaining such information in real-world environments is extremely difficult. Traditional motion planning usually assumes access to full 6D poses, often using tools like OptiTrack[1] with reflective markers attached to objects. Unfortunately, placing markers on every object and setting up an OptiTrack system is not feasible in practical situations. Another option is to use RGB-based 6D pose estimation systems, but these become failures when objects are heavily occluded—something very common in realworld cluttered scenes. This work addresses these challenges by proposing a tracking algorithm that combines physics predictions with vision information based on the particle filtering algorithm. The system uses the physics simulation and the robot’s joint states as the motion model input and camera images as the observation model input. This setup helps recover the object’s pose even when it becomes temporarily occluded. Initially, the method used only RGB images to track a single object. Results showed that the physics-based approach worked well, outperforming baseline methods in tracking accuracy. However, using only RGB input struggled with occlusions and made it hard to handle multiple objects. To overcome this, I proposed a new approach that uses depth images and a visibility score indicator. This enhanced method performs better than baseline systems when tracking multiple objects and dealing with heavy occlusions, while still maintaining good tracking accuracy.
Degree
thesis:*- Name dc:type.qualificationname
- Ph.D
- Level dc:type.qualificationlevel
- doctoral
- Grantor dc:publisher.institution
- University of Leeds
- Year dc:date.issued
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Xu, Zisong
- Advisors dc:contributor.advisor
-
- Dogar, Mehmet
- Wang, He
Rights
- Language dc:language.dcterms_iso639_2
- eng