Back to search

University of South Florida

Developing Reinforcement Learning Algorithms for Robots to Aim and Pour Solid Objects

Abstract

dc:description

Pouring is one of the most commonly executed tasks in a variety of environments. Thereis less attention paid to pouring solid objects and avoiding spillage. Learning the dynamics for pouring solid objects can be a challenge because the collisions and static frictions between objects make their trajectories less predictable than liquid. Nonetheless, pouring solid objects is an important task in real life. In this work, we propose a solution to help robots aim and pour solid objects. The agents will learn how to interact with the environment and identify the optimal pouring trajectories, then manipulate the arm to aim at the target area by moving the end-effector in Inverse Kinematics (IK). The proposed solution is able to pour accurately into the target and minimize spillage. It also has the ability to generalize to unseen objects. Three reinforcement learning algorithms have been implemented, namely TD3, DDPG, and Double-DQN to address the proposed problem. Our study shows that the robot can learn to pour accurately by 1) using a continuous action space, and 2) avoiding overestimation in state-action value approximation.

Degree

thesis:*
Name thesis:degree_name
MS in Computer Science (M.S.C.S.)
Level thesis:degree_level
Thesis
Discipline thesis:degree_discipline
Computer Science and Engineering
Grantor
University of South Florida
Year dc:date
2022

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Li, Haoxuan
Contributors dc:contributor
  • Yu Sun, Ph.D.
  • Shaun Canavan, Ph.D.
  • Yasin Yilmaz, Ph.D.

Subjects

dc:subject × 6

Identifiers

dc:identifier.*
Repository record dc:identifier
https://digitalcommons.usf.edu/etd/10318

Chain of custody

source
Harvested from
University of South Florida
Base URL
digitalcommons.usf.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Li, Haoxuan. Developing Reinforcement Learning Algorithms for Robots to Aim and Pour Solid Objects. Thesis thesis, University of South Florida, 2022. https://digitalcommons.usf.edu/etd/10318