{"id":{"repo_id":"usf","oai_identifier":"oai:digitalcommons.usf.edu:etd-11605"},"canonical_url":"https://search.ndltd.org/etd/usf/oai:digitalcommons.usf.edu:etd-11605","repository":{"repo_id":"usf","name":"University of South Florida","base_url":"https://digitalcommons.usf.edu/do/oai/"},"display":{"title":"Developing Reinforcement Learning Algorithms for Robots to Aim and Pour Solid Objects","abstract":"Pouring is one of the most commonly executed tasks in a variety of environments. Thereis less attention paid to pouring solid objects and avoiding spillage. Learning the dynamics for pouring solid objects can be a challenge because the collisions and static frictions between objects make their trajectories less predictable than liquid. Nonetheless, pouring solid objects is an important task in real life. In this work, we propose a solution to help robots aim and pour solid objects. The agents will learn how to interact with the environment and identify the optimal pouring trajectories, then manipulate the arm to aim at the target area by moving the end-effector in Inverse Kinematics (IK). The proposed solution is able to pour accurately into the target and minimize spillage. It also has the ability to generalize to unseen objects. Three reinforcement learning algorithms have been implemented, namely TD3, DDPG, and Double-DQN to address the proposed problem. Our study shows that the robot can learn to pour accurately by 1) using a continuous action space, and 2) avoiding overestimation in state-action value approximation.","abstract_html":"Pouring is one of the most commonly executed tasks in a variety of environments. Thereis less attention paid to pouring solid objects and avoiding spillage. Learning the dynamics for pouring solid objects can be a challenge because the collisions and static frictions between objects make their trajectories less predictable than liquid. Nonetheless, pouring solid objects is an important task in real life. In this work, we propose a solution to help robots aim and pour solid objects. The agents will learn how to interact with the environment and identify the optimal pouring trajectories, then manipulate the arm to aim at the target area by moving the end-effector in Inverse Kinematics (IK). The proposed solution is able to pour accurately into the target and minimize spillage. It also has the ability to generalize to unseen objects. Three reinforcement learning algorithms have been implemented, namely TD3, DDPG, and Double-DQN to address the proposed problem. Our study shows that the robot can learn to pour accurately by 1) using a continuous action space, and 2) avoiding overestimation in state-action value approximation.","abstract_has_math":false,"creators":["Li, Haoxuan"],"institution":"University of South Florida","degree_name":"MS in Computer Science (M.S.C.S.)","degree_level":"Thesis","degree_discipline":"Computer Science and Engineering","degree_department":null,"school":null,"contributors":["Yu Sun, Ph.D.","Shaun Canavan, Ph.D.","Yasin Yilmaz, Ph.D."],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-03-10T08:00:00Z","date_published":"2022-03-10T08:00:00Z","updated_at":"2026-11-19T08:48:59Z","subjects":["Continuous Action","Actor-Critic","DQN","Function Approximation","Robot Manipulation","Computer Sciences"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://digitalcommons.usf.edu/etd/10318","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://digitalcommons.usf.edu/etd/10318","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://digitalcommons.usf.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Adigitalcommons.usf.edu%3Aetd-11605","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Yu Sun, Ph.D.","Shaun Canavan, Ph.D.","Yasin Yilmaz, Ph.D."]},{"key":"dc:creator","label":"Author","values":["Li, Haoxuan"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2022-03-10T08:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science and Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis","M.S.C.S."]},{"key":"thesis:degree_name","label":"Degree Name","values":["MS in Computer Science (M.S.C.S.)"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of South Florida"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Continuous Action","Actor-Critic","DQN","Function Approximation","Robot Manipulation","Computer Sciences"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.usf.edu/etd/10318"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Pouring is one of the most commonly executed tasks in a variety of environments. Thereis less attention paid to pouring solid objects and avoiding spillage. Learning the dynamics for pouring solid objects can be a challenge because the collisions and static frictions between objects make their trajectories less predictable than liquid. Nonetheless, pouring solid objects is an important task in real life. In this work, we propose a solution to help robots aim and pour solid objects. The agents will learn how to interact with the environment and identify the optimal pouring trajectories, then manipulate the arm to aim at the target area by moving the end-effector in Inverse Kinematics (IK). The proposed solution is able to pour accurately into the target and minimize spillage. It also has the ability to generalize to unseen objects. Three reinforcement learning algorithms have been implemented, namely TD3, DDPG, and Double-DQN to address the proposed problem. Our study shows that the robot can learn to pour accurately by 1) using a continuous action space, and 2) avoiding overestimation in state-action value approximation."]},{"key":"dc:title","label":"Title","values":["Developing Reinforcement Learning Algorithms for Robots to Aim and Pour Solid Objects"]}]}],"canonical_facts":{"dc:contributor":["Yu Sun, Ph.D.","Shaun Canavan, Ph.D.","Yasin Yilmaz, Ph.D."],"dc:creator":["Li, Haoxuan"],"dc:date":["2022-03-10T08:00:00Z"],"dc:description":["Pouring is one of the most commonly executed tasks in a variety of environments. Thereis less attention paid to pouring solid objects and avoiding spillage. Learning the dynamics for pouring solid objects can be a challenge because the collisions and static frictions between objects make their trajectories less predictable than liquid. Nonetheless, pouring solid objects is an important task in real life. In this work, we propose a solution to help robots aim and pour solid objects. The agents will learn how to interact with the environment and identify the optimal pouring trajectories, then manipulate the arm to aim at the target area by moving the end-effector in Inverse Kinematics (IK). The proposed solution is able to pour accurately into the target and minimize spillage. It also has the ability to generalize to unseen objects. Three reinforcement learning algorithms have been implemented, namely TD3, DDPG, and Double-DQN to address the proposed problem. Our study shows that the robot can learn to pour accurately by 1) using a continuous action space, and 2) avoiding overestimation in state-action value approximation."],"dc:identifier":["https://digitalcommons.usf.edu/etd/10318"],"dc:subject":["Continuous Action","Actor-Critic","DQN","Function Approximation","Robot Manipulation","Computer Sciences"],"dc:title":["Developing Reinforcement Learning Algorithms for Robots to Aim and Pour Solid Objects"],"thesis:degree_discipline":["Computer Science and Engineering"],"thesis:degree_level":["Thesis","M.S.C.S."],"thesis:degree_name":["MS in Computer Science (M.S.C.S.)"],"thesis:institution_name":["University of South Florida"]},"updated_at":"2026-11-19T08:48:59Z"}