Back to search

University of South Florida

Functional Object-Oriented Network: A Knowledge Representation for Service Robotics

Abstract

dc:description

In this dissertation, we discuss our work behind the development of the functional object-oriented network (abbreviated as FOON), a graphical knowledge representation for robotic manipulation and understanding of its own actions and (potentially) the intentions of humans in the household. Based on the theory of affordance, this representation captures manipulations and their effects on actions through the coupling of object and motion nodes as fundamental learning units known as functional units. The activities currently represented in FOON are cooking related, but this representation can be extended to other activities that involve manipulation of objects which result in observable changes of state. Typically, a FOON is created after annotating many demonstrations of how tasks are executed from start to finish and merging them all together to form a universal FOON. A robot programmed to use FOON will be equipped with the knowledge needed to solve manipulation problems, given a target goal as a node in FOON; we show how this procedure known as task tree retrieval can be executed by a robot. To circumvent possible physical limitations of the robot in executing manipulations (from the task tree retrieval procedure) successfully for cooking, we demonstrated how human-robot collaboration can also be used to overcome constraints. Complementary to the universal FOON creation procedure, we also investigated other means of learning concepts through semantic similarity as a solution to learning without the annotation of new demonstration videos. In addition to the retrieval algorithm, we also proposed motion embedding for representation of motions based on mechanical characteristics of said motions. Through this proposed representation, known as the motion taxonomy, we can solve the problem of ambiguity, which is inherent to human language when defining labels for motions or manipulations seen in demonstrations, by representing motions in a binary machine language.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy (Ph.D.)
Level thesis:degree_level
Dissertation
Discipline thesis:degree_discipline
Computer Science and Engineering
Grantor
University of South Florida
Year dc:date
2020

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Paulius Ramos, David Andrés
Contributors dc:contributor
  • Yu Sun, Ph.D.
  • Changhyun Kwon, Ph.D.
  • Xiaoning Qian, Ph.D.
  • Paul Rosen, Ph.D.
  • Sudeep Sarkar, Ph.D.
  • Yicheng Tu, Ph.D.

Subjects

dc:subject × 6

Identifiers

dc:identifier.*
Repository record dc:identifier
https://digitalcommons.usf.edu/etd/8276

Chain of custody

source
Harvested from
University of South Florida
Base URL
digitalcommons.usf.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Paulius Ramos, David Andrés. Functional Object-Oriented Network: A Knowledge Representation for Service Robotics. Dissertation thesis, University of South Florida, 2020. https://digitalcommons.usf.edu/etd/8276