Back to search

University of Nevada - Reno

A New Way to Interact With Robots

Abstract

dc:description.abstract

This thesis presents a new and portable robot control architecture for a multimodal user interface for interactive conversations with a robot. Multimodal user interfaces allow the robot to interact with more than one person within a group. Previous research produced architectures that allowed the robot to respond to different people in the conversation and determine which person is talking. However, there has been no architecture that can keep track of a multi-person conversation in the same way that a human can, by establishing and maintaining an identity for each person involved in the conversation. This thesis presents an architecture that attempts to mimic the way humans interact and view a conversation, allowing the robot to carry different conversations with multiple people. The proposed architecture brings the following main contributions. First, it allows for on-line learning of people's faces, which can later be used for appropriately tracking the conversations. Second, it tracks the faces that are recognized in the field of vision to determine who is in the conversation and who has left the conversation. Third, this architecture allows for the robot to respond differently to various people who are in a different stage in the conversation. The robot can give an appropriate response to different people by keeping track of each person and where they are in the conversation. Fourth, in addition to using vision and facial recognition to track and identify users, the architecture will also incorporate sound localization to determine who has spoken. The architecture will also incorporate speech recognition to determine what was spoken. To achieve these goals, the architecture uses finite state machines (FSM) to track interactions between the robot and the person. For each person who joins the conversation, an instance of the FSM is created for that person. The FSM records each person's interaction as well the robot's response to the user's interaction. FSMs also facilitate correct responses from the robot. While existing implementations have been done on specialized robots in the past, this thesis presents a portable architecture that utilizes robots that are already on the market and libraries that are readily available. The advantage of such an implementation is the potential for further development, to an 'off the shelf' implementation that can be adapted for use on different robots. While the implementation in this thesis has not been tested on different robots, it has been successfully emulated on a laptop as well as the NAO robotic platform.

Degree

thesis:*
Level thesis:degree_level
Master's Degree
Department thesis:degree_department
Computer Science and Engineering
Year dc:date.issued
2016

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Duong, Adeline
Advisor dc:contributor.advisor
  • Nicolescu, Monica
Committee members dc:contributor.committeemember
  • Nicolescu, Mircea
  • Shen, Yantao

Subjects

dc:subject × 5

Rights

dc:rights
Statement dc:rights
  • In Copyright(All Rights Reserved)
  • Author(s)

Identifiers

dc:identifier.*
Handle dc:identifier.uri
http://hdl.handle.net/11714/2147

Chain of custody

source
Harvested from
University of Nevada - Reno
Base URL
scholarwolf.unr.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Duong, Adeline. A New Way to Interact With Robots. Master's Degree thesis, 2016. http://hdl.handle.net/11714/2147