Designing an effective human-robot interaction paradigm is particularly important for complex tasks such as multi-robot manipulation that require the human and robot to work together in a tightly coupled fashion. Although increasing the number of robots can expand the area that the robots can cover within a bounded period of time, a poor human-robot interface will ultimately compromise the performance of the team of robots. However, introducing a human operator to the team of robots, does not automatically improve performance due to the difficulty of teleoperating mobile robots with manipulators. The human operator’s concentration is divided not only among multiple robots but also between controlling each robot’s base and arm. This complexity substantially increases the potential neglect time, since the operator’s inability to effectively attend to each robot during a critical phase of the task leads to a significant degradation in task performance. There are several proven paradigms for increasing the efficacy of human-robot interaction: 1) multimodal interfaces in which the user controls the robots using voice and gesture; 2) configurable interfaces which allow the user to create new commands by demonstrating them; 3) adaptive interfaces which reduce the operator’s workload as necessary through increasing robot autonomy. This dissertation presents an evaluation of the relative benefits of different types of user interfaces for multi-robot systems composed of robots with wheeled bases and three degree of freedom arms. It describes a design for constructing low-cost multi-robot manipulation systems from off the shelf parts. User expertise was measured along three axes (navigation, manipulation, and coordination), and participants who performed above threshold on two out of three dimensions on a calibration task were rated as expert. Our experiments reveal that the relative expertise of the user was the key determinant of the best performing interface paradigm for that user, indicating that good user modiii eling is essential for designing a human-robot interaction system that will be used for an extended period of time. The contributions of the dissertation include: 1) a model for detecting operator distraction from robot motion trajectories; 2) adjustable autonomy paradigms for reducing operator workload; 3) a method for creating coordinated multi-robot behaviors from demonstrations with a single robot; 4) a user modeling approach for identifying expert-novice differences from short teleoperation traces.

Thesis (MScEng)--Stellenbosch University, 2011. / ENGLISH ABSTRACT: RoboCup promotes research in robotics and multi-robot systems (MRS). The RoboCup Small Size League (SSL), in particular, offers an entry level opportunity to take part in this field of study. This thesis presents a starting phase for research in robotics and MRS at Stellenbosch University. It includes the full documentation of the mechanical, electronic and software design of an omni-directional soccer robot for RoboCup SSL. The robot is also meant to operate as a hardware and software development platform for research in MRS. The platform was therefore designed with high-level programming language compatibility, a wide range of connectivity, and modularity in mind. The robot uses a single board computer (SBC) running a Linux operating system to accomplish these objectives. Moreover, a driver class library was written in C++ as a software application interface (API) for future development on the robot platform. The robot was also developed with a particular focus on a distributed control architecture. "Player" was implemented as the middleware, which can be used for communication between multiple robots in a distributed environment. Additionally, three tests were performed to demonstrate the functionality of the prototype: a PI speed control test, a direction accuracy test and a static communication test using the middleware. Recommendations for possible future work are also given. / AFRIKAANSE OPSOMMING: RoboCup bevorder navorsing in robotika en multi-robot-stelsels (MRS). Die RoboCup Klein Liga (KL) bied in die besonder die geleentheid om op intreevlak navorsing te doen in hierdie veld. Hierdie tesis verteenwoordig die eerste fase van navorsing in robotika en MRS by Stellenbosch Universiteit. Dit sluit die volledige dokumentasie van die meganiese, elektroniese en sagteware-ontwerp van ’n omnidireksionele sokker-robot vir die KL in. Die robot is ook veronderstel om te dien as ’n hardeware- en sagteware-ontwikkelingsplatform vir navorsing in MRS. Die platform is dus ontwerp met ’n verskeidenheid van uitbreingsmoontlikhede en modulariteit in gedagte asook die moontlikheid om gebruik te maak van ’n hoë-vlak programmeertaal. Om hierdie doelwitte te bereik, maak die robot gebruik van ’n enkel-bord-rekenaar met ’n Linux bedryfstelsel. Verder was ’n sagteware drywer in C++ geskryf om te dien as ’n sagteware-koppelvlak vir toekomstige ontwikkeling op die robot platform. Die robot is ook ontwikkel met die besondere fokus op ’n gedesentraliseerde beheerstels. Player was geïmplementeer as die middelware, wat gebruik kan word vir kommunikasie tussen verskeie robotte in ’n gedesentralliseerde beheerstelsel. Daar is drie toetse uitgevoer om die funksionaliteit van die prototipe te demonstreer, ’n PI spoed beheer toets, ’n rigting akkuraatheidstoets en ’n statiese kommunikasie toets deur van die middelware gebruik te maak. Aanbevelings vir moontlike toekomstige werk word ook verskaf.

The objective of this research is to develop a framework for multi-robot coordination and control with emphasis on human-swarm and inter-agent interactions. We focus on two problems: in the first we address how to enable a single human operator to externally influence large teams of robots. By directly imposing density functions on the environment, the user is able to abstract away the size of the swarm and manipulate it as a whole, e.g., to achieve specified geometric configurations, or to maneuver it around. In order to pursue this approach, contributions are made to the problem of coverage of time-varying density functions. In the second problem, we address the characterization of inter-agent interactions and enforcement of desired interaction patterns in a provably safe (i.e., collision free) manner, e.g., for achieving rich motion patterns in a shared space, or for mixing of sensor information. We use elements of the braid group, which allows us to symbolically characterize classes of interaction patterns. We further construct a new specification language that allows us to provide rich, temporally-layered specifications to the multi-robot mixing framework, and present algorithms that significantly reduce the search space of specification-satisfying symbols with exactness guarantees. We also synthesize provably safe controllers that generate and track trajectories to satisfy these symbolic inputs. These controllers allow us to find bounds on the amount of safe interactions that can be achieved in a given bounded domain.

Advances in mobile robot technology allow an increasing variety of applications to be imagined, including: search and rescue, exploration of unknown areas and working with hazardous materials. State of the art robots are able to behave autonomously and without direct human control, using on-board devices to perceive, navigate and reason about the world. Unmanned Aerial Vehicles (UAVs) are particularly well suited to performing advanced sensing tasks by moving rapidly through the environment irrespective of the terrain. Deploying groups of mobile robots offers advantages, such as robustness to individual failures and a reduction in task completion time. However, to operate efficiently these teams require specific approaches to enable the individual agents to cooperate. This thesis proposes coordinated approaches to search scenarios for teams of UAVs. The primary application considered is Wilderness Search and Rescue (WiSaR), although the techniques developed are applicable elsewhere. A novel frontier-based search approach is developed for rotor-craft UAVs, taking advantage of available terrain information to minimise altitude changes during flight. This is accompanied by a lightweight coordination mechanism to enable cooperative behaviour with minimal additional overhead. The concept of a team rendezvous is introduced, at which all team members attend to exchange data. This also provides an ideal opportunity to create a comprehensive team solution to relay newly gathered data to a base station. Furthermore, the delay between sensing and the acquired data becoming available to mission commanders is analysed and a technique proposed for adapting the team to meet a latency requirement. These approaches are evaluated and characterised experimentally through simulation. Coordinated frontier search is shown to outperform greedy walk methods, reducing redundant sensing coverage using only a minimal coordination protocol. Combining the search, rendezvous and relay techniques provides a holistic approach to the deployment of UAV teams, meeting mission objectives without extensive pre-configuration.

Autonomy in multi-robot systems is bounded by coordination among its agents. Coordination implies simultaneous task decomposition, task allocation, team formation, task scheduling and routing; collectively termed as task planning. In many real-world applications of multi-robot systems such as commercial cleaning, delivery systems, warehousing and inventory management: spatial & temporal constraints, variable execution time, and energy limitations need to be integrated into the planning module. Spatial constraints comprise of the location of the tasks, their reachability, and the structure of the environment; temporal constraints express task completion deadlines. There has been significant research in multi-robot task allocation involving spatio-temporal constraints. However, limited attention has been paid to combine them with team formation and non- instantaneous task execution time. We achieve team formation by including quota constraints which ensure to schedule the number of robots required to perform the task. We introduce and integrate task activation (time) windows with the team effort of multiple robots in performing tasks for a given duration. Additionally, while visiting tasks in space, energy budget affects the robots operation time. We map energy depletion as a function of time to ensure long-term operation by periodically visiting recharging stations. Research on task planning approaches which combines all these conditions is still lacking. In this thesis, we propose two variants of Team Orienteering Problem with task activation windows and limited energy budget to formulate the simultaneous task allocation and scheduling as an optimization problem. A complete mixed integer linear programming (MILP) formulation for both variants is presented in this work, implemented using Gurobi Optimizer and analyzed for scalability. This work compares the different objectives of the formulation like maximizing the number of tasks visited, minimizing the total distance travelled, and/or maximizing the reward, to suit various applications. Finally, analysis of optimal solutions discover trends in task selection based on the travel cost, task completion rewards, robot's energy level, and the time left to task inactivation.

Cette thèse s'inscrit dans le cadre du projet Cart-O-Matic mis en place pour participer au défi CAROTTE (CArtographie par ROboT d'un TErritoire) organisé par l'ANR et la DGA. Le but de ce défi est de construire une carte en deux et trois dimensions et de localiser des objets dans un environnement inconnu statique de type appartement. Dans ce contexte, l'utilisation de plusieurs robots est avantageuse car elle permet d'augmenter l'efficacité en temps de la couverture. Cependant, comme nous le montrons, le gain est conditionné par le niveau de coopération entre les robots. Nous proposons une stratégie de coopération pour une cartographie multirobot efficace. Une difficulté est la construction d'une carte commune, nécessaire, afin que chaque robot puisse connaître les zones de l'environnement encore inexplorées. Pour obtenir une bonne coopération avec un algorithme simple nous proposons une technique de déploiement fondée sur le choix d'une cible par chaque robot. L'algorithme proposé cherche à distribuer les robots vers différentes directions. Il est fondé sur le calcul partiel de champs de potentiels permettant à chaque robot de calculer efficacement son prochain objectif. En complément de ces contributions théoriques, nous décrivons le système robotique complet mis en oeuvre au sein de l'équipe Cart-O-Matic ayant permis de remporter la dernière édition du défi CAROTTE / This thesis is part of Cart-O-Matic project set up to participate in the challenge CARROTE (mapping of a territory) organized by the ANR and the DGA. The purpose of this challenge is to build 2D and 3D maps of a static unknown 'apartment-like' environment. In this context, the use of several robots is advantageous because it increases the time efficiency to discover fully the environment. However, as we show, the gain is determined by the level of cooperation between robots. We propose a cooperation strategy for efficient multirobot mapping. A difficulty is the construction of a common map, necessary so that each robot can know the areas of the environment which remain unexplored.For a good cooperation with a simple algorithm we propose a deployment technique based on the choice of a target by each robot. The proposed algorithm tries to distribute the robots in different directions. It is based on calculation of the partial potential fields allowing each robot to compute efficiently its next target. In addition to these theoretical contributions, we describe the complete robotic system implemented in the Cart-O-Matic team that helped win the last edition of the CARROTE challenge

Afin de parvenir à l'autonomie des robots mobiles, la localisation efficace est une condition préalable nécessaire. Le suivi de position, la localisation globale et le problème du robot kidnappé sont les trois sous-problèmes que nous étudions. Dans cette thèse, nous comparons en simulation trois algorithmes de localisation Markovienne. Nous proposons ensuite une amélioration de l'algorithme de localisation de Monte Carlo par filtre particulaire. Cet algorithme (nommé SAMCL) utilise des particules auto-adaptatives. En employant une technique de pré-mise en cache pour réduire le temps de calcul en ligne, l'algorithme SAMCL est plus efficace que la méthode de Monte Carlo usuelle. En outre, nous définissons la notion de région d'énergie similaire (SER), qui est un ensemble de poses (cellules de la grille) dont l'énergie-capteur est similaire avec celle du robot dans l'espace réel. En distribuant les échantillons globaux dans SER lieu de les distribuer au hasard dans la carte, SAMCL obtient une meilleure performance dans la localisation et résout ces trois sous-problèmes. La localisation coopérative de plusieurs robots est également étudiée. Nous avons développé un algorithme (nommé PM) pour intégrer l'information de localisation échangée par les robots lors d'une rencontre au cours d'une mission commune. Cet algorithme apparaît comme une extension à l'algorithme de SAMCL et a été validé en simulation. La validité et l'efficacité de notre approche sont démontrées par des expériences sur un robot réel évoluant dans un environnement connu et préalablement cartographié. / In order to achieve the autonomy of mobile robots, effective localization is a necessary prerequisite. In this thesis, we study and compare three regular Markov localization algorithms by simulations. Then we propose an improved Monte Carlo localization algorithm using self-adaptive samples, abbreviated as SAMCL. By employing a pre-caching technique to reduce the on-line computational burden, SAMCL is more efficient than regular MCL. Further, we define the concept of similar energy region (SER), which is a set of poses (grid cells) having similar energy with the robot in the robot space. By distributing global samples in SER instead of distributing randomly in the map, SAMCL obtains a better performance in localization. Position tracking, global localization and the kidnapped robot problem are the three sub-problems of the localization problem. Most localization approaches focus on solving one of these sub-problems. However, SAMCL solves all the three sub-problems together thanks to self-adaptive samples that can automatically separate themselves into a global sample set and a local sample set according to needs. Cooperative localization among multiple robots is carried out by exchanging localization information derived from cooperation. We devise the Position Mapping (PM) algorithm to integrate this information, which can merge into the SAMCL algorithm as an extension. The validity and the efficiency of our algorithms are demonstrated by experiments carried out with a real robot in a structured and known environment.

Le problème d’exploration est un sujet fondamental de la robotique mobileautonome qui traite la réalisation de la catographie complète (mapping) d’un environnementprécédemment inconnu. Il y a plusieurs scénarios où l’achèvement de l’exploration d’unezone est une composante principale de la mission à accomplir. Par exemple: l’explorationplanétaire, la reconnaissance, la recherche et le sauvetage, l’agriculture, le nettoyage des lieuxdangereux, comme champs de mines et des zones radioactives. D’autre part, la communicationsans fil joue un rôle important dans les stratégies multi-robot collaboratives. Malheureusement,la supposition ou l’exigence de communication stable, ou encore, la connectivité continue,peuvent être compromises dans des scénarios réels. Dans cette thèse, deux nouvellesapproches abordent le problème d’exploration multi-robot d’environnements, en considérantune communication restreinte. D’abord, une stratégie multi-objectif auto-adaptative est proposéepour diriger la sélection de tâches en tennant compte de la performance d’explorationet du niveau de connectivité. Deuxièmement, deux rôles – l’explorateur et le relais de communication– sont considérés pour améliorer la stratégie de sélection de tâche précédente. Basésur le modèle de communication, une nouvelle approche de placement de robot relais pour desmissions d’exploration multi-robot est présentée en détail. Comparé avec d’autres approchesde l’état de l’art, les deux approches proposées dans cette thèse sont capables de diminuer ladurée de périodes de déconnexion sans dégradation considérable sur temps d’exploration. / The exploration problem is a fundamental subject in autonomous mobilerobotics that deals with achieving the complete coverage of a previously unknown environment.There are several scenarios where completing exploration of a zone is a main part of themission, e.g. planetary exploration, reconnaissance, search and rescue, agriculture, cleaning,or dangerous places as mined lands and radioactive zones. Wireless communication plays animportant role in collaborative multi-robot strategies. Unfortunately, the assumption or requirementof stable communication and end-to-end connectivity may be easily compromised inreal scenarios. In this thesis, two novel approaches to tackle the problem of multi-robot explorationof communication constrained environments are proposed. At first, an auto-adaptivemulti-objective strategy is followed in order to support the selection of tasks regarding bothexploration performance and connectivity level. Secondly, two roles –explorer and communicationrelay– are considered in order to improve the benefits of the previous task selectionstrategy. Based on the communication model, a novel polynomial-time relay placement approachfor multi-robot exploration missions is introduced in detail. Compared with others,the proposed approaches are capable of decreasing the last of disconnection periods withouta noticeable degradation of the completion exploration time.

We present automatic tools for configuring and deploying multi-robot networks of decentralized, mobile robots. These methods are tailored to the decentralized nature of the multi-robot network and the limited information available to each robot. We present methods for determining if user-defined network tasks are feasible or infeasible for the network, considering the limited range of its sensors. To this end, we define rigid and persistent feasibility and present necessary and sufficient conditions (along with corresponding algorithms) for determining the feasibility of arbitrary, user-defined deployments. Control laws for moving multi-robot networks in acyclic, persistent formations are defined. We also present novel Embedded Graph Grammar Systems (EGGs) for coordinating and deploying the network. These methods exploit graph representations of the network, as well as graph-based rules that dictate how robots coordinate their control. Automatic systems are defined that allow the robots to assemble arbitrary, user-defined formations without any reliance on localization. Further, this system is augmented to deploy these formations at the user-defined, global location in the environment, despite limited localization of the network. The culmination of this research is an intuitive software program with a Graphical User Interface (GUI) and a satellite image map which allows users to enter the desired locations of sensors. The automatic tools presented here automatically configure an actual multi-robot network to deploy and execute user-defined network tasks.

Depuis la fin des années 1980, le développement de véhicules autonomes, capables de rouler sans l'intervention d'un être humain, est un champ de recherche très actif dans la plupart des grands pays industrialisés. La diminution du nombre d'accidents, des temps de trajet plus courts, une meilleure efficacité énergétique et des besoins en infrastructure plus limités, sont autant d'effets socio-économiques espérés de leur déploiement. Des formes de coopération inter-véhicules et entre les véhicules et l'infrastructure routière sont nécessaires au fonctionnement sûr et efficace du système de transport dans sa globalité. Cette thèse s'intéresse à une forme de coopération particulière en étudiant la coordination de robots mobiles aux intersections. La majorité des systèmes de coordination existants planifie une trajectoire que les robots doivent exécuter afin d'assurer l'absence de collision. C'est une approche classique de la planification, qui est alors considérée comme un mécanisme de génération de l'action. Dans cette thèse, seules les priorités entre les véhicules sont planifiées, c'est-à-dire l'ordre relatif de passage des véhicules dans l'intersection, ce qui est bien plus faible car un grand nombre de trajectoires respectent les même priorités. Les priorités sont alors simplement utilisées comme une ressource de coordination pour guider les robots dans l'intersection. Une fois les priorités affectées, les robots suivent une loi de contrôle qui s'assure qu'elles soient bien respectées. Il en découle un système de coordination robuste, capable de gérer toute une classe d'événements imprévisibles de façon réactive, ce qui est particulièrement adapté pour une application à la coordination de véhicules autonomes aux intersections où voitures, transports en commun et piétons partagent la route. / Since the end of the 1980's, the development of self-driven autonomous vehicles is an intensive research area in most major industrial countries. Positive socio-economic potential impacts include a decrease of crashes, a reduction of travel times, energy efficiency improvements, and a reduced need of costly physical infrastructure. Some form of vehicle-to-vehicle and/or vehicle-to-infrastructure cooperation is required to ensure a safe and efficient global transportation system. This thesis deals with a particular form of cooperation by studying the problem of coordinating multiple mobile robots at an intersection area. Most of coordination systems proposed in previous work consist of planning a trajectory and to control the robots along the planned trajectory: that is the plan-as-program paradigm where planning is considered as a generative mechanism of action. The approach of the thesis is to plan priorities – the relative order of robots to go through the intersection – which is much weaker as many trajectories respect the same priorities. Then, priorities are merely used as a coordination resource to guide robots through the intersection. Once priorities are assigned, robots are controlled through a control law preserving the assigned priorities. It results in a more robust coordination system – able to handle a large class of unexpected events in a reactive manner – particularly well adapted for an application to the coordination of autonomous vehicles at intersections where cars, public transport and pedestrians share the road.