Robotic swarms are multi-robot systems whose global behavior emerges from local interactions between individual robots and spatially proximal neighboring robots. Each robot can be programmed with several local control laws that can be activated depending on an operator’s choice of global swarm behavior (e.g. flocking, aggregation, formation control, area coverage). In contrast to other multi-robot systems, robotic swarms are inherently scalable since they are robust to addition and removal of members with minimal system reconfiguration. This makes them ideal for applications such as search and rescue, environmental exploration and surveillance. Practical missions often require a combination of swarm behaviors and may have dynamically changing mission goals. However, a robotic swarm is a complex distributed dynamical system, so its state evolution depends on the timing as well as sequence of the supervisory inputs. Thus, it is difficult to predict the effects of an input on the state evolution of the swarm. More specifically, after becoming aware of a change in mission goals, it is unclear at what time a supervisory operator must convey this information to the swarm or which combination of behaviors to use to accomplish the new goals. The main challenges we address in this thesis are characterizing the effects of input timing on swarm performance and using this theory to inform automated composition of swarm behaviors to accomplish updated mission goals. We begin by formalizing the notion of Neglect Benevolence — the idea that delaying the application of an input can sometimes be beneficial to overall swarm performance — and using the developed theory to demonstrate experimentally that humans can learn to approximate optimal input timing. In an adversarial setting, we also demonstrate that by altering only the timing of consensus updates for a subset of the swarm, we can influence the agreement point of the entire swarm. Given a library of swarm behaviors, automated behavior composition consists of identifying a behavior schedule that must specify (1) the appropriate sequence of behaviors and (2) the corresponding duration of execution for each behavior. Applying our notion of Neglect Benevolence, it is clear these two parts are intricately interdependent. By first assuming the durations are known, we present an algorithm to identify the optimal behavior sequence to achieve a desired swarm mission goal when our library contains general swarm behaviors. By restricting our library to consensus-based swarm behaviors, we then relax the assumption on known durations and present an algorithm to simultaneously find the sequence and durations of swarm behaviors to time-optimally accomplish multiple unordered goals.

Human-swarm interaction studies how human beings can interact with a robotswarm---a large number of robots cooperating with each other without any form of centralizedcontrol. In today's human-swarm interaction literature, the large majority of the works investigatehow human beings can issue commands to and receive feedback from a robot swarm. However, only a few ofthese works study the effect of the interaction with a robot swarm on human psychology (e.g. on thehuman stress or on the human workload). Understanding human psychology in human-swarm interaction isimportant because the human psychological state can have significant impact on the way humansinteract with robot swarms (e.g. a high level of stress can cause a human operator to freeze in themiddle of a critical task, such as a search-and-rescue task). Most existing works that study human psychology in human-swarm interaction conduct their experimentsusing robot swarms simulated on a computer screen. The use of simulation is convenient becauseexperimental conditions can be repeated perfectly in different experimental runs and becauseexperimentation using real robots is expensive both in money and time. However, simulation suffersfrom the so-called reality gap: the inherent discrepancy between simulation and reality. Itis therefore important to study whether this inherent discrepancy can affect humanpsychology---human operators interacting with a simulated robot swarm can react differently thanwhen interacting with a real robot swarm.A large literature in human-robot interaction has studied the psychological impact of theinteraction between human beings and single robots. This literature could in principle be highlyrelevant to human-swarm interaction. However, an inherent difference between human-robot interactionand human-swarm interaction is that in the latter, human operators interact with a large number ofrobots. This large number of robots can affect human psychology---human operators interacting with alarge number of robots can react differently than when interacting with a single robot or with asmall number of robots. It is therefore important to understand whether the large number of robotsthat composes a robot swarm affects human psychology. In fact, if this is the case, it would not bepossible to directly apply the results of human-robot interaction research to human-swarminteraction.We conducted several experiments in order to understand the effect of the reality gap and the effectof the group size (i.e. the number of robots that composes a robot swarm) on the humanpsychological state. In these experiments our participants are exposed to swarms of robots and arepurely passive---they do not issue commands nor receive feedback from the robots. Making theinteraction passive allowed us to study the effects of the reality gap and of the group size on thehuman psychological state without the risk that an interaction interface (such as a joystick)influences the psychological responses of the participants (and thus limiting the visibility of both thereality gap and group size effects). In the reality gap experiments, participants are exposed tosimulated robot swarms displayed either on a computer screen or in a virtual reality environment, and toreal robot swarms. In the group size experiments, participants are exposed to an increasing numberof real robots.In this thesis, we show that the reality gap and the group size affect the human psychological stateby collecting psychophysiological measures (heart rate and skin conductance), self-reported (viaquestionnaires) affective state measures (arousal and valence), self-reported workload (the amountof mental resource needed to carry out a task) and reaction time (the time needed to respond to astimulus). Firstly, we show with our results that our participants' psychophysiological measures,affective state measures, workload and reaction time are significantly higher when they interactwith a real robot swarm compared to when they interact with a robot swarm simulated on a computerscreen, confirming that the reality gap significantly affects the human psychological state.Moreover, we show that it is possible to mitigate the effect of the reality gap using virtualreality---our participants' arousal, workload and reaction time are significantly higher when theyinteract with a simulated robot swarm displayed in a virtual reality environment as opposed to whenit is displayed on a computer screen. Secondly, we show that our participants' psychophysiologicalmeasures and affective state measures increase when the number of robots they are exposed toincreases. Our results have important implications for research in human-swarm interaction. Firstly, for thefirst time, we show that experiments in simulation change the human psychological state compared toexperiments with real robots. Secondly, we show that a characteristic that is inherent to thedefinition of swarm robotics---the large number of robots that composes a robotswarm---significantly affects the human psychological state. Finally, our results show thatpsychophysiological measures, such as heart rate and skin conductance, provide researchers with moreinformation on human psychology than the information provided by using traditional self-reportedmeasures (collected via psychological questionnaires). / Doctorat en Sciences de l'ingénieur et technologie / info:eu-repo/semantics/nonPublished

There are many current and future scenarios that require teams of air, ground or humanoid robots to gather information in complex and often dangerous environments, where it would be unreasonable or impossible for humans to be physically present [1-6]. The current state of the art involves a single robot being monitored by one or many human operators [7], but a single operator managing a team of autonomous robots is preferred as long as effective and time-efficient management of the team is maintained [8-9]. This is limited by the operator's ability to command actions of multiple robots, be aware of robot states, and respond to less important tasks, while accomplishing a primary objective defined by the application scenario. The operator's ability to multi-task could be improved with the use of a multimodal interface, using both visual and haptic feedback. This thesis investigates the use of haptic feedback in developing intuitive, shape-based interaction to maintain heads-up control and increase an operator's situation awareness (SA) while managing a robot team.In this work, the autonomous behavior of the team is modeled after a patrol and cordon scenario, where the team travels to and surrounds buildings of interest. A novel approach that involves treating the team as a moldable volume is presented, where deformations of this volume correspond to changes in team shape. During surround mode, the operator may explore or manipulate the team shape to create custom formations around a building. A spacing interaction method also allows the operator to adjust how robots are spaced within the current shape. Separate haptic feedback is developed for each method to allow the operator to "feel" the shape or spacing manipulation. During travel mode, the operator chooses desired travel locations and receives feedback to help identify how and where the team travels. RoTHSim, an experimental robot team haptic simulator, was developed and used as a test bed for single-operator management of a robot team in a multitasking reconnaissance and surveillance scenario. Using RoTHSim, a human subject experiment was conducted with 19 subjects to determine the effects of haptic feedback and task demand difficulty on levels of performance, SA and workload. Results from the experiment suggest that haptic feedback significantly improves operator performance in a reconnaissance task when task demand is higher, but may slightly increase operator workload. Due to the experimental setup, these results suggest that haptic feedback may make it easier for the operator to experience heads-up control of a team of autonomous robots. There were no significance differences on SA scores due to haptic feedback in this study.

Traditionally robots are controlled using devices like joysticks, keyboards, mice and other similar human computer interface (HCI) devices. Although this approach is effective and practical for some cases, it is restrictive only to healthy individuals without disabilities, and it also requires the user to master the device before its usage. It becomes complicated and non-intuitive when multiple robots need to be controlled simultaneously with these traditional devices, as in the case of Human Swarm Interfaces (HSI). This work presents a novel concept of using human bio-signals to control swarms of robots. With this concept there are two major advantages: Firstly, it gives amputees and people with certain disabilities the ability to control robotic swarms, which has previously not been possible. Secondly, it also gives the user a more intuitive interface to control swarms of robots by using gestures, thoughts, and eye movement. We measure different bio-signals from the human body including Electroencephalography (EEG), Electromyography (EMG), Electrooculography (EOG), using off the shelf products. After minimal signal processing, we then decode the intended control action using machine learning techniques like Hidden Markov Models (HMM) and K-Nearest Neighbors (K-NN). We employ formation controllers based on distance and displacement to control the shape and motion of the robotic swarm. Comparison for ground truth for thoughts and gesture classifications are done, and the resulting pipelines are evaluated with both simulations and hardware experiments with swarms of ground robots and aerial vehicles.

A swarm is a group of uninformed individuals that exhibit collective behaviors. Without any information about the external world, a swarm has limited ability to achieve complex goals. Prior work on human-swarm interaction methods allow a human to influence these uninformed individuals through either leadership or predation as informed agents that directly interact with humans. These methods of influence have two main limitations: (1) although leaders sustain influence over nominal agents for a long period of time, they tend to cause all collective structures to turn in to flocks (negating the benefit of other swarm formations) and (2) predators tend to cause collective structures to fragment. In this thesis, we present the use of mediators as a novel form for human-swarm influence and use mediators to shape the perimeter of a swarm. The mediator method uses special agents that operate from within the spatial center of a swarm. This approach allows a human operator to coordinate multiple mediators to modulate a rotating torus into various shapes while sustaining influence over the swarm, avoiding fragmentation, and maintaining the swarm's connectivity. The use of mediators allows a human to mold and adapt the torus' behavior and structure to a wide range of spatio-temporal tasks such as military protection and decontamination tasks. Results from an experiment that compares previous forms of human influence with mediator-based control indicate that mediator-based control is more amenable to human influence for certain types of problems.

Robot swarms have recently become of interest in both industry and academia for their potential to perform various difficult or dangerous tasks efficiently. As real robot swarms become more of a possibility, many desire swarms to be controlled or directed by a human, which raises questions regarding how that should be done. Part of the challenge of human-swarm interaction is the difficulty of understanding swarm state and how to drive the swarm to produce emergent behaviors. Human input could inhibit desirable swarm behaviors if their input is poor and has sufficient influence over swarm agents, affecting its overall performance. Thus, with too little influence, human input is useless, but with too much, it can be destructive. We suggest that there is some middle level, or interval, of human influence that allows the swarm to take advantage of useful human input while minimizing the effect of destructive input. Further, we propose that human-swarm interaction schemes can be designed to maintain an appropriate level of human influence over the swarm and maintain or improve swarm performance in the presence of both useful and destructive human input. We test this theory by implementing a piece of software to dynamically moderate influence and then testing it with a simulated honey bee colony performing nest site selection, simulated human input, and actual human input via a user study. The results suggest that moderating influence, as suggested, is important for maintaining high performance in the presence of both useful and destructive human input. However, while our software seems to successfully moderate influence with simulated human input, it fails to do so with actual human input.

Swarm robotics is an emerging field that is expected to provide robust solutions to spatially distributed problems. Human operators will often be required to guide a swarm in the fulfillment of a mission. Occasionally, large tasks may require multiple spatial swarms to cooperate in their completion. We hypothesize that when latency, bandwidth, operator dropout, and communication noise are significant factors, human organizations that promote individual initiative perform more effectively and resiliently than hierarchies in the cooperative best-m-of-n task. Simulations automating the behavior of hub-based swarm robotic agents and groups of human operators are used to evaluate this hypothesis. To make the comparisons between the team and hierarchies meaningful, we explore parameter values determining how simulated human operators behave in teams and hierarchies to optimize the performance of the respective organizations. We show that simulation results generally support the hypothesis with respect to the effect of latency and bandwidth on organizational performance.