Les systèmes de contrôle des événements discrets ont la possibilité de résoudre les défis importants de la société moderne. En particulier, cela représente une solution fondamentale pour gérer et contrôler les nouvelles avancées technologiques en conformité avec la requis du développement durable. Le paramétrage, la configuration et la prise de décision de ces systèmes de contrôle sont des aspects critiques qui influent sur les performances et la productivité. Les approches d'architecture de contrôle dynamique, telles que les systèmes de contrôle reconfigurables, ont été proposées pour la modélisation de ces systèmes. Cependant, ils n'ont pas réussi à optimiser le processus de reconfiguration car celles-ci se concentrent sur la continuité de l'exécution plutôt que sur l'optimisation de la reconfiguration. Cette dissertation propose une architecture de référence pour un système de contrôle reconfigurable, nommé Pollux, conçu pour gérer et ajuster de manière optimale et en temps réel l'architecture d'un système de contrôle, soit pour guider l'exécution opérationnelle ou répondre à une perturbation du système. En considérant une proposition d'une configuration optimale des architectures de contrôle basées sur la gouvernance partagée, cette approche proposée un système de contrôle reconfigurable compose d’une entité décisionnelle flexible et personnalisable, d’une représentation qui caractérise la configuration unique et la solution de contrôle de l'architecture de contrôle et d’un mécanisme de reconfiguration à trois modules qui intègre les principes basés sur l'optimalité dans la reconfiguration. Notre approche est appliquée dans le domaine de la fabrication et est validée dans une simulation et une cellule réelle de fabrication située à l'Université de Valenciennes, en France. La validation effectuée dans trois scénarios expérimentaux a permis de vérifier les avantages de notre approche et de nous encourager à continuer la recherche. / Discrete-event control systems have the opportunity to resolve significant challenges of modern society. In particular, these represent a fundamental solution to manage and control the new technological advances in compliance to the increased consciousness of sustainable development. The parameterization, configuration and decision-making of these control systems are critical aspects that impact the performance and productivity required. Dynamic control architecture approaches, such as reconfigurable control systems, have been proposed for modelling such systems. However, such approaches have failed to address the recovery of the reconfiguration process as these focus on the continuity of execution rather than on the optimisation of the reconfiguration. This dissertation proposes a reference architecture for a reconfigurable control system, named Pollux, designed to manage and adjust optimally and in real time the architecture of a control system, either to guide operational execution or to respond to a system perturbation. Considering a proposed framework of an optimal configuration of control architectures based on shared governance, this proposed approach aims to orchestrate a flexible and customizable decisional entity, a representation that characterize the unique configuration and control solution of the control architecture, and a three-module reconfiguration mechanism that integrates the optimality-based principles into the reconfiguration process, to ensure a recovery of global performance and/or minimise the degradation caused by perturbations. Our approach is applied in the manufacturing domain and is validated in a simulation and a real flexible manufacturing system cell located at the University of Valenciennes, France. The validation conducted in three experimental scenarios verified the benefits of our approach and encourage us to continue research in this direction.

<div>Multi-robot systems have demonstrated strong capability in handling environmental operations. In this study, We examine how a team of robots can be utilized in covering and removing spill patches in a dynamic environment by executing three consecutive stages: deployment, coverage, and rendezvous. </div><div> </div><div>For the deployment problem, we aim for robot allocation based on the discreteness of the patches that need to be covered. With the deep neural network (DNN) based spill detector and remote sensing facilities such as drones with vision sensors and satellites, we are able to obtain the spill distribution in the workspace. Then, we formulate the allocation problem in a general optimization form and provide solutions using an integer linear programming (ILP) solver under several realistic constraints. After the allocation process is completed and the robot team is divided according to the number of spills, we deploy robots to their computed optimal goal positions. In the robot deployment part, control laws based on artificial potential field (APF) method are proposed and practiced on robots with a common unicycle model. </div><div> </div><div>For the coverage control problem, we show two strategies that are tailored for a wirelessly networked robot team. We propose strategies for coverage with and without path planning, depending on the availability of global information. Specifically, in terms of coverage with path planning, we partition the workspace from the aerial image into pieces and let each robot take care of one of the pieces. However, path-planning-based coverage relies on GPS signals or other external positioning systems, which are not applicable for indoor or GPS-denied circumstances. Therefore, we propose an asymptotic boundary shrink control that enables a collective coverage operation with the robot team. Such a strategy does not require a planned path, and because of its distributedness, it shows many advantages, including system scalability, dynamic spill adaptability, and collision avoidance. In case of a large-scale patch that poses challenges to robot connectivity maintenance during the operation, we propose a pivot-robot coverage strategy by mean of an a priori geometric tessellation (GT). In the pivot-robot-based coverage strategy, a team of robots is sent to perform complete coverage to every packing area of GT in sequence. Ultimately, the entire spill in the workspace can be covered and removed.</div><div> </div><div>For the rendezvous problem, we investigate the use of graph theory and propose control strategies based on network topology to motivate robots to meet at a designated or the optimal location. The rendezvous control strategies show a strong robustness to some common failures, such as mobility failure and communication failure. To expedite the rendezvous process and enable herding control in a distributed way, we propose a multi-robot multi-point rendezvous control strategy. </div><div> </div><div>To verify the validity of the proposed strategies, we carry out simulations in the Robotarium MATLAB platform, which is an open source swarm robotics experiment testbed, and conduct real experiments involving multiple mobile robots.</div>

Le service des urgences est un important service de soins qui représente le goulot d'étranglement de l'hôpital. Les urgences sont souvent confrontées à des problèmes de tension dans de nombreux pays à travers le monde. L'une des causes de la tension dans les urgences est l'interférence permanente entre trois types de patients : les patients déjà programmés, les patients non programmés et les patients non programmés urgents. Le but de cette thèse est de contribuer à l'étude et au développement d'un système d’aide à la décision pour améliorer la prise en charge des patients aussi bien en mode de fonctionnement normal qu’en mode tension. Deux principaux processus ont été développé. Un processus d’ordonnancement à horizon glissant en utilisant un algorithme mimétique avec l’intégration des opérateurs génétiques contrôlés pour déterminer un calendrier optimal de passage des patients. Le deuxième processus d’orchestration dynamique, à base d’agents communicants, tient compte de la nature dynamique et incertaine de l'environnement des urgences en actualisant continuellement ce calendrier. Cette orchestration pilote en temps réel le workflow du parcours patient, améliore pas à pas les indicateurs de performance durant l'exécution. Grâce aux comportements des agents et aux protocoles de communication, le système proposé a établi un lien direct en temps réel entre les performances requises sur le terrain et les actions afin de diminuer l'impact de la tension. Les résultats expérimentaux, mis en œuvre au CHRU de Lille, indiquent que l’application de nos approches permet d’améliorer les indicateurs de performance grâce aux pilotage par les agents du workflow en cours exécution. / The emergency department is an important care service that represents the hospital's bottleneck. Emergencies often face overcrowding problems in many countries worldwide. One of the causes of the emergency department overcrowding is the permanent interference between three types of arriving patients: already programmed patients, non-programmed patients and urgent non-programmed patients. The aim of this thesis is to contribute to the study and development a decision support system to improve patient management in both normal and overcrowding situation. Two main processes have been developed. A rolling-horizon scheduling process using a memetic algorithm with the integration of controlled genetic operators to determine an optimal schedule for patient. The second dynamic orchestration process, based on communicating agents, takes into account the dynamic and uncertain nature of the emergency environment by continually updating this schedule for patient. This orchestration monitoring in real time the workflow of the patient pathway improves step by step the performance indicators during the execution. Through agent behaviors and communication protocols, the proposed system has established a direct real-time link between the required performances and the effective actions in order to decrease the overcrowding impact. The experimental results in this thesis, implemented at the Regional University Hospital Center (RUHC) of Lille, justify the interest of the application of our approaches to improve the performance indicators thanks to the agents driven patient pathway workflows during their execution.

Distributed coordination of multi-agent systems (MASs) has been widely studied in various emerging engineering applications, including connected vehicles, wireless networks, smart grids, and cyber-physical systems. In these contexts, agents make the decision locally, relying on the interaction with their immediate neighbors over the connected communication networks. The study of distributed coordination for the multi-agent system (MAS) with constraints is significant yet challenging, especially in terms of ubiquitous uncertainties, the heavy communication burden, and communication delays, to name a few. Hence, it is desirable to develop distributed algorithms for the constrained MAS with these practical issues. In this dissertation, we develop the theoretical results on robust distributed model predictive control (DMPC) algorithms for two types of control problems (i.e., formation stabilization problem and consensus problem) of the constrained and uncertain MAS and apply robust DMPC algorithms in applications of cooperative marine vehicles. More precisely, Chapter 1 provides a systematic literature review, where the state-of-the-art DMPC for formation stabilization and consensus, robust MPC, and MPC for motion control of marine vehicles are introduced. Chapter 2 introduces some notations, necessary definitions, and some preliminaries. In Chapter 3, we study the formation stabilization problem of the nonlinear constrained MAS with un- certainties and bounded time-varying communication delays. We develop a min-max DMPC algorithm with the self-triggered mechanism, which significantly reduces the communication burden while ensuring closed-loop stability and robustness. Chapter 4 investigates the consensus problem of the general linear MAS with input constraints and bounded time-varying delays. We design a robust DMPC-based consensus protocol that integrates a predesigned consensus protocol with online DMPC optimization techniques. Under mild technical assumptions, the estimation errors propagated over prediction due to delay-induced inaccurate neighboring information are proved bounded, based on which a robust DMPC strategy is deliberately designed to achieve robust consensus while satisfying control input constraints. Chapter 5 proposes a Lyapunov-based DMPC approach for the formation tracking control problem of co-operative autonomous underwater vehicles (AUVs) subject to environmental disturbances. A stability constraint leveraging the extended state observer-based auxiliary control law and the associated Lyapunov function is incorporated into the optimization problem to enforce the stability and enhance formation tracking performance. A collision-avoidance cost is designed and employed in the DMPC optimization problem to further guarantee the safety of AUVs. Chapter 6 presents a tube-based DMPC approach for the platoon control problem of a group of heterogeneous autonomous surface vehicles (ASVs) with input constraints and disturbances. In particular, a coupled inter-vehicle safety constraint is added to the DMPC optimization problem; it ensures that neighboring ASVs maintain the safe distance and avoid inter-vehicle collision. Finally, we summarize the main results of this dissertation and discuss some potential directions for future research in Chapter 7. / Graduate / 2023-04-19

Tesis por compendio / [ES] Los seres humanos somos por naturaleza seres afectivos, las emociones, el estado de ánimo, nuestra personalidad, o nuestras relaciones con los demás guían nuestras motivaciones y nuestras decisiones. Una de las principales habilidades cognitivas relacionadas con el afecto es la empatía. La empatía es un constructo psicológico cuya definición ha ido evolucionando a lo largo de los años y cuyo significado hace referencia a un amplio abanico de competencias afectivas y cognitivas que son fundamentales en el desarrollo del ser humano como ser social. El uso de la empatía en el ámbito de la inteligencia artificial puede revolucionar la forma en la que interactuamos con las máquinas así como la forma en la que simulamos el comportamiento humano. Por otro lado, hay que tener en cuenta que los seres humanos habitualmente acudimos al uso de distintas palabras como ``triste'' o ``contento'' para expresar o verbalizar el estado afectivo. Sin embargo, estas palabras son simplificaciones que abarcan un amplio espectro de procesos cognitivos y estados mentales. Además, hay que considerar que estas palabras tienen una alta dependencia del idioma y la cultura en la que se utilizan. Por tanto, los modelos de representación computacional de los estados afectivos deben se capaces de adaptarse a distintos entornos culturales y de permitir que un agente exprese o represente, mediante palabras, un determinado estado afectivo. En esta tesis se propone un nuevo modelo de agente empático capaz de adaptar su comportamiento a distintos entornos culturales. Para ello, en primer lugar, se presenta una nueva metodología basada en la experimentación para adaptar un espacio de representación de emociones basado en las dimensiones del placer y la activación para la simulación y el reconocimiento computacional afectivo a diferentes entornos culturales. Los resultados del experimento realizado con hispanohablantes europeos se utilizan para proporcionar un nuevo modelo basado en la lógica difusa para representar estados afectivos en las dimensiones de placer y activación utilizando un enfoque de coordenadas polares. Para demostrar que las diferencias culturales afectan a los valores de placer y activación asociados a cada emoción, el experimento se repitió con participantes portugueses y suecos. En segundo lugar, se presenta un nuevo modelo de elicitación de emociones en agentes afectivos que utiliza lógica difusa. Las emociones generadas en el agente por las reglas de valoración difusa se expresan en el modelo de representación del afecto resultante de los experimentos previamente descritos. Además, se propone un nuevo proceso de regulación del afecto que adapta el estado de ánimo del agente, representado mediante un vector en el espacio placer-activación, cada vez que una emoción es elicitada. En tercer lugar, se propone una formalización de la sintaxis, la semántica y el ciclo de razonamiento de AgentSpeak para permitir el desarrollo de agentes afectivos con capacidades empáticas. Partiendo de las teorías de valoración empática y regulación empática, se modifica la estructura de razonamiento del agente para permitir que la empatía afecte al proceso de toma de decisiones. Finalmente, se presenta un modelo de agente pedagógico empático para la educación sobre buenas prácticas en el uso de las redes sociales. El agente es capaz de reconocer la emoción del usuario cuando interactúa con la red social. En base a la emoción del usuario y su comportamiento en la red social, el agente estima un plan para educar al usuario en el uso correcto y seguro de las redes sociales. / [CA] Els éssers humans som per naturalesa éssers afectius, les emocions, l'estat d'ànim, la nostra personalitat o les nostres relacions amb els altres guien les nostres motivacions i les nostres decisions. Una de les habilitats cognitives principals relacionades amb l'afecte és l'empatia. L'empatia és un constructe psicològic la definició del qual ha anat evolucionant al llarg dels anys i el significat del qual fa referència a un ampli ventall de competències afectives i cognitives que són fonamentals en el desenvolupament de l'ésser humà com a ésser social. L'ús de l'empatia en l'àmbit de la intel·ligència artificial pot revolucionar la forma en la qual interactuem amb les màquines així com la forma en què simulem el comportament humà. D'altra banda, cal tenir en compte que els éssers humans habitualment acudim a l'ús de diferents paraules com ``trist'' o ``content'' per expressar o verbalitzar l'estat afectiu. Tot i això, aquestes paraules són simplificacions que abasten un ampli espectre de processos cognitius i estats mentals. A més, cal considerar que aquestes paraules tenen una alta dependència de l'idioma i la cultura en què s'utilitzen. Per tant, els models de representació computacional dels estats afectius han de ser capaços d'adaptar-se a diferents entorns culturals i de permetre que un agent expresse o represente, mitjançant paraules, un estat afectiu determinat. En aquesta tesi es proposa un nou model d'agent empàtic capaç d'adaptar el seu comportament a diferents entorns culturals. Per això, en primer lloc, es presenta una metodologia nova basada en l'experimentació per adaptar un espai de representació d'emocions basat en les dimensions del plaer i l'activació per a la simulació i el reconeixement computacional afectiu a diferents entorns culturals. Els resultats de l'experiment realitzat amb hispanoparlants europeus es fan servir per proporcionar un nou model basat en la lògica difusa per representar estats afectius en les dimensions de plaer i activació mitjançant un enfocament de coordenades polars. Per demostrar que les diferències culturals afecten els valors de plaer i activació associats a cada emoció, l'experiment es va repetir amb participants portuguesos i suecs. En segon lloc, es presenta un nou model d'elicitació d'emocions en agents afectius que fa servir lògica difusa. Les emocions generades a l'agent per les regles de valoració difusa s'expressen en el model de representació de l'afecte resultant dels experiments descrits prèviament. A més, es proposa un nou procés de regulació de l'afecte que adapta l'estat d'ànim de l'agent, representat mitjançant un vector a l'espai plaer-activació, cada cop que una emoció és elicitada. En tercer lloc, es proposa una formalització de la sintaxi, semàntica i cicle de raonament d'AgentSpeak per permetre el desenvolupament d'agents afectius amb capacitats empàtiques. Partint de les teories de valoració empàtica i regulació empàtica, es modifica l'estructura de raonament de l'agent per permetre que l'empatia afecti el procés de presa de decisions. Finalment, es presenta un model d'agent pedagògic empàtic per a l'educació sobre bones pràctiques en l'ús de les xarxes socials. L'agent és capaç de reconèixer l'emoció de l'usuari quan interactua amb la xarxa social. En base a l'emoció de l'usuari i el seu comportament a la xarxa social, l'agent estima un pla per educar l'usuari en l'ús correcte i segur de les xarxes socials. / [EN] Human beings are, by nature, affective beings; our emotions, moods, personality, or relationships with others guide our motivations and our decisions. One of the main cognitive abilities related to affect is empathy. Empathy is a psychological construct whose definition has evolved over the years and whose meaning refers to a wide range of affective and cognitive competencies that are fundamental in the development of human beings as social beings. The use of empathy in the field of artificial intelligence can revolutionize the way we interact with machines as well as the way we simulate human behavior. On the other hand, it must be considered that human beings usually resort to the use of different words such as ``sad'' or ``happy'' to express or verbalize our affective state. However, these words are simplifications that cover a wide spectrum of cognitive processes and mental states. Moreover, it should be considered that these words have a high dependence on the language and culture in which they are used. Therefore, computational representation models of affective states must adaptable to different cultural environments and to allow an agent to express or represent, by means of words, a given affective state. In this thesis, a new model of empathic agent capable of adapting its behavior to different cultural environments is proposed. To this end, first, a new experiment-based methodology to adapt an emotion representation space based on the dimensions of pleasure and arousal for simulation and affective computational recognition to different cultural environments is presented. The results of an experiment conducted with European Spanish speakers are used to provide a new fuzzy logic-based model for representing affective states in the dimensions of pleasure and arousal using a polar coordinate approach. To prove that cultural differences affect the pleasure and arousal values associated with each emotion, the experiment was replicated with Portuguese and Swedish participants. Secondly, a new model of emotion elicitation in affective agents using fuzzy logic is presented. The emotions generated in the agent by the fuzzy appraisal rules are expressed in the model of affect representation resulting from the previously described experiments. In addition, a new affect regulation process is proposed to adapt the agent's mood, represented by a vector in the pleasure-arousal space, when an emotion is elicited. Third, a formalization of the syntax, semantics and reasoning cycle of AgentSpeak to enable the development of affective agents with empathic capabilities is presented. Drawing on the theories of empathic appraisal and empathic regulation, the agent's reasoning structure is modified to allow empathy to affect the decision-making process. Finally, a model of an empathic pedagogical agent for education on good practices in the use of social networks is introduced. The agent is able to recognize the user's emotion when interacting with the social network. Based on the user's emotion and behavior in the social network, the agent estimates a plan to educate the user in the correct and secure use of social networks. / This thesis has been partially supported by the Generalitat Valenciana and European Social Fund by the FPI grant ACIF/2017/085 and by the Spanish Government project PID2020- 113416RB-I00. / Taverner Aparicio, JJ. (2022). A BDI Empathic Agent Model Based on a Multidimensional Cross-Cultural Emotion Representation [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/181181 / TESIS / Compendio

Abstract in English / Mathematical modeling of the spread of infectious diseases in a population has always been recognized as a powerful tool that can help decision-makers understand how a disease evolves over time. With the evolution of science and humanity, it has become evident that Mathematical models are too simplistic and have some limitations in modeling environmental phenomena, such as the spread of epidemics in a population, when they are applied without combining them with other sciences. In understanding the dynamics of epidemics in a population, the weakness of these models is their difficulty in grasping the complexity inherent in the spread of diseases in real life because, life is supported by human interactions and behaviors that are understood through networks of social and spatial interactions. Modeling the spread of epidemics which takes this reality into account requires the implementation of new tools to refine the results already obtained by mathematical models. The aim of this thesis is to explore and attempt to extend new developments in mathematical modeling of the spread of infectious diseases by proposing new tools based on mathematical models from differential equations and agent-based models from intelligent agents derived from artificial intelligence. To achieve this objective, the study starts from a comparative study of two ways of modeling and simulation of the spread of infectious diseases in the population, namely mathematical modeling and agent-based modeling with a concrete case study of the spread of tuberculosis based on data from the Democratic Republic of the Congo (DRC). Then comes a coupling study of these two approaches in a single model and its implementation in a multi-scale environment. The results show that the coupled model is more realistic compared to mathematical models generally implemented in the literature. Four case studies are presented in this thesis. Mathematical modeling based on differential equations is used in the first and second cases. The third case is based on intelligent agents model while the last one is based on the coupling of mathematical models and agent-based models. Application of implemented models to the spread of tuberculosis reveals that detection of people with latent tuberculosis and their treatment are among the actions to be taken into account in addition to those currently carried out by the Congolese health system. The models assert that the current TB situation in DRC remains endemic and that the necessary measures need to be taken to reduce the burden of TB, especially to control it, through the tuberculosis elimination strategy and its elimination in the future in accordance with the Sustainable Development Goals. Our hybrid model benefiting from the advantages of EBM and ABM confirms that taking the individual into account as a fully-fledged entity and managing their behavior gives the microscopic aspect of the model set up and brings it closer as much as possible to reality. Mathematical management of the spread of the disease in cities gives a macroscopic aspect to the model. Numerical simulations of this last model on a multi-scale virtual environment affirm that the mobility of individuals from city to city has a significant impact on the spread of tuberculosis in the population. Controlling the rate of population mobility from one city to another is one of the most important measures for large-scale disease control. This model therefore draws its richness from this dynamic at two different scales (two time scales modeling approaches: at the microscopic/individual level (ABM) and macroscopic/city level (ODE)), which gives the emergence of the model at the global level. As a result, it seems that the coupling of mathematical models to agent-based models should be applied when the dynamics of the complex system under consideration is at different scales. Based on our research results, it seems that the choice of an approach must depend on how the modeler would like to achieve the expected results. Mathematical models remain essential due to their analytical and synthetic aspect, but their coupling with intelligent agent-based models makes it possible to refine known results and thus reflect the reality of real life, because the resulting model integrate interactions of individuals and their heterogeneous behaviors that are necessary for understanding the spread of infectious diseases in the population that only mathematical models based on differential equations can not capture. / Mathematical Sciences / Ph D. (Applied Mathematics)

Les travaux de thèse s’intéressent au réglage de la puissance active injectée dans le réseau, ce qui représente aujourd'hui l'une des problématiques principales du pilotage des parcs éoliens participant à la gestion du réseau. Dans le même temps, l'un des buts reste de maximiser la puissance extraite du vent en considérant les effets de couplage aérodynamique entre les éoliennes.La structure du contrôle-commande choisie est de type hiérarchisée et distribuée. Dans la première partie de la thèse, les travaux portent sur la commande de la turbine d'une éolienne autour des points de fonctionnement classiques mais également autour des points à puissance extraite réduite. En fait, cela relève d’une condition de fonctionnement nécessaire pour l'atteinte des objectifs imposés au pilotage d'un parc éolien.Dans la deuxième partie, le problème du contrôle à l'échelle d'un parc est posé sous la forme d'une optimisation distribuée parmi les turbines. Deux nouveaux algorithmes d'optimisation métaheuristique sont proposés et leur performance testée sur différents exemples de parcs éoliens. Les deux algorithmes s'appuient sur la méthode d'optimisation par essaim particulaire, qui est ici modifiée et adaptée pour les cas d'application aux systèmes multi agents. L'architecture de contrôlecommande globale est enfin évaluée en considérant les dynamiques des turbines contrôlées. Les simulations effectuées montrent des gains potentiels significatifs en puissance.Finalement, dans la troisième partie de la thèse, l'introduction d'une nouvelle étape de coopération au niveau des contrôleurs locaux des turbines, par l'utilisation de la technique de contrôle par consensus, permet d'améliorer les performances du système global. / In this PhD work we focus on the wind farm (WF) active power control since some of the new set grid requirements of interest can be expressed as specifications on its injection in the electric grid. Besides, one of our main objectives is related to the wind farm power maximization problem under the presence on non-negligible wake effect. The chosen WF control architecture has a two-layer hierarchical distributed structure. First of all, the wind turbine (WT) control is addressed. Here, a nonlinear controller lets a WT work in classic zones of functioning as well as track general deloaded power references. This last feature is a necessary condition to accomplish the WF control specifications. Secondly, the high level WF control problem is formulated as an optimization problem distributed among the WTs. Two novel distributed optimization algorithms are proposed, and their performance tested on different WF examples. Both are based on the well-known particle swarm optimization algorithm, which we modify and extend to be applicable in the multi-agent system framework. Finally, the overall WF control is evaluated by taking into account the WTs controlled dynamics. Simulations show potential significant power gains. Eventually, the introduction of a new control level in the hierarchical structure between the WF optimization and the WTs controllers is proposed. The idea is to let further cooperation among the WT local controllers, via a consensusbased technique, to enhance the overall system performance.

<p>Autonomous systems are often deployed in dynamic environments and are challenged with unexpected changes (novelties) in the environments where they receive novel data that was not seen during training. Given the uncertainty, they should be able to operate without (or with limited) human intervention and they are expected to (1) Adapt to such changes while still being effective and efficient in performing their multiple tasks. The system should be able to provide continuous availability of its critical functionalities. (2) Make informed decisions independently from any central authority. (3) Be Cognitive: learns the new context, its possible actions, and be rich in knowledge discovery through mining and pattern recognition. (4) Be Reflexive: reacts to novel unknown data as well as to security threats without terminating on-going critical missions. These characteristics combine to create the workflow of autonomous decision-making process in multi-agent environments (i.e.,) any action taken by the system must go through these characteristic models to autonomously make an ideal decision based on the situation. </p> <p><br></p> <p>In this dissertation, we propose novel learning-based models to enhance the decision-making process in autonomous multi-agent systems where agents are able to detect novelties (i.e., unexpected changes in the environment), and adapt to it in a timely manner. For this purpose, we explore two complex and highly dynamic domains </p> <p>(1) Transportation Networks (e.g., Ridesharing application): where we develop AdaPool: a novel distributed diurnal-adaptive decision-making framework for multi-agent autonomous vehicles using model-free deep reinforcement learning and change point detection. (2) Multi-agent games (e.g., Monopoly): for which we propose a hybrid approach that combines deep reinforcement learning (for frequent but complex decisions) with a fixed-policy approach (for infrequent but straightforward decisions) to facilitate decision-making and it is also adaptive to novelties. (3) Further, we present a domain agnostic approach for decision making without prior knowledge in dynamic environments using Bootstrapped DQN. Finally, to enhance security of autonomous multi-agent systems, (4) we develop a machine learning based resilience testing of address randomization moving target defense. Additionally, to further  improve the decision-making process, we present (5) a novel framework for multi-agent deep covering option discovery that is designed to accelerate exploration (which is the first step of decision-making for autonomous agents), by identifying potential collaborative agents and encouraging visiting the under-represented states in their joint observation space. </p>

Cooperative control problems for multiple vehicle systems can be categorized as either formation control problems with applications to mobile robots, unmanned air vehicles, autonomous underwater vehicles, satellites, aircraft, spacecraft, and automated highway systems, or non-formation control problems such as task assignment, cooperative transport, cooperative role assignment, air traffic control, cooperative timing, and cooperative search. The cooperative control of multiple vehicle systems poses significant theoretical and practical challenges. For cooperative control strategies to be successful, numerous issues must be addressed. We consider three important and correlated issues: consensus seeking, formation keeping, and trajectory tracking. For consensus seeking, we investigate algorithms and protocols so that a team of vehicles can reach consensus on the values of the coordination data in the presence of imperfect sensors, communication dropout, sparse communication topologies, and noisy and unreliable communication links. The main contribution of this dissertation in this area is that we show necessary and/or sufficient conditions for consensus seeking with limited, unidirectional, and unreliable information exchange under fixed and switching interaction topologies (through either communication or sensing). For formation keeping, we apply a so-called "virtual structure" approach to spacecraft formation flying and multi-vehicle formation maneuvers. As a result, single vehicle path planning and trajectory generation techniques can be employed for the virtual structure while trajectory tracking strategies can be employed for each vehicle. The main contribution of this dissertation in this area is that we propose a decentralized architecture for multiple spacecraft formation flying in deep space with formation feedback introduced. This architecture ensures the necessary precision in the presence of actuator saturation, internal and external disturbances, and stringent inter-vehicle communication limitations. A constructive approach based on the satisficing control paradigm is also applied to multi-robot coordination in hardware. For trajectory tracking, we investigate nonlinear tracking controllers for fixed wing unmanned air vehicles and nonholonomic mobile robots with velocity and heading rate constraints. The main contribution of this dissertation in this area is that our proposed tracking controllers are shown to be robust to input uncertainties and measurement noise, and are computationally simple and can be implemented with low-cost, low-power microcontrollers. In addition, our approach allows piecewise continuous reference velocity and heading rate and can be extended to derive a variety of other trajectory tracking strategies.

Recent urbanization and industrialization have brought tremendous pressure and challenges to modern autonomous systems. When considering multiple complex tasks, cooperation and coordination between multiple agents can improve efficiency in a system. In real-world applications, multi-agent systems (MAS) are widely used in various fields, such as robotics, unmanned aerial systems, autonomous vehicles, distributed sensor networks, etc. Unlike traditional MAS systems based on pre-defined algorithms and rules, a special human-in-loop (HIL) based MAS involves human interactions to enhance the system’s adaptability for special scenarios, as well as apply human preferences for robot control. However, existing HIL strategies are primarily based on human involvement at a low level, such as mixed-initiative control and mixed-agent scenarios with both human-driven and intelligent robots. There are fewer investigations on applying HIL in high-level coordination. In particular, designing a coordination strategy for multi-task multi-agent scenarios, which can also deal with real-time human commands, will be one of the key topics of this Master’s thesis project. In this thesis work, different kinds of tasks described by signal temporal logic (STL) are created for agents, which can be enforced by control barrier function (CBF) constraints. Both centralized and distributed frameworks are designed for agent coordination. In detail, the centralized strategy is developed for machine-to-infrastructure (M2I) communication, by using the nonlinear model predictive control (NMPC) method to obtain collision-free trajectories. The distributed strategy utilizing graph theory is proposed for machine-to-machine (M2M), in order to reduce computation time by offloading. Most importantly, a HIL model is generated for both frameworks to apply online human commands to the coordination, with a novel task allocation protocol. Simulations and experiments are carried out on both Matlab and Python-based ROS simulators, to show that proposed frameworks can achieve obvious performance advantages in safety, smoothness, and stability for task completion. Numerical results are provided to validate the feasibility and applicability of our algorithms. / Den senaste urbaniseringen och industrialiseringen har medfört enormt tryck och utmaningar för moderna autonoma system. Vid beaktande av flera komplexa uppgifter kan samarbete och samordning mellan flera agenter förbättra effektiviteten i ett system. I verkliga tillämpningar används multiagent-system (MAS) i stor utsträckning inom olika områden, såsom robotik, obemannade luftfarkoster, autonoma fordon, distribuerade sensorsystem etc. Till skillnad från traditionella MAS-system baserade på fördefinierade algoritmer och regler, innebär ett särskilt människa-i-loop (HIL)-baserat MAS mänsklig interaktion för att förbättra systemets anpassningsförmåga till speciella scenarier samt anpassa mänskliga preferenser för robotstyrning. Emellertid är befintliga HIL-strategier främst baserade på mänsklig inblandning på en låg nivå, såsom mixad-initiativkontroll och mixade agentscenarier med både människa-drivna och intelligenta robotar. Det finns färre undersökningar om att tillämpa HIL på högnivåkoordination. Särskilt att utforma en koordineringsstrategi för fleruppgiftsfleragent-scenarier, som också kan hantera mänskliga kommandon i realtid, kommer att vara ett av huvudämnena för detta masterprojekt. I detta examensarbete skapas olika typer av uppgifter beskrivna av signaltemporallogik (STL) för agenter, som kan upprätthållas genom styrbarriärfunktions (CBF) -begränsningar. Både centraliserade och distribuerade ramverk utformas för agentkoordination. Mer specifikt utvecklas den centraliserade strategin för maskin-till-infrastruktur (M2I)-kommunikation genom att använda icke-linjär modellprediktiv reglering (NMPC) för att erhålla kollisionsfria trajektorier. Den distribuerade strategin med användning av grafteori föreslås för maskin-till-maskin (M2M) för att minska beräkningstiden genom avlastning. Viktigast av allt genereras en HIL-modell för båda ramverken för att tillämpa online-mänskliga kommandon på koordinationen med en ny protokoll för uppgiftstilldelning. Simuleringar och experiment utförs på både Matlab och Python-baserade ROS-simulatorer för att visa att de föreslagna ramverken kan uppnå tydliga prestandafördelar när det gäller säkerhet, smidighet och stabilitet för uppgiftsslutförande. Numeriska resultat presenteras för att validera genomförbarheten och tillämpligheten hos våra algoritmer.