Cette thèse de doctorat analyse les concepts impliqués dans la prise de décisions de groupes d'agents et applique ces concepts dans la création d'un cadre théorique et pratique qui permet la reconnaissance de comportements de groupes. Nous allons présenter une vue d'ensemble de la théorie de l'intention, étudiée dans le passé par quelques grands théoriciens comme Searle, Bratmann et Cohen, et nous allons montrer le lien avec des recherches plus récentes dans le domaine de la reconnaissance de comportements. Nous allons étudier les avantages et inconvénients des techniques les plus avancées dans ce domaine et nous allons créer un nouveau modèle qui représente et détecte les comportements de groupes. Ce nouveau modèle s'appelle Multiagent-Abstract Hidden Markov mEmory Model (M-AHMEM) et résulte de la fusion de modèles déjà existants, le but étant de créer une approche unifiée du problème. La plus grande partie de cette thèse est consacrée à la présentation détaillée du M-AHMEM et de l'algorithme responsable de la reconnaissance de comportements. Notre modèle sera testé sur deux applications différentes : l'analyse gesturale humaine et la fusion multimodale des données audio et vidéo. A travers ces deux applications, nous avançons l'argument qu'un ensemble de données constitué de plusieurs variables corrélées peut être analysé efficacement sous un cadre unifié de reconnaissance de comportements. Nous allons montrer que la corrélation entre les différentes variables peut être modélisée comme une coopération ayant lieu à l'intérieur d'une équipe et que la reconnaissance de comportements constitue une approche moderne de classification et de reconnaissance de patrons.

One key to more effective cooperative interaction in a multi-robot team is the ability to understand the behavior and intent of other robots. Observed teammate action sequences can be learned to perform trajectory recognition which can be used to determine their current task. Previously, we have applied behavior histograms, hidden Markov models (HMMs), and conditional random fields (CRFs) to perform trajectory recognition as an approach to task monitoring in the absence of commu- nication. To demonstrate trajectory recognition of various autonomous vehicles, we used trajectory-based techniques for model generation and trajectory discrimination in experiments using actual data. In addition to recognition of trajectories, we in- troduced strategies, based on the honeybee’s waggle dance, in which cooperating autonomous teammates could leverage recognition during periods of communication loss. While the recognition methods were able to discriminate between the standard trajectories performed in a typical survey mission, there were inaccuracies and delays in identifying new trajectories after a transition had occurred. Inaccuracies in recog- nition lead to inefficiencies as cooperating teammates acted on incorrect data. We then introduce the Trajectory Adaptation for Recognition (TAR) framework which seeks to directly address difficulties in recognizing the trajectories of autonomous vehicles by modifying the trajectories they follow to perform them. Optimization techniques are used to modify the trajectories to increase the accuracy of recognition while also improving task objectives and maintaining vehicle dynamics. Experiments are performed which demonstrate that using trajectories optimized in this manner lead to improved recognition accuracy.

The work presented in this dissertation investigates techniques for robot Learning from Demonstration (LFD). LFD is a well established approach where the robot is to learn from a set of demonstrations. The dissertation focuses on LFD where a human teacher demonstrates a behavior by controlling the robot via teleoperation. After demonstration, the robot should be able to reproduce the demonstrated behavior under varying conditions. In particular, the dissertation investigates techniques where previous behavioral knowledge is used as bias for generalization of demonstrations. The primary contribution of this work is the development and evaluation of a semi-reactive approach to LFD called Predictive Sequence Learning (PSL). PSL has many interesting properties applied as a learning algorithm for robots. Few assumptions are introduced and little task-specific configuration is needed. PSL can be seen as a variable-order Markov model that progressively builds up the ability to predict or simulate future sensory-motor events, given a history of past events. The knowledge base generated during learning can be used to control the robot, such that the demonstrated behavior is reproduced. The same knowledge base can also be used to recognize an on-going behavior by comparing predicted sensor states with actual observations. Behavior recognition is an important part of LFD, both as a way to communicate with the human user and as a technique that allows the robot to use previous knowledge as parts of new, more complex, controllers. In addition to the work on PSL, this dissertation provides a broad discussion on representation, recognition, and learning of robot behavior. LFD-related concepts such as demonstration, repetition, goal, and behavior are defined and analyzed, with focus on how bias is introduced by the use of behavior primitives. This analysis results in a formalism where LFD is described as transitions between information spaces. Assuming that the behavior recognition problem is partly solved, ways to deal with remaining ambiguities in the interpretation of a demonstration are proposed. The evaluation of PSL shows that the algorithm can efficiently learn and reproduce simple behaviors. The algorithm is able to generalize to previously unseen situations while maintaining the reactive properties of the system. As the complexity of the demonstrated behavior increases, knowledge of one part of the behavior sometimes interferes with knowledge of another parts. As a result, different situations with similar sensory-motor interactions are sometimes confused and the robot fails to reproduce the behavior. One way to handle these issues is to introduce a context layer that can support PSL by providing bias for predictions. Parts of the knowledge base that appear to fit the present context are highlighted, while other parts are inhibited. Which context should be active is continually re-evaluated using behavior recognition. This technique takes inspiration from several neurocomputational models that describe parts of the human brain as a hierarchical prediction system. With behavior recognition active, continually selecting the most suitable context for the present situation, the problem of knowledge interference is significantly reduced and the robot can successfully reproduce also more complex behaviors.

Behavioral research involves the study of the behaviors of one or more agents (often animals) in order to better understand the agents' thoughts and actions. Identifying subject movements and behaviors based upon those movements is a critical, time-consuming step in behavioral research. To successfully perform behavior analysis, three goals must be met. First, the agents of interest are observed, and their movements recorded. Second, each individual must be uniquely identified. Finally, behaviors must be identified and recognized. I explore a system that can uniquely identify and track agents, then use these tracks to automatically build behavioral models and recognize similar behaviors in the future. I address the tracking and identification problems using a combination of laser range finders, active RFID sensors, and probabilistic models for real-time tracking. The laser range component adds environmental flexibility over vision based systems, while the RFID tags help disambiguate individual agents. The probabilistic models are important to target identification during the complex interactions with other agents of similar appearance. In addition to tracking, I present work on automatic methods for generating behavioral models based on supervised learning techniques using the agents' tracked data. These models can be used to classify new tracked data and identify the behavior exhibited by the agent, which can then be used to help automate behavior analysis.

Dans un contexte applicatif de surveillance de zone maritime, nous voulons fournir à un opérateur humain des informations sémantiquement riches et dynamiques relatives aux comportements des entités sous surveillance. Réussir à relier les mesures brutes en provenance d’un système de capteurs aux descriptions abstraites de ces comportements est un problème difficile. Ce dernier est d’ailleurs en général traité en deux temps: tout d’abord, réaliser un prétraitement sur les données hétérogènes, multidimensionnelles et imprécises pour les transformer en un flux d’évènements symbolique, puis utiliser des techniques de reconnaissance de plans sur ces mêmes évènements. Ceci permet de décrire des étapes de plans symboliques de haut niveau sans avoir à se soucier des spécificités des capteurs bas niveau. Cependant, cette première étape est destructrice d’information et de ce fait génère une ambigüité supplémentaire dans le processus de reconnaissance. De plus, séparer les tâches de reconnaissance de comportements est générateur de calculs redondants et rend l’écriture de la bibliothèque de plans plus ardue. Ainsi, nous proposons d’aborder cette problématique sans séparer en deux le processus de reconnaissance. Pour y parvenir, nous proposons un nouveau modèle hiérarchique, inspiré de la théorie des langages formels, nous permettant de construire un pont au-dessus du fossé sémantique séparant les mesures des capteurs des intentions des entités. Grâce à l’aide d’un ensemble d’algorithmes manipulant ce modèle, nous sommes capables, à partir d’observations, de déduire les plausibles futures évolutions de la zone sous surveillance, tout en les justifiant des explications nécessaires. / In a maritime area supervision context, we seek providing a human operator with dynamic information on the behaviors of the monitored entities. Linking raw measurements, coming from sensors, with the abstract descriptions of those behaviors is a tough challenge. This problem is usually addressed with a two-stepped treatment: filtering the multidimensional, heterogeneous and imprecise measurements into symbolic events and then using efficient plan recognition techniques on those events. This allows, among other things, the possibility of describing high level symbolic plan steps without being overwhelmed by low level sensor specificities. However, the first step is information destructive and generates additional ambiguity in the recognition process. Furthermore, splitting the behavior recognition task leads to unnecessary computations and makes the building of the plan library tougher. Thus, we propose to tackle this problem without dividing the solution into two processes. We present a hierarchical model, inspired by the formal language theory, allowing us to describe behaviors in a continuous way, and build a bridge over the semantic gap between measurements and intents. Thanks to a set of algorithms using this model, we are able, from observations, to deduce the possible future developments of the monitored area while providing the appropriate explanations.

Automated analysis of temporal data is a task of utmost importance for intelligent machines. For example, ubiquitous computing systems need to understand the intention of humans from the stream of sensory information, and health-care monitoring systems can assist patients and doctors by providing automatically annotated daily health reports. We present a set of extensions of switching linear dynamic systems (SLDSs) which provide the ability to capture the higher-order temporal structures within data and to produce more accurate results for the tasks such as labeling and estimation of global variations within data. The presented models are formulated within a dynamic Bayesian network formulation along with the inference and learning methods thereof. First, segmental SLDSs (S-SLDSs) produce superior labeling results by capturing the descriptive duration patterns within each LDS segment. The encoded duration models describe data more descriptively and allow us to avoid the severe problem of over-segmented labels, which leads to superior accuracy. Second, parametric SLDSs (P-SLDSs) allows us to encode the temporal data with global variations. In particular, we have identified two types of global systematic variations : temporal and spatial variations. The P-SLDS model assumes that there is an underlying canonical model which is globally transformed in time and space by the two associated global parameters respectively. Third, we present hierarchical SLDSs (H-SLDSs), a generalization of standard SLDSs with hierarchic Markov chains. H-SLDSs are able to encode temporal data which exhibits hierarchic structure where the underlying low-level temporal patterns repeatedly appear among different higher-level contexts. The developed SLDS extensions have been applied to two real-world problems. The first problem is to automatically decode the dance messages of honey bee dances where the goal is to correctly segment the dance sequences into different regimes and parse the messages about the location of food sources embedded in the data. The second problem is to analyze wearable exercise data where we aim to provide an automatically generated exercise record at multiple temporal and semantic resolutions. It is demonstrated that the H-SLDS model with multiple layers can be learned from data, and can be successfully applied to interpret the exercise data at multiple granularities.

The work presented in this dissertation investigates techniques for robot Learning from Demonstration (LFD). LFD is a well established approach where the robot is to learn from a set of demonstrations. The dissertation focuses on LFD where a human teacher demonstrates a behavior by controlling the robot via teleoperation. After demonstration, the robot should be able to reproduce the demonstrated behavior under varying conditions. In particular, the dissertation investigates techniques where previous behavioral knowledge is used as bias for generalization of demonstrations. The primary contribution of this work is the development and evaluation of a semi-reactive approach to LFD called Predictive Sequence Learning (PSL). PSL has many interesting properties applied as a learning algorithm for robots. Few assumptions are introduced and little task-specific configuration is needed. PSL can be seen as a variable-order Markov model that progressively builds up the ability to predict or simulate future sensory-motor events, given a history of past events. The knowledge base generated during learning can be used to control the robot, such that the demonstrated behavior is reproduced. The same knowledge base can also be used to recognize an on-going behavior by comparing predicted sensor states with actual observations. Behavior recognition is an important part of LFD, both as a way to communicate with the human user and as a technique that allows the robot to use previous knowledge as parts of new, more complex, controllers. In addition to the work on PSL, this dissertation provides a broad discussion on representation, recognition, and learning of robot behavior. LFD-related concepts such as demonstration, repetition, goal, and behavior are defined and analyzed, with focus on how bias is introduced by the use of behavior primitives. This analysis results in a formalism where LFD is described as transitions between information spaces. Assuming that the behavior recognition problem is partly solved, ways to deal with remaining ambiguities in the interpretation of a demonstration are proposed. The evaluation of PSL shows that the algorithm can efficiently learn and reproduce simple behaviors. The algorithm is able to generalize to previously unseen situations while maintaining the reactive properties of the system. As the complexity of the demonstrated behavior increases, knowledge of one part of the behavior sometimes interferes with knowledge of another parts. As a result, different situations with similar sensory-motor interactions are sometimes confused and the robot fails to reproduce the behavior. One way to handle these issues is to introduce a context layer that can support PSL by providing bias for predictions. Parts of the knowledge base that appear to fit the present context are highlighted, while other parts are inhibited. Which context should be active is continually re-evaluated using behavior recognition. This technique takes inspiration from several neurocomputational models that describe parts of the human brain as a hierarchical prediction system. With behavior recognition active, continually selecting the most suitable context for the present situation, the problem of knowledge interference is significantly reduced and the robot can successfully reproduce also more complex behaviors.

Vehicle communication using sensors and wireless channels plays an important role to allow exchanging information. Adding more components to allow exchanging more information with infrastructure enhanced the capabilities of vehicles and enabled the rise of Cooperative Intelligent Transport Systems (C-ITS). Leveraging such capabilities, more applications such as Cooperative Adaptive Cruise Control (CACC) and platooning were introduced. CACC is an enhancement of Adaptive Cruise Control (ACC). It enables longitudinal automated vehicle control and follows the Constant Time Gap (CTG) strategy where, distance between vehicles is proportional to the speed. Platooning is different in terms of addressing both longitudinal and lateral control. In addition, it adopts the Constant Distance Gap (CDG) control strategy, with separation between vehicles unchanged with speed. Platooning requires close coupling and accordingly achieves goals of increased lane throughput and reduced energy consumption. When a longitudinal controller only is used, platooning operates in car-following mode and no Platoon Management Protocol (PMP) is used. On the other hand, when both longitudinal and lateral controllers are used, platooning operates in maneuver mode and coordination between vehicles is needed to perform maneuvers. Exchanging information allows the platoon to make real time maneuvering decisions. However, all the aforementioned benefits of platooning cannot be achieved if the system is vulnerable to misbehavior (i.e., the platoon is behaving incorrectly). Most of work in the literature attributes this misbehavior to malicious actors where an attacker injects malicious messages. Standards made efforts to develop security services to authenticate and authorize the sender. However, authenticated users equipped with cryptographic primitives can mount attacks (i.e., falsification attacks) and accordingly they cannot be detected by standard services such as cryptographic signatures. Misbehavior can disturb platoon behavior or even cause collision. Many Misbehavior Detection Schemes (MDSs) are proposed in the literature in the context of Vehicular ad hoc network (VANET) and CACC. These MDSs apply algorithms or rules to detect sudden or gradual changes of kinematic information disseminated by other vehicles. Reusing these MDSs directly during maneuvers can lead to false positives when they treat changes in kinematic information during the maneuver as an attack. This thesis addresses this gap by designing a new modular framework that has the capability to discern maneuvering process from misbehavior by leveraging platoon behavior recognition, that is, the platoon mode of operation (e.g., car-following mode or maneuver mode). In addition, it has the ability to recognize the undergoing maneuver (e.g., middle join or exit). Based on the platoon behavior recognition module, the anomaly detection module detects deviations from expected behavior. Unsupervised machine learning, notably Hidden Markov Model with Gaussian Mixture Model emission (GMMHMM), is used to learn the nominal behavior of the platoon during different modes and maneuvers. This is used later by the platoon behavior recognition and anomaly detection modules. GMMHMM is trained with nominal behavior of platoon using multivariate time series representing kinematic characteristics of the vehicles. Different models are used to detect attacks in different scenarios (e.g., different speeds). Two approaches for anomaly detection are investigated, Viterbi algorithm based anomaly detection and Forward algorithm based anomaly detection. The proposed framework managed to detect misbehavior whether the compromised vehicle is a platoon leader or follower. Empirical results show very high performance, with the platoon behavior recognition module reaching 100% in terms of accuracy. In addition, it can predict ongoing platoon behavior at early stages and accordingly, use the correct model representing the nominal behavior. Forward algorithm based anomaly detection, which rely on computing likelihood, showed better performance reaching 98% with slight variations in terms of accuracy, precision, recall and F1 score. Different platooning controllers can be resilient to some attacks and accordingly, the attack can result in slight deviation from nominal behavior. However, The anomaly detection module was able to detect this deviation. / Kommunikation mellan fordon som använder sensorer och radiokommunikation spelar en viktig roll för att kunna möjliggöra informationsutbyte. Genom att lägga till er komponenter för infrastrukturkommunikation förbättras fordonens generella kommunikationskapacitet och möjliggör C-ITS. Det möjliggör också för att introducera ytterligare applikationer, exempelvis CACC samt plutonering. CACC är en förbättring av ACC -konceptet. Denna teknik möjliggör longitudinell automatiserad fordonskontroll och följer en CTG -strategi där avståndet mellan fordon är proportionellt mot hastigheten. Plutonering är annorlunda med avseende på att hantera longitudinell och lateral kontroll. Dessutom antar den en kontrollstrategi för CDG där avståndet mellan fordon förblir oförändrat med hastighet. Plutonering kräver en nära koppling mellan fordon för att uppnå målet med ökad filgenomströmning och reducerad energikonsumtion. När enbart longitudinell kontroll är aktiverad, fungerar plutonering i bilföljande läge och funktionen PMP används inte. När både longitudinella och laterala kontroller används, arbetar plutonen istället i manöverläge och samordning mellan fordon behövs för att utföra olika manövrar. Informationsutbytet möjliggör att plutonen kan man manövrera i realtid. Alla ovan nämnda fördelar med plutonering kan emellertid inte uppnås om systemet är sårbart för felbeteende, det vill säga att plutonen beter sig fel. I litteraturen kopplas detta missförhållande till skadliga aktörer där en angripare injicerar skadliga meddelanden. I standardiseringsarbeten har man försökt utveckla säkerhetstjänster för att autentisera och auktorisera avsändaren. Trots detta kan autentiserade användare utrustade med kryptografiska primitiv upprätta förfalskningsattacker som inte detekteras av standardtjänster som kryptografiska signaturer. Felaktigt handhavande kan orsaka störningar i plutonens beteende eller till och med orsaka kollisioner och följaktligen påverka tillförlitligheten. Det finns manga MDSs beskrivna i litteraturen i relation till VANET och CACC. MDSs använder algoritmer eller regler för att detektera snabba eller långsamma förändringar kinematisk information som sprids av andra fordon. Direkt användning av MDSs under manövrar kan leda till falska positiva resultat eftersom de kommer att behandla förändringar i kinematisk information under manövern som en attack. Denna avhandling adresserar detta gap genom utformningen av ett modulärt ramverk som kan urskilja manöverprocessen från misskötsamhet genom att utnyttja plutonens beteendeigenkänningsmodul för att intelligent känna igen plutonläget (t.ex. bilföljande läge eller manöverläge). Ramverket har vidare egenskapen att känna igen pågående manövrar (frikoppling eller växelbyte) och avvikelser från förväntat beteende. Modulen använder en oövervakad maskininlärningssmodell, GMMHMM, för att lära en plutons normala beteende under olika lägen och manövrar som sedan används för plutonbeteendeigenkänning och avvikelsedetektion. GMMHMM tränas på data från plutoneringens normalbeteende i form av multivariata tidsserier som representerar fordonets kinematiska karakteristik. Olika modeller används för att upptäcka attacker i olika scenarier (t.ex. olika hastigheter). Två tillvägagångssätt för avvikelsedetektion undersöks, Viterbi-algoritmen samt Forward-algoritmen. Det föreslagna systemet lyckas upptäcka det felaktiga beteendet oavsett om det komprometterade fordonet är en plutonledare eller följare. Empiriska resultat visar mycket hög prestanda för beteendeigenkänningsmodulen som när 100%. Dessutom kan den känna igen plutonens beteende i ett tidigt skede. Resultat med Forward- algoritmen för avvikelsedetektion visar på en prestanda på 98% med små variationer med avseende på måtten accuracy, precision, recall och F1-score. Avvikelsedetektionsmodulen kan även upptäcka små avvikelser i beteende.

Cette thèse s'est déroulée dans le cadre du projet SECMAR qui visait à sécuriser le Grand Port Maritime de Marseille. Notre objectif était d'aider les personnels du port à identifier les comportements menaçant des navires afin de pouvoir agir efficacement en cas de danger réel. A ce titre, nous avons développé un système d'analyse et de reconnaissance de comportements de navires formé de deux sous-modules complémentaires. Le premier est construit à partir de l'approche probabiliste Modèle de Markov Cachée et traite principalement des comportements nominaux des gros bateaux qui se caractérisent par un déplacement régulier et récurrent dans le port. Le second est construit à partir du langage réactif synchrone Esterel et prend en compte les comportements agressifs et transgressifs de tous types de navires, notamment ceux des petits bateaux qui circulent librement et aléatoirement dans le port. Le système global d'aide à la décision a permis une bonne reconnaissance en temps-réel des différents comportements de navires au cours de leurs évolutions dans le port. Au regard des résultats prometteurs que nous avons obtenu à travers ce module, il est envisageable de le généraliser à d'autres ports mondiaux ainsi qu'à d'autres domaines d'application, notamment le domaine aéroportuaire. / The overall aim of this thesis was to create a decision support system that identifies discrepancies in ship behavior. The thesis was a part of the SECMAR project that aimed to improve security at the Marseilles harbor by the creation of decision support system for port staff. For this purpose, we developed a recognition behavior system consisting of two complementary sub-systems.The first system was based on the probabilistic Hidden Markov model approach and deals with nominal behavior of large to medium size commercial ships showing regular and recurrent behavior. The second system was based on the reactive synchronous language Esterel and concerns aggressive and transgressive behavior of small ships that may navigate freely in the harbor. Real-time evaluations showed that the proposed decision support system efficiently captured and evaluated ship behaviors. The promising results of the system and its diversity in origin makes it suitable for applications in other harbors as well as other environment such as airports.