Crowds of people are gathering at multiple venues, such as concerts, political rallies, as well as in commercial malls, or just simply walking on the streets. More and more people are flocking to live in urban areas, thus generating a lot of scenarios of crowds. As a consequence, there is an increasing demand for automatic tools that can analyze and predict the behavior of crowds to ensure safety. Crowd motion analysis is a key feature in surveillance and monitoring applications, providing useful hints about potential threats to safety and security in urban and public spaces. It is well known that people gatherings are generally difficult to model, due to the diversity of the agents composing the crowd. Each individual is unique, being driven not only by the destination but also by personality traits and attitude. The domain of crowd analysis has been widely investigated in the literature. However, crowd gatherings have sometimes resulted in dangerous scenarios in recent years, such as stampedes or during dangerous situations. To take a step toward ensuring the safety of crowds, in this work we investigate two main research problems: we try to predict each person future position and we try to understand which are the key factors for simulating crowds. Predicting in advance how a mass of people will fare in a given space would help in ensuring the safety of public gatherings.

This degree project addresses poaching challenges by employing predictive analysis of animal movements and their correlation with the dynamic environment using a machine learning approach. The goal is to provide accurate predictions of animal movements, enabling rangers to intercept potential threats and safeguard wildlife from snares. A wide analysis considers previous studies on animal movements and both animal and environment data availability. To efficiently represent the dynamic environment and correlate it with animal movement data, accurate matching of environment variables to each animal measurement is crucial. We selected multiple environment datasets to capture a sufficient amount ofenvironmental properties. Due to practical constraints, daily representation of the environment is not achievable, and weekly mean or monthly mode values are used instead. Data insights are obtained through the training of a regression neural network using the filtered environmental and animal movement data. The results highlight the significant role ofenvironmental features in predicting animal movements, emphasizing their importance for accurate predictions. Despite some offset and few erroneous predictions, a strong similarity between animal predicted trajectory and animal true trajectory was achieved, indicating that the model is capable to capture general patterns and to correctly tune in predictions of detailed movements as well. The overall offset of the trajectories is still a weak point of this model, but it may just indicate the presence of some underlying systematic error that can be corrected through further work. The integration of such a developed prediction model into existing frameworks could assist law enforcingauthorities in preventing poaching activities.

With the development of autonomous driving technology, vehicle trajectory prediction has become a hot topic in the intelligent traffic area. However, complex road conditions may bring multiple challenges to the vehicle trajectory prediction model. To address this, most recent studies mainly focus on designing different neural network structures to learn vehicles’ dynamics and interaction features for better prediction. In this thesis we restrict our research scope to highway scenarios. Based on the experimental comparison among Vanilla Recurrent Neural Network (Vanilla RNN), Vanilla Long short-term memory (Vanilla LSTM), and Vanilla-Transformer, we find the best configuration of the Dynamics-Only encoder module and utilize it to design a novel model called the LSTM-Attention model for vehicle trajectory prediction. The objective of our design is to explore whether the Self-Attention mechanism based encoder outperforms the pooling mechanism based encoder utilized in most current baseline models. The experiment results on the interaction encoder module show that the Self- Attention mechanism based encoder with 8 heads outperforms the pooling mechanism based encoder for the longer prediction horizons. To test the robustness of our LSTM-Attention model, we also compare the prediction performance between using Maneuver-Based decoder and using Maneuver-Free decoder, respectively. According to the experiment results, we find the Maneuver-Based decoder performs better on the heavily unbalanced Next Generation Simulation (NGSIM) dataset. Finally, to explore other latent interaction features our LSTM-Attention model might fuse, we analyze the Graph-Based encoder and the Polar-Based encoder, respectively. Based on this, we find more meaningful designs that could be exploited in our future work. / Med utvecklingen av självkörande fordon har förmågan att förutsäga fordonsbanan blivit ett attraktivt ämne inom intelligenta trafiksystem. Däremot kan komplexa vägförhållanden medföra flera utmaningar för modellering av fordonets bana. För att ta itu med detta fokuserar de senaste studierna huvudsakligen på att designa olika neurala nätverksstrukturer för att lära sig fordons dynamiker och interaktioner för bättre kunna förutsäga resebanan. I denna avhandling begränsar vi vårt forskningsområde till motorvägsscenarier. Baserat på den experimentella jamförelsen mellan Vanilla Recurrent Neural Network (Vanilla RNN), Vanilla Long-korttidsminne (Vanilla LSTM) och Vanilla-Transformer, hittar vi den bästa konfigurationen av Dynamic-Only kodningsmodulen och använder den för att designa en enkel modell som vi kallar LSTM- Attention-modellen för förutsägelse av fordonets resebana. Målet med vår design är att undersöka om den Self-Attention-baserade kodaren överträffar den pooling-baserade kodaren som används i de flesta nuvarande basmodeller. Experimentens resultat på interaktionskodarmodulen visar att Self-Attention kodaren med 8 huvuden överträffar den poolning baserade kodaren när de gäller längre fönster av förutsägelser. För att testa robustheten hos vår LSTM-Attention-modell, jämför vi också prestandan mellan att använda manöverbaserad avkodare respektive att använda manöverfri avkodare. Enligt experimentens resultat finner vi att den manöverbaserade avkodaren presterar bättre på den kraftigt obalanserade Next Generation Simulation (NGSIM) datamängden. Slutligen, för att utforska andra möjliga egenskaper som vår LSTM-Attention-modell kan utnytja, analyserar vi den grafbaserade kodaren respektive den polbaserade kodaren. Baserat på detta så hittar vi mer meningsfulla mönster som skulle kunna utnyttjas i framtida arbeten.

In the realm of autonomous vehicles, ensuring safety through advanced anomaly detection is crucial. This thesis integrates Reservoir Computing with temporal-aware data analysis to enhance driver behavior assessment and traffic flow prediction. Our approach combines Reservoir Computing with autoencoder-based feature extraction to analyze driving metrics from vehicle sensors, capturing complex temporal patterns efficiently. Additionally, we extend our analysis to forecast traffic flow dynamics within road networks using the same framework. We evaluate our model using the PEMS-BAY and METRA-LA datasets, encompassing diverse traffic scenarios, along with a GPS dataset of 10,000 taxis, providing real-world driving dynamics. Through a support vector machine (SVM) algorithm, we categorize drivers based on their performance, offering insights for tailored anomaly detection strategies. This research advances anomaly detection for autonomous vehicles, promoting safer driving experiences and the evolution of vehicle safety technologies. By integrating Reservoir Computing with temporal-aware data analysis, this thesis contributes to both driver behavior assessment and traffic flow prediction, addressing critical aspects of autonomous vehicle systems. / Master of Science / Our cities are constantly growing, and traffic congestion is a major challenge. This project explores how innovative technology can help us predict traffic patterns and develop smarter management strategies. Inspired by the rigorous safety systems being developed for self-driving cars, we'll delve into the world of machine learning. By combining advanced techniques for identifying unusual traffic patterns with tools that analyze data over time, we'll gain a deeper understanding of traffic flow and driver behavior. We'll utilize data collected by car sensors, such as speed and turning patterns, to not only predict traffic jams but also see how drivers react in different situations. However, our project has a broader scope than just traffic flow. We aim to leverage this framework to understand driver behavior in general, with a particular focus on its implications for self-driving vehicles. Through meticulous data analysis and sophisticated algorithms, we can categorize drivers based on their performance. This valuable information can be used to develop improved methods for detecting risky situations, ultimately leading to safer roads and smoother traffic flow for everyone. To ensure the effectiveness of our approach, we'll rigorously test it using real-world data from GPS data from taxi fleets and nationally recognized traffic datasets. By harnessing the power of machine learning and tools that can adapt to changing data patterns, this project has the potential to revolutionize traffic management in cities. This paves the way for a future with safer roads, less congestion, and a more positive experience for everyone who lives in and travels through our bustling urban centers.

<p dir="ltr">Autonomous vehicles require motion forecasting of their surrounding multiagents (pedestrians</p><p dir="ltr">and vehicles) to make optimal decisions for navigation. The existing methods focus on</p><p dir="ltr">techniques to utilize the positions and velocities of these agents and fail to capture semantic</p><p dir="ltr">information from the scene. Moreover, to mitigate the increase in computational complexity</p><p dir="ltr">associated with the number of agents in the scene, some works leverage Euclidean distance to</p><p dir="ltr">prune far-away agents. However, distance-based metric alone is insufficient to select relevant</p><p dir="ltr">agents and accurately perform their predictions. To resolve these issues, we propose the</p><p dir="ltr">Semantics-aware Interactive Multiagent Motion Forecasting (SIMMF) method to capture</p><p dir="ltr">semantics along with spatial information and optimally select relevant agents for motion</p><p dir="ltr">prediction. Specifically, we achieve this by implementing a semantic-aware selection of relevant</p><p dir="ltr">agents from the scene and passing them through an attention mechanism to extract</p><p dir="ltr">global encodings. These encodings along with agents’ local information, are passed through</p><p dir="ltr">an encoder to obtain time-dependent latent variables for a motion policy predicting the future</p><p dir="ltr">trajectories. Our results show that the proposed approach outperforms state-of-the-art</p><p dir="ltr">baselines and provides more accurate and scene-consistent predictions. </p>

Autonomous driving could bring profound benefits for our society. The benefits range from economic and safety benefits due to the reduction of the number of traffic accidents, to environmental gains due to reduced traffic congestion. However, the utopian future of self-driving vehicles is yet to come. To this end, we propose machine learning methods to address three pivotal aspects of autonomous driving: visual privacy, 3D depth perception, and trajectory prediction modeling. We begin by exploring the crucial issue of visual privacy within person-aware visual systems. We propose the use of depth-information to protect privacy in person-aware visual systems while preserving important foreground subjects and scene structures. We aim to preserve the identity of foreground subjects while hiding superfluous details in the background that may contain sensitive information. In particular, for an input color and depth image pair, we first create a sensitivity map which favors background regions (where privacy should be preserved) and low depth-gradient pixels (which often relates a lot to scene structure but little to identity). We then combine this per-pixel sensitivity map with an inhomogeneous image obscuration process for privacy protection. We tested the proposed method using data involving different scenarios including various illumination conditions, various number of subjects, different context, etc. The experiments demonstrate the quality of preserving the identity of humans and edges obtained from the depth information while obscuring privacy intrusive information in the background. Next, we focus on the label layout problem: AR technologies can overlay virtual annotations directly onto the real-world view of a self-driving vehicle (SDV). Autonomous vehicles operate in dynamic environments, due to the complexity of the traffic scene and the interactions between the participating agents. Overlaying virtual annotations directly onto the real-world view of a SDV, can provide additional context, such as highlighting important information or projecting the future trajectories of other participants. Designing a layout of labels that does not violate domain-specific design requirements, while at the same time satisfying aesthetic and functional principles of good design, can be a daunting task even for skilled visual designers. Presenting the annotations in 3D object space instead of projection space, allows for the preservation of spatial and depth cues. This results in stable layouts in dynamic environments, since the annotations are anchored in 3D space. In this domain, we make two major contributions. First, we propose a technique for managing the layout and rendering of annotations in Virtual/Augmented Reality scenarios by manipulating the annotations directly in 3D space. For this, we make use of Artificial Potential Fields and use 3D geometric constraints to adapt them in 3D space. Second, we introduce PartLabeling: an open source platform in the form of a web application that acts as a much-needed generic framework allowing to easily add labeling algorithms and 3D models. This serves as a catalyst for researchers in this field to make their algorithms and implementations publicly available, as well as ensure research reproducibility. The PartLabeling framework relies on a dataset that we generate as a subset of the original PartNet dataset consisting of models suitable for the label management task. The dataset consists of 1,000 3D models with part annotations. Finally, we focus on the trajectory prediction task in the context of autonomous driving. Predicting the trajectories of multiple participating agents in the context of autonomous driving is a challenging problem due to the complexity of the traffic scene and the interactions between the agents. Autonomous vehicles need to effectively anticipate the behavior of other movingparticipants in the traffic scene (human pedestrians, cyclists, animals, other moving vehicles). The task of modeling human driver behavior, as well as the interactions between the traffic participants must be addressed to enable a safe and optimized autonomous vehicle systems. There are many factors that traffic participants take into consideration in order to safely interact with other traffic participants. Human drivers have sophisticated interaction strategies that come naturally to them. Given the highly interactive nature of traffic scenarios, representing the interactions between multiple participating agents in a traffic scene in the form of a graph structure is a natural conclusion. In order to leverage the influences between multiple agents in a traffic scene, we structure the scene as a graph whose nodes represent the traffic participants. The node features are each agent’s surrounding context encoded as a raster image. For this purpose, we leveragel R-GCN (Relational Graph-Convolutional Netowrks). Then, we propose a novel Cross-Modal Attention Network (CMAN) to encourage interactions between two modalities: 1) the latent features of an ego-agent’s raster image and 2) the latent features of the surrounding agents’ influences on the ego-agent, in a manner that allows these two modalities to complement each other. / Computer and Information Science

There is a need for a fuel-optimal required time of arrival (RTA) mode for aircraft flight management systems capable of enabling controlled time of arrival functionality in the presence of wind speed forecast uncertainty. A computationally tractable two-stage stochastic algorithm utilizing a data-driven, location-specific forecast uncertainty model to generate forecast uncertainty scenarios is proposed as a solution. Three years of Aircraft Communications Addressing and Reporting Systems (ACARS) wind speed reports are used in conjunction with corresponding wind speed forecasts from the Rapid Update Cycle (RUC) forecast product to construct an inhomogeneous Markov model quantifying forecast uncertainty characteristics along specific route through the national airspace system. The forecast uncertainty modeling methodology addresses previously unanswered questions regarding the regional uncertainty characteristics of the RUC model, and realizations of the model demonstrate a clear tendency of the RUC product to be positively biased along routes following the normal contours of the jet stream. A two-stage stochastic algorithm is then developed to calculate the fuel optimal stage one cruise speed given a required time of arrival at a destination waypoint and wind forecast uncertainty scenarios generated using the inhomogeneous Markov model. The algorithm utilizes a quadratic approximation of aircraft fuel flow rate as a function of cruising Mach number to quickly search for the fuel-minimum stage one cruise speed while keeping computational footprint small and ensuring RTA adherence. Compared to standard approaches to the problem utilizing large scale linear programming approximations, the algorithm performs significantly better from a computational complexity standpoint, providing solutions in fractional power time while maintaining computational tractability in on-board systems.

Cette thèse s’inscrit dans le développement et la conception de fonctions d’aide à la conduite pour les véhicules autonomes de niveau 3 et plus en milieu urbain ou péri urbain. Du fait d’un environnement plus complexe et de trajectoires possibles plus nombreuses et sinueuses, les algorithmes des véhicules autonomes développés pour l’autoroute ne sont pas adaptés pour le milieu urbain. L’objectif de la thèse est de mettre à disposition des méthodes et des réalisations pour permettre au véhicule autonome d’évoluer en milieu urbain. Cette thèse se focalise sur la proposition de solutions pour améliorer le guidage latéral des véhicules autonomes en milieu urbain à travers l’étude de la planification de trajectoire en situation complexe, l’analyse du comportement des usagers et l’amélioration du suivi de ces trajectoires complexes à faibles vitesses. Les solutions proposées doivent fonctionner en temps réel dans les calculateurs des prototypes pour pouvoir ensuite être appliquées sur route ouverte. L’apport de cette thèse est donc autant théorique que pratique.Après une synthèse des fonctions d’aide à la conduite présentes à bord des véhicules et une présentation des moyens d’essais mis à disposition pour la validation des algorithmes proposés, une analyse complète de la dynamique latérale est effectuée dans les domaines temporel et fréquentiel. Cette analyse permet alors la mise en place d’observateurs de la dynamique latérale pour estimer des signaux nécessaires aux fonctions de guidage latéral et dont les grandeurs ne sont pas toujours mesurables, fortement dégradées ou bruitées. La régulation latérale du véhicule autonome se base sur les conclusions apportées par l’analyse de cette dynamique pour proposer une solution de type multirégulateur capable de générer une consigne en angle volant pour suivre une trajectoire latérale quelle que soit la vitesse. La solution est validée tant en simulation que sur prototype pour plusieurs vitesses sur des trajectoires de changement de voie. La suite de la thèse s’intéresse à la génération d’une trajectoire en milieu urbain tenant compte non seulement de l’infrastructure complexe (intersection/rond-point) mais également des comportements des véhicules autour. C’est pourquoi, une analyse des véhicules de l’environnement est menée afin de déterminer leur comportement et leur trajectoire. Cette analyse est essentielle pour la méthode de génération de trajectoire développée dans cette thèse. Cette méthode, basée sur l’algorithme A* et enrichie pour respecter les contraintes géométriques et dynamiques du véhicule, se focalise d’abord dans un environnement statique complexe de type parking ou rond-point. Des points de passage sont intégrés à la méthode afin de générer des trajectoires conformes au code de la route et d’améliorer le temps de calcul. La méthode est ensuite adaptée pour un environnement dynamique où le véhicule est alors capable, sur une route à double sens de circulation, de dépasser un véhicule avec un véhicule arrivant en sens inverse. / This thesis is about the design of driving assistance systems for level 3 urban automated driving. Because of a more complex of the environment and a larger set of possible trajectories, the algorithms of highway automated driving are not adapted to urban environment. This thesis objective is to provide methods and algorithms to enable the vehicle to perform automated driving in urban scenarios, focusing on the vehicle lateral guidance and on the path planning. The proposed solutions operate in real-time on board of the automated vehicle prototypes. The contribution of this thesis is as theoretical as practical.After a synthesis of the driving assistance systems available on current cars and a presentation of the prototypes used for the validation of the algorithms developed in the thesis, a complete analysis of the vehicle lateral dynamics is carried out in time and frequency domains. This analysis enables the design of observers of the lateral dynamics in order not only to estimate signals required for the lateral guidance functions but also to increase reliability of available measurements. Based on the conclusions from the analysis of lateral dynamic, a multi-controller solution has been proposed. It enables the computation of a steering wheel input to follow a trajectory at any longitudinal speed. The solution is validated in simulation and on real road traffic for lane change scenarios. Another contribution consist in an analysis on the other vehicles of the environment is conducted in order to identify their behaviors and which maneuver there are performing. This analysis is essential for the path planning function developed in the thesis. This method, based on the A* algorithm and extended to respect geometric and dynamic constraints, firstly focuses on static environment such as a parking lot. Waypoints are added to the method in order to compute trajectories compatible with traffic regulation and improve the computation time. The method is then adapted for dynamic environment where, in the end, the vehicle is able to perform overtaking manoeuvers in a complex environment.

Occlusion is a frequently occuring challenge in vision systems for autonomous driving. The density of objects in the field-of-view of the vehicle may be so high that some objects are only visible intermittently. It is therefore beneficial to investigate ways to predict the paths of objects under occlusion. In this thesis, we investigate whether trajectory prediction methods can be used to solve the occlusion prediction problem. We investigate two different types of approaches, one based on motion models, and one based on machine learning models. Furthermore, we investigate whether these two approaches can be fused to produce an even more reliable model. We evaluate our models on a pedestrian trajectory prediction dataset, an autonomous driving dataset, and a subset of the autonomous driving dataset that only includes validation examples of occlusion. The comparison of our different approaches shows that pure motion model-based methods perform the worst out of the three. On the other hand, machine learning-based models perform better, yet they require additional computing resources for training. Finally, the fused method performs the best on both the driving dataset and the occlusion data. Our results also indicate that trajectory prediction methods, both motion model-based and learning-based ones, can indeed accurately predict the path of occluded objects up to at least 3 seconds in the autonomous driving scenario.

Les travaux présentés dans ce mémoire s’inscrivent dans le cadre duprojet PANsafer (Vers un Passage A Niveau plus sûr), lauréat de l’appel ANR-VTT2008. Ce projet est labellisé par les deux pôles de compétitivité i-Trans et Véhiculedu Futur. Le travail de la thèse est mené conjointement par le laboratoire IRTESSETde l’UTBM et le laboratoire LEOST de l’IFSTTAR.L’objectif de cette thèse est de développer un système de perception permettantl’interprétation de scénarios dans l’environnement d’un passage à niveau. Il s’agitd’évaluer des situations potentiellement dangereuses par l’analyse spatio-temporelledes objets présents autour du passage à niveau.Pour atteindre cet objectif, le travail est décomposé en trois étapes principales. Lapremière étape est consacrée à la mise en place d’une architecture spatiale des capteursvidéo permettant de couvrir de manière optimale l’environnement du passageà niveau. Cette étape est mise en oeuvre dans le cadre du développement d’unsimulateur d’aide à la sécurité aux passages à niveau en utilisant un système deperception multi-vues. Dans ce cadre, nous avons proposé une méthode d’optimisationpermettant de déterminer automatiquement la position et l’orientation descaméras par rapport à l’environnement à percevoir.La deuxième étape consisteà développer une méthode robuste de suivi d’objets enmouvement à partir d’une séquence d’images. Dans un premier temps, nous avonsproposé une technique permettant la détection et la séparation des objets. Le processusde suivi est ensuite mis en oeuvre par le calcul et la rectification du flotoptique grâce respectivement à un modèle gaussien et un modèle de filtre de Kalman.La dernière étape est destinée à l’analyse des trajectoires 2D reconstruites parl’étape précédente pour l’interprétation de scénarios. Cette analyse commence parune modélisation markovienne des trajectoires 2D. Un système de décision à basede théorie de l’évidence est ensuite proposé pour l’évaluation de scénarios, aprèsavoir modélisé les sources de danger.L’approche proposée a été testée et évaluée avec des données issues de campagnesexpérimentales effectuées sur site réel d’un passage à niveau mis à disposition parRFF. / The main objective of this thesis is to develop a system for monitoringthe close environment of a level crossing. It aims to develop a perception systemallowing the detection and the evaluation of dangerous situations around a levelcrossing.To achieve this goal, the overall problem of this work has been broken down intothree main stages. In the first stage, we propose a method for optimizing automaticallythe location of video sensors in order to cover optimally a level crossingenvironment. This stage addresses the problem of cameras positioning and orientationin order to view optimally monitored scenes.The second stage aims to implement a method for objects tracking within a surveillancezone. It consists first on developing robust algorithms for detecting and separatingmoving objects around level crossing. The second part of this stage consistsin performing object tracking using a Gaussian propagation optical flow based modeland Kalman filtering.On the basis of the previous steps, the last stage is concerned to present a newmodel to evaluate and recognize potential dangerous situations in a level crossingenvironment. This danger evaluation method is built using Hidden Markov Modeland credibility model.Finally, synthetics and real data are used to test the effectiveness and the robustnessof the proposed algorithms and the whole approach by considering various scenarioswithin several situations.This work is developed within the framework of PANsafer project (Towards a saferlevel crossing), supported by the ANR-VTT program (2008) of the French NationalAgency of Research. This project is also labelled by Pôles de compétitivité "i-Trans"and "Véhicule du Futur". All the work, presented in this thesis, has been conductedjointly within IRTES-SET laboratory from UTBM and LEOST laboratory fromIFSTTAR.