Vehicle recognition and trajectory tracking plays an integral role in many aspects of Intelligent Transportation Systems (ITS) applications; from behavioral modeling and car-following analyses to congestion prevention, crash prediction, dynamic signal timing, and active traffic management. This dissertation aims to improve the tasks of multi-object detection and tracking (MOT) as it pertains to urban traffic by utilizing the domain knowledge of traffic flow then utilize this improvement for applications in real-time traffic performance assessment, safety evaluation, and driver behavior modeling. First, the author proposes an ad-hoc framework for real-time turn count and trajectory reconstruction for vehicles passing through urban intersections. This framework introduces the concept of virtual traffic lanes representing the eight standard National Electrical Manufacturers Association (NEMA) movements within an intersection as spatio-temporal clusters utilized for movement classification and vehicle re-identification. The proposed framework runs as an additional layer to any multi-object tracker with minimal additional computation. The results obtained for a case study and on the AI City benchmark dataset indicate the high ability of the proposed framework in obtaining reliable turn count, speed estimates, and efficiently resolving the vehicle identity switches which occur within the intersection due to detection errors and occlusion. The author then proposes the utilization of the high accuracy and granularity trajectories obtained from video inference to develop a real-time safety-based driver behavior model, which managed to effectively capture the observed driving behavior in the site of study. Finally, the developed model was implemented as an external driver model in VISSIM and managed to reproduce the observed behavior and safety conflicts in simulation, providing an effective decision-support tool to identify appropriate safety interventions that would mitigate those conflicts. The work presented in this dissertation provides an efficient end-to-end framework and blueprint for trajectory extraction from road-side traffic video data, driver behavior modeling, and their applications for real-time traffic performance and safety assessment, as well as improved modeling of safety interventions via microscopic simulation. / Doctor of Philosophy / Traffic crashes are one of the leading causes of death in the world, averaging over 3,000 deaths per day according to the World Health Organization. In the United States alone, there are around 40,000 traffic fatalities annually. Approximately, 21.5% of all traffic fatalities occur due to intersection-related crashes. Intelligent Transportation Systems (ITS) is a field of traffic engineering that aims to transform traffic systems to make safer, more coordinated, and 'smarter' use of transport networks. Vehicle recognition and trajectory tracking, the process of identifying a specific vehicle's movement through time and space, plays an integral role in many aspects of ITS applications; from understanding how people drive and modeling that behavior, to congestion prevention, on-board crash avoidance systems, adaptive signal timing, and active traffic management. This dissertation aims to bridge the gaps in the application of ITS, computer vision, and traffic flow theory and create tools that will aid in evaluating and proactively addressing traffic safety concerns at urban intersections. The author presents an efficient, real-time framework for extracting reliable vehicle trajectories from roadside cameras, then proposes a safety-based driving behavior model that succeeds in capturing the observed driving behavior. This work is concluded by implementing this model in simulation software to replicate the existing safety concerns for an area of study, allowing practitioners to accurately model the existing safety conflicts and evaluate the different operation and safety interventions that would best mitigate them to proactively prevent crashes.

In this dissertation, new algorithms that address three traffic problems of major importance are developed. First automatic identification and prediction algorithms are developed to identify and predict the occurrence of traffic congestion. The identification algorithms concoct a model to identify speed thresholds by exploiting historical spatiotemporal speed matrices. We employ the speed model to define a cutoff speed separating free-flow from congested traffic. We further enhance our algorithm by utilizing weather and visibility data. To our knowledge, we are the first to include weather and visibility variables in formulating an automatic congestion identification model. We also approach the congestion prediction problem by adopting an algorithm which employs Adaptive Boosting machine learning classifiers again something novel that has not been done previously. The algorithm is promising where it resulted in a true positive rate slightly higher than 0.99 and false positive rate less than 0.001. We next address the issue of travel time modeling. We propose algorithms to model travel time using various machine learning and statistical learning techniques. We obtain travel time models by employing the historical spatiotemporal speed matrices in conjunction with our algorithms. The algorithms yield pertinent information regarding travel time reliability and prediction of travel times. Our proposed algorithms give better predictions compared to the state of practice algorithms. Finally we consider driver safety at signalized intersections and uncontrolled intersections in a connected vehicles environment. For signalized intersections, we exploit datasets collected from four controlled experiments to model the stop-run behavior of the driver at the onset of the yellow indicator for various roadway surface conditions and multiple vehicle types. We further propose a new variable (predictor) related to driver aggressiveness which we estimate by monitoring how drivers respond to yellow indications. The performance of the stop-run models shows improvements after adding the new aggressiveness predictor. The proposed models are practical and easy to implement in advanced driver assistance systems. For uncontrolled intersections, we present a game theory based algorithm that models the intersection as a chicken game to solve the conflicts between vehicles crossing the intersection. The simulation results show a 49% saving in travel time on average relative to a stop control when the vehicles obey the Nash equilibrium of the game. / Ph. D.

Au cours des dernières décennies, les constructeurs automobiles ont constamment introduit des innovations technologiques visant à rendre les véhicules plus sûrs. Le niveau de sophistication de ces systèmes avancés d’aide à la conduite s’est accru parallèlement aux progrès de la technologie des capteurs et de la puissance informatique intégrée. Plus récemment, une grande partie de la recherche effectuée par l'industrie et les institutions s'est concentrée sur l'obtention d'une conduite entièrement automatisée. Les avantages sociétaux potentiels de cette technologie sont nombreux, notamment des routes plus sûres, des flux de trafic améliorés et une mobilité accrue pour les personnes âgées et les handicapés. Toutefois, avant que les véhicules autonomes puissent être commercialisés, ils doivent pouvoir partager la route en toute sécurité avec d’autres véhicules conduits par des conducteurs humains. En d'autres termes, ils doivent pouvoir déduire l'état et les intentions du trafic environnant à partir des données brutes fournies par divers capteurs embarqués, et les utiliser afin de pouvoir prendre les bonnes décisions de conduite sécurisée. Malgré la complexité apparente de cette tâche, les conducteurs humains ont la capacité de prédire correctement l’évolution du trafic environnant dans la plupart des situations. Cette capacité de prédiction est rendu plus simple grâce aux règles imposées par le code de la route qui limitent le nombre d’hypothèses; elle repose aussi sur l’expérience du conducteur en matière d’évaluation et de réduction du risque. L'absence de cette capacité à comprendre naturellement une scène de trafic constitue peut-être, le principal défi qui freine le déploiement à grande échelle de véhicules véritablement autonomes sur les routes.Dans cette thèse, nous abordons les problèmes de modélisation du comportement du conducteur, d'inférence sur le comportement des autres véhicules, et de la prise de décision pour la navigation sûre. En premier lieu, nous modélisons automatiquement le comportement d'un conducteur générique à partir de données de conduite démontrées, évitant ainsi le réglage manuel traditionnel des paramètres du modèle. Ce modèle codant les préférences d’un conducteur par rapport au réseau routier (par exemple, voie ou vitesse préférées) et aux autres usagers de la route (par exemple, distance préférée au véhicule de devant). Deuxièmement, nous décrivons une méthode qui utilise le modèle appris pour prédire la séquence des actions à long terme de tout conducteur dans une scène de trafic. Cette méthode de prédiction suppose que tous les acteurs du trafic se comportent de manière aversive au risque, et donc ne peut pas prévoir les manœuvres dangereux ou les accidents. Pour pouvoir traiter de tels cas, nous proposons un modèle probabiliste plus sophistiqué, qui estime l'état et les intentions du trafic environnant en combinant la prédiction basée sur le modèle avec les preuves dynamiques fournies par les capteurs. Le modèle proposé imite ainsi en quelque sorte le processus de raisonnement des humains. Nous humains, savons ce qu’un véhicule est susceptible de faire compte tenu de la situation (ceci est donné par le modèle), mais nous surveillerons sa dynamique pour en détecter les écarts par rapport au comportement attendu. En pratique, la combinaison de ces deux sources d’informations se traduit par une robustesse accrue des estimations de l’intention par rapport aux approches reposant uniquement sur des preuves dynamiques. En dernière partie, les deux modèles présentés (comportemental et prédictif) sont intégrés dans le cadre d´une approche décisionnel probabiliste. Les méthodes proposées se sont vues évalués avec des données réelles collectées avec un véhicule instrumenté, attestant de leur efficacité dans le cadre de la conduite autonome sur autoroute. Bien que centré sur les autoroutes, ce travail pourrait être facilement adapté pour gérer des scénarios de trafic alternatifs. / During the past few decades automakers have consistently introduced technological innovations aimed to make road vehicles safer. The level of sophistication of these advanced driver assistance systems has increased parallel to developments in sensor technology and embedded computing power. More recently, a lot of the research made both by industry and institutions has concentrated on achieving fully automated driving. The potential societal benefits of this technology are numerous, including safer roads, improved traffic flows, increased mobility for the elderly and the disabled, and optimized human productivity. However, before autonomous vehicles can be commercialized they should be able to safely share the road with human drivers. In other words, they should be capable of inferring the state and intentions of surrounding traffic from the raw data provided by a variety of onboard sensors, and to use this information to make safe navigation decisions. Moreover, in order to truly navigate safely they should also consider potential obstacles not observed by the sensors (such as occluded vehicles or pedestrians). Despite the apparent complexity of the task, humans are extremely good at predicting the development of traffic situations. After all, the actions of any traffic participant are constrained by the road network, by the traffic rules, and by a risk-aversive common sense. The lack of this ability to naturally understand a traffic scene constitutes perhaps the major challenge holding back the large-scale deployment of truly autonomous vehicles in the roads.In this thesis, we address the full pipeline from driver behavior modeling and inference to decision-making for navigation. In the first place, we model the behavior of a generic driver automatically from demonstrated driving data, avoiding thus the traditional hand-tuning of the model parameters. This model encodes the preferences of a driver with respect to the road network (e.g. preferred lane or speed) and also with respect to other road users (e.g. preferred distance to the leading vehicle). Secondly, we describe a method that exploits the learned model to predict the future sequence of actions of any driver in a traffic scene up to the distant future. This model-based prediction method assumes that all traffic participants behave in a risk-aware manner and can therefore fail to predict dangerous maneuvers or accidents. To be able to handle such cases, we propose a more sophisticated probabilistic model that estimates the state and intentions of surrounding traffic by combining the model-based prediction with the dynamic evidence provided by the sensors. In a way, the proposed model mimics the reasoning process of human drivers: we know what a given vehicle is likely to do given the situation (this is given by the model), but we closely monitor its dynamics to detect deviations from the expected behavior. In practice, combining both sources of information results in an increased robustness of the intention estimates in comparison with approaches relying only on dynamic evidence. Finally, the learned driver behavioral model and the prediction model are integrated within a probabilistic decision-making framework. The proposed methods are validated with real-world data collected with an instrumented vehicle. Although focused on highway environments, this work could be easily adapted to handle alternative traffic scenarios.

La réalité augmentée (Augmented Reality ou AR) peut potentiellement changer significativement l’expérience utilisateur. Au contraire les applications sur Smartphone ou tablette, les technologies d’affichage tête haute (Head Up Display ouHUD) aujourd’hui sont capables de projeter localement sur une zone du pare-brise ou globalement sur tout le pare-brise. Le conducteur peut alors percevoir l’information directement dans son champ de vision. Ce ne sont pas que les informations basiques comme vitesse ou navigation, le système peut aussi afficher des aides, des indicateurs qui guident l’attention du conducteur vers les dangers possibles. Il existe alors un chalenge scientifique qui est de concevoir des visualisations d’interactions qui s’adaptent en fonction de l’observation de la scène mais aussi en fonction de l’observation du conducteur. Dans le contexte des systèmes d’alerte de collision avec les piétons (Pedestrian Collision Warning System ou PCWS), l’efficacité de la détection du piéton a atteint un niveau élevé grâce à la technologie de vision. Pourtant, les systèmes d’alerte ne s’adaptent pas au conducteur et à la situation, ils deviennent alors une source de distraction et sont souvent négligés par le conducteur. Pour ces raisons, ce travail de thèse consiste à proposer un nouveau concept de PCWS avec l’AR (nommé the AR-PCW system). Premièrement, nous nous concentrons sur l’étude de la conscience de la situation (Situation Awareness ou SA) du conducteur lorsqu’il y a un piéton présent devant le véhicule. Nous proposons une approche expérimentale pour collecter les données qui représentent l’attention du conducteur vis-à-vis du piéton (Driver Awareness of Pedestrian ou DAP) et l’inattention du conducteur vis-à-vis de celui-ci (Driver Unawareness of Pedestrian ou DUP). Ensuite, les algorithmes basées sur les charactéristiques, les modèles d’apprentissage basés sur les modèles discriminants (ex, Support Vector Machine ou SVM) ou génératifs (Hidden Markov Model ou HMM) sont proposés pour estimer le DUP et le DAP. La décision de notre AR-PCW system est effectivement basée sur ce modèle. Deuxièmement, nous proposons les aides ARs pour améliorer le DAP après une étude de l’état de l’art sur les ARs dans le contexte de la conduite automobile. La boite englobante autour du piéton et le panneau d’alerte de danger sont utilisés. Finalement, nous étudions expérimentalement notre système AR-PCW en analysant les effets des aides AR sur le conducteur. Un simulateur de conduite est utilisé et la simulation d’une zone HUD dans la scène virtuelle sont proposés. Vingt-cinq conducteurs de 2 ans de permis de conduite ont participé à l’expérimentation. Les situations ambigües sont créées dans le scénario de conduite afin d’analyser le DAP. Le conducteur doit suivre un véhicule et les piétons apparaissent à différents moments. L’effet des aides AR sur le conducteur est analysé à travers ses performances à réaliser la tâche de poursuite et ses réactions qui engendrent le DAP. Les résultats objectifs et subjectifs montrent que les aides AR sont capables d’améliorer le DAP défini en trois niveaux : perception, vigilance et anticipation. Ce travail de thèse a été financé sur une bourse ministère et a été réalisé dans le cadre des projets FUI18 SERA et Labex MS2T qui sont financé par le Gouvernement Français, à travers le programme « Investissement pour l’avenir » géré par le ANR (Référence ANR-11-IDEX-0004-02). / Augmented reality (AR) can potentially change the driver’s user experience in significant ways. In contrast of the AR applications on smart phones or tablets, the Head-Up-Displays (HUD) technology based on a part or all wind-shield project information directly into the field of vision, so the driver does not have to look down at the instrument which maybe causes to the time-critical event misses. Until now, the HUD designers try to show not only basic information such as speed and navigation commands but also the aids and the annotations that help the driver to see potential dangers. However, what should be displayed and when it has to be displayed are still always the questions in critical driving context. In another context, the pedestrian safety becomes a serious society problem when half of traffic accidents around the world are among pedestrians and cyclists. Several advanced Pedestrian Collision Warning Systems (PCWS) have been proposed to detect pedestrians using the on-board sensors and to inform the driver of their presences. However, most of these systems do not adapt to the driver’s state and can become extremely distracting and annoying when they detect pedestrian. For those reasons, this thesis focuses on proposing a new concept for the PCWS using AR (so called the AR-PCW system). Firstly, for the «When» question, the display decision has to take into account the driver’s states and the critical situations. Therefore, we investigate the modelisation of the driver’s awareness of a pedestrian (DAP) and the driver’s unawareness of a pedestrian (DUP). In order to do that, an experimental approach is proposed to observe and to collect the driving data that present the DAP and the DUP. Then, the feature-based algorithms, the data-driven models based on the discriminative models (e.g. Support Vector Machine) or the generative models (e.g. Hidden Markov Model) are proposed to recognize the DAP and the DUP. Secondly, for the «What» question, our proposition is inspired by the state-of-the-art on the AR in the driving context. The dynamic bounding-box surrounding the pedestrian and the static danger panel are used as the visual aids. Finally, in this thesis, we study experimentally the benefits and the costs of the proposed AR-PCW system and the effects of the aids on the driver. A fixed-based driving simulator is used. A limited display zone on screen is proposed to simulate the HUD. Twenty five healthy middle-aged licensed drivers in ambiguous driving scenarios are explored. Indeed, the heading-car following is used as the main driving task whereas twenty three pedestrians appear in the circuit at different moment and with different behaviors. The car-follow task performance and the awareness of pedestrian are then accessed through the driver actions. The objective results as well as the subjective results show that the visual aids can enhance the driver’s awareness of a pedestrian which is defined with three levels: perception, vigilance and anticipation. This work has been funded by a Ministry scholarship and was carried out in the framework of the FUI18 SERA project, and the Labex MS2T which is funded by the French Government, through the program ”Investments for the future” managed by the National Agency for Research (Reference ANR-11-IDEX-0004-02).