In this thesis, we present a vehicle detection and tracking system based on image processing and pattern recognition of single camera image sequences. Both software design and hardware implementation are considered. In the hypothesis generation (HG) step and the hypothesis verification (HV) step, we use the shadow detection technique combined with the proposed constrained vehicle width/distance ratio to eliminate unreasonable hypotheses. Furthermore, we use SVM classifier, a popular machine learning technique, to verify the generated hypothesis more precisely. In the vehicle tracking step, we limit vehicle tracking duration and periodic vehicle detection mechanisms. These tracking methods alleviate our driver-assistant system from executing complex operations of vehicle detection repeatedly and thus increase system performance without sacrificing too much in case of tracking wrong objects. Based on the the profiling of the software execution time, we implement by hardware the most critical part, the preprocessing of intensity conversion and edge detection. The complete software/hardware embedded system is realized in a FPGA prototype board, so that performance of whole system could achieve real-time processing without too much hardware cost.

Nowadays, autonomous driving revolution is getting closer to reality. To achieve the Autonomous driving the first step is to develop the Advanced Driver Assistance System (ADAS). Driver-assistance systems are one of the fastest-growing segments in automotive electronics since already there are many forms of ADAS available. To investigate state of art of development of ADAS towards Autonomous Driving, we develop Systematic Mapping Study (SMS). SMS methodology is used to collect, classify, and analyze the relevant publications. A classification is introduced based on the developments carried out in ADAS towards Autonomous driving. According to SMS methodology, we identified 894 relevant publications about ADAS and its developmental journey toward Autonomous Driving completed from 2012 to 2016. We classify the area of our research under three classifications: technical classifications, research types and research contributions. The related publications are classified under thirty-three technical classifications. This thesis sheds light on a better understanding of the achievements and shortcomings in this area. By evaluating collected results, we answer our seven research questions. The result specifies that most of the publications belong to the Models and Solution Proposal from the research type and contribution. The least number of the publications belong to the Automated…Autonomous driving from the technical classification which indicated the lack of publications in this area.

Cette thèse s’inscrit dans le développement et la conception de fonctions d’aide à la conduite pour les véhicules autonomes de niveau 3 et plus en milieu urbain ou péri urbain. Du fait d’un environnement plus complexe et de trajectoires possibles plus nombreuses et sinueuses, les algorithmes des véhicules autonomes développés pour l’autoroute ne sont pas adaptés pour le milieu urbain. L’objectif de la thèse est de mettre à disposition des méthodes et des réalisations pour permettre au véhicule autonome d’évoluer en milieu urbain. Cette thèse se focalise sur la proposition de solutions pour améliorer le guidage latéral des véhicules autonomes en milieu urbain à travers l’étude de la planification de trajectoire en situation complexe, l’analyse du comportement des usagers et l’amélioration du suivi de ces trajectoires complexes à faibles vitesses. Les solutions proposées doivent fonctionner en temps réel dans les calculateurs des prototypes pour pouvoir ensuite être appliquées sur route ouverte. L’apport de cette thèse est donc autant théorique que pratique.Après une synthèse des fonctions d’aide à la conduite présentes à bord des véhicules et une présentation des moyens d’essais mis à disposition pour la validation des algorithmes proposés, une analyse complète de la dynamique latérale est effectuée dans les domaines temporel et fréquentiel. Cette analyse permet alors la mise en place d’observateurs de la dynamique latérale pour estimer des signaux nécessaires aux fonctions de guidage latéral et dont les grandeurs ne sont pas toujours mesurables, fortement dégradées ou bruitées. La régulation latérale du véhicule autonome se base sur les conclusions apportées par l’analyse de cette dynamique pour proposer une solution de type multirégulateur capable de générer une consigne en angle volant pour suivre une trajectoire latérale quelle que soit la vitesse. La solution est validée tant en simulation que sur prototype pour plusieurs vitesses sur des trajectoires de changement de voie. La suite de la thèse s’intéresse à la génération d’une trajectoire en milieu urbain tenant compte non seulement de l’infrastructure complexe (intersection/rond-point) mais également des comportements des véhicules autour. C’est pourquoi, une analyse des véhicules de l’environnement est menée afin de déterminer leur comportement et leur trajectoire. Cette analyse est essentielle pour la méthode de génération de trajectoire développée dans cette thèse. Cette méthode, basée sur l’algorithme A* et enrichie pour respecter les contraintes géométriques et dynamiques du véhicule, se focalise d’abord dans un environnement statique complexe de type parking ou rond-point. Des points de passage sont intégrés à la méthode afin de générer des trajectoires conformes au code de la route et d’améliorer le temps de calcul. La méthode est ensuite adaptée pour un environnement dynamique où le véhicule est alors capable, sur une route à double sens de circulation, de dépasser un véhicule avec un véhicule arrivant en sens inverse. / This thesis is about the design of driving assistance systems for level 3 urban automated driving. Because of a more complex of the environment and a larger set of possible trajectories, the algorithms of highway automated driving are not adapted to urban environment. This thesis objective is to provide methods and algorithms to enable the vehicle to perform automated driving in urban scenarios, focusing on the vehicle lateral guidance and on the path planning. The proposed solutions operate in real-time on board of the automated vehicle prototypes. The contribution of this thesis is as theoretical as practical.After a synthesis of the driving assistance systems available on current cars and a presentation of the prototypes used for the validation of the algorithms developed in the thesis, a complete analysis of the vehicle lateral dynamics is carried out in time and frequency domains. This analysis enables the design of observers of the lateral dynamics in order not only to estimate signals required for the lateral guidance functions but also to increase reliability of available measurements. Based on the conclusions from the analysis of lateral dynamic, a multi-controller solution has been proposed. It enables the computation of a steering wheel input to follow a trajectory at any longitudinal speed. The solution is validated in simulation and on real road traffic for lane change scenarios. Another contribution consist in an analysis on the other vehicles of the environment is conducted in order to identify their behaviors and which maneuver there are performing. This analysis is essential for the path planning function developed in the thesis. This method, based on the A* algorithm and extended to respect geometric and dynamic constraints, firstly focuses on static environment such as a parking lot. Waypoints are added to the method in order to compute trajectories compatible with traffic regulation and improve the computation time. The method is then adapted for dynamic environment where, in the end, the vehicle is able to perform overtaking manoeuvers in a complex environment.

Driving safety is the major issue not only for the drivers, but also for the government. The happening rate of traffic accidents is the critical benchmark of traffic improvement for the Ministry of Communications. Even the government officials constantly urge the drivers not to drunk driving, or over speed driving, the happening rate still cannot be decreased largely. Most of accidents are made by the careless or ignorant driving habits. With the evolution of driving safety technology, the driving assistance system helps drivers to avoid the collision and lower the happening rate of accidents. Among these driving safety technologies, collision detection system is well-known one. It can detect not only any object around the vehicle, but also notify the drivers to stop passively or stop the vehicle actively. One major function of the collision detection system is used to eliminate the blind spots for the drivers. Some blind spots are generated by the rigid structure of vehicle which is designed to protect the inside passengers and not allowed to change. Fortunately, with the evolution of Augmented Reality and the lower manufacturing cost of video equipment, Driving Lens is targeted to eliminate the blind spots for the drivers. Currently, there are still some limitations about the existing products such as the around view monitor or rear view monitor. In order to improve the driving safety and enhance the driving experience, Driving Lens will offer different customized solutions for the driver such as 180 degree front view without the blind spots behind the pillars, and these solutions won’t be limited by the specific auto brand or vehicle model.

<div>With more and more videos taken from dash cams on thousands of cars, retrieving these videos and searching for important information is a daunting task. The purpose of this work is to mine some key road and vehicle motion attributes in a large-scale driving video data set for traffic analysis, sensing algorithm development and autonomous driving test benchmarks. Current sensing and control of autonomous cars based on full-view identification makes it difficult to maintain a high-frequency with a fast-moving vehicle, since computation is increasingly used to cope with driving environment changes.</div><div><br></div><div>A big challenge in video data mining is how to deal with huge amounts of data. We use a compact representation called the road profile system to visualize the road environment in long 2D images. It reduces the data from each frame of image to one line, thereby compressing the video clip to the image. This data dimensionality reduction method has several advantages: First, the data size is greatly compressed. The data is compressed from a video to an image, and each frame in the video is compressed into a line. The data size is compressed hundreds of times. While the size and dimensionality of the data has been compressed greatly, the useful information in the driving video is still completely preserved, and motion information is even better represented more intuitively. Because of the data and dimensionality reduction, the identification algorithm computational efficiency is higher than the full-view identification method, and it makes the real-time identification on road is possible. Second, the data is easier to be visualized, because the data is reduced in dimensionality, and the three-dimensional video data is compressed into two-dimensional data, the reduction is more conducive to the visualization and mutual comparison of the data. Third, continuously changing attributes are easier to show and be captured. Due to the more convenient visualization of two-dimensional data, the position, color and size of the same object within a few frames will be easier to compare and capture. At the same time, in many cases, the trouble caused by tracking and matching can be eliminated. Based on the road profile system, there are three tasks in autonomous driving are achieved using the road profile images.</div><div><br></div><div>The first application is road edge detection under different weather and appearance for road following in autonomous driving to capture the road profile image and linearity profile image in the road profile system. This work uses naturalistic driving video data mining to study the appearance of roads, which covers large-scale road data and changes. This work excavated a large number of naturalistic driving video sets to sample the light-sensitive area for color feature distribution. The effective road contour image is extracted from the long-time driving video, thereby greatly reducing the amount of video data. Then, the weather and lighting type can be identified. For each weather and lighting condition obvious features are I identified at the edge of the road to distinguish the road edge. </div><div><br></div><div>The second application is detecting vehicle interactions in driving videos via motion profile images to capture the motion profile image in the road profile system. This work uses visual actions recorded in driving videos taken by a dashboard camera to identify this interaction. The motion profile images of the video are filtered at key locations, thereby reducing the complexity of object detection, depth sensing, target tracking and motion estimation. The purpose of this reduction is for decision making of vehicle actions such as lane changing, vehicle following, and cut-in handling.</div><div><br></div><div>The third application is motion planning based on vehicle interactions and driving video. Taking note of the fact that a car travels in a straight line, we simply identify a few sample lines in the view to constantly scan the road, vehicles, and environment, generating a portion of the entire video data. Without using redundant data processing, we performed semantic segmentation to streaming road profile images. We plan the vehicle's path/motion using the smallest data set possible that contains all necessary information for driving.</div><div><br></div><div>The results are obtained efficiently, and the accuracy is acceptable. The results can be used for driving video mining, traffic analysis, driver behavior understanding, etc.</div>