Localization is one of the fundamental components of autonomous vehicles by enabling tasks as overtaking, lane keeping and self-navigation. Urban canyons and bad weather interfere with the reception of GPS satellite signal which prohibits the exclusive use of such technology for vehicle localization in urban places. Alternatively, map-aided localization methods have been employed to enable position estimation without the dependence on GPS devices. In this solution, the vehicle position is given as the place that best matches the sensor measurement to the environment map. Before building the maps, feature sof the environment must be extracted from sensor measurements. In vehicle localization, curbs and road markings have been extensively employed as mapping features. However, most of the urban mapping methods rely on a street free of obstacles or require repetitive measurements of the same place to avoid occlusions. The construction of an accurate representation of the environment is necessary for a proper match of sensor measurements to the map during localization. To prevent the necessity of a manual process to remove occluding obstacles and unobserved areas, a vehicle localization method that supports maps built from partial observations of the environment is proposed. In this localization system,maps are formed by curb and road markings extracted from multilayer laser sensor measurements. Curb structures are detected even in the presence of vehicles that occlude the roadsides, thanks to the use of robust regression. Road markings detector employs Otsu thresholding to analyze infrared remittance data which makes the method insensitive to illumination. Detected road features are stored in two map representations: occupancy grid map (OGM) and Gaussian process occupancy map (GPOM). The first approach is a popular map structure that represents the environment through fine-grained grids. The second approach is a continuous representation that can estimate the occupancy of unseen areas. The Monte Carlo localization (MCL) method was adapted to support the obtained maps of the urban environment. In this sense, vehicle localization was tested in an MCL that supports OGM and an MCL that supports GPOM. Precisely, for MCL based on GPOM, a new measurement likelihood based on multivariate normal probability density function is formulated. Experiments were performed in real urban environments. Maps were built using sparse laser data to verify there ronstruction of non-observed areas. The localization system was evaluated by comparing the results with a high precision GPS device. Results were also compared with localization based on OGM. / No contexto de veículos autônomos, a localização é um dos componentes fundamentais, pois possibilita tarefas como ultrapassagem, direção assistida e navegação autônoma. A presença de edifícios e o mau tempo interferem na recepção do sinal de GPS que consequentemente dificulta o uso de tal tecnologia para a localização de veículos dentro das cidades. Alternativamente, a localização com suporte aos mapas vem sendo empregada para estimar a posição sem a dependência do GPS. Nesta solução, a posição do veículo é dada pela região em que ocorre a melhor correspondência entre o mapa do ambiente e a leitura do sensor. Antes da criação dos mapas, características dos ambientes devem ser extraídas a partir das leituras dos sensores. Dessa forma, guias e sinalizações horizontais têm sido largamente utilizados para o mapeamento. Entretanto, métodos de mapeamento urbano geralmente necessitam de repetidas leituras do mesmo lugar para compensar as oclusões. A construção de representações precisas dos ambientes é essencial para uma adequada associação dos dados dos sensores como mapa durante a localização. De forma a evitar a necessidade de um processo manual para remover obstáculos que causam oclusão e áreas não observadas, propõe-se um método de localização de veículos com suporte aos mapas construídos a partir de observações parciais do ambiente. No sistema de localização proposto, os mapas são construídos a partir de guias e sinalizações horizontais extraídas a partir de leituras de um sensor multicamadas. As guias podem ser detectadas mesmo na presença de veículos que obstruem a percepção das ruas, por meio do uso de regressão robusta. Na detecção de sinalizações horizontais é empregado o método de limiarização por Otsu que analisa dados de reflexão infravermelho, o que torna o método insensível à variação de luminosidade. Dois tipos de mapas são empregados para a representação das guias e das sinalizações horizontais: mapa de grade de ocupação (OGM) e mapa de ocupação por processo Gaussiano (GPOM). O OGM é uma estrutura que representa o ambiente por meio de uma grade reticulada. OGPOM é uma representação contínua que possibilita a estimação de áreas não observadas. O método de localização por Monte Carlo (MCL) foi adaptado para suportar os mapas construídos. Dessa forma, a localização de veículos foi testada em MCL com suporte ao OGM e MCL com suporte ao GPOM. No caso do MCL baseado em GPOM, um novo modelo de verossimilhança baseado em função densidade probabilidade de distribuição multi-normal é proposto. Experimentos foram realizados em ambientes urbanos reais. Mapas do ambiente foram gerados a partir de dados de laser esparsos de forma a verificar a reconstrução de áreas não observadas. O sistema de localização foi avaliado por meio da comparação das posições estimadas comum GPS de alta precisão. Comparou-se também o MCL baseado em OGM com o MCL baseado em GPOM, de forma a verificar qual abordagem apresenta melhores resultados.

Nowadays, satellite-based localization is a well-established technical solution to support several navigation tasks in daily life. Besides the application inside of portable devices, satellite-based positioning is used for in-vehicle navigation systems as well. Moreover, due to its global coverage and the availability of inexpensive receiver hardware it is an appealing technology for numerous applications in the area of Intelligent Transportation Systems (ITSs). However, it has to be admitted that most of the aforementioned examples either rely on modest accuracy requirements or are not sensitive to temporary integrity violations. Although technical concepts of Advanced Driver Assistance Systems (ADASs) based on Global Navigation Satellite Systems (GNSSs) have been successfully demonstrated under open sky conditions, practice reveals that such systems suffer from degraded satellite signal quality when put into urban areas. Thus, the main research objective of this thesis is to provide a reliable vehicle positioning concept which can be used in urban areas without the aforementioned limitations. Therefore, an integrated probabilistic approach which preforms fault detection & exclusion, localization and multi-sensor data fusion within one unified Bayesian framework is proposed. From an algorithmic perspective, the presented concept is based on a probabilistic data association technique with explicit handling of outlier measurements as present in urban areas. By that approach, the accuracy, integrity and availability are improved at the same time, that is, a consistent positioning solution is provided. In addition, a comprehensive and in-depth analysis of typical errors in urban areas within the pseudorange domain is performed. Based on this analysis, probabilistic models are proposed and later on used to facilitate the positioning algorithm. Moreover, the presented concept clearly targets towards mass-market applications based on low-cost receivers and hence aims to replace costly sensors by smart algorithms. The benefits of these theoretical contributions are implemented and demonstrated on the example of a real-time vehicle positioning prototype as used inside of the European research project GAlileo Interactive driviNg (GAIN). This work describes all necessary parts of this system including GNSS signal processing, fault detection and multi-sensor data fusion within one processing chain. Finally, the performance and benefits of the proposed concept are examined and validated both with simulated and comprehensive real-world sensor data from numerous test drives.

Localization is one of the fundamental components of autonomous vehicles by enabling tasks as overtaking, lane keeping and self-navigation. Urban canyons and bad weather interfere with the reception of GPS satellite signal which prohibits the exclusive use of such technology for vehicle localization in urban places. Alternatively, map-aided localization methods have been employed to enable position estimation without the dependence on GPS devices. In this solution, the vehicle position is given as the place that best matches the sensor measurement to the environment map. Before building the maps, feature sof the environment must be extracted from sensor measurements. In vehicle localization, curbs and road markings have been extensively employed as mapping features. However, most of the urban mapping methods rely on a street free of obstacles or require repetitive measurements of the same place to avoid occlusions. The construction of an accurate representation of the environment is necessary for a proper match of sensor measurements to the map during localization. To prevent the necessity of a manual process to remove occluding obstacles and unobserved areas, a vehicle localization method that supports maps built from partial observations of the environment is proposed. In this localization system,maps are formed by curb and road markings extracted from multilayer laser sensor measurements. Curb structures are detected even in the presence of vehicles that occlude the roadsides, thanks to the use of robust regression. Road markings detector employs Otsu thresholding to analyze infrared remittance data which makes the method insensitive to illumination. Detected road features are stored in two map representations: occupancy grid map (OGM) and Gaussian process occupancy map (GPOM). The first approach is a popular map structure that represents the environment through fine-grained grids. The second approach is a continuous representation that can estimate the occupancy of unseen areas. The Monte Carlo localization (MCL) method was adapted to support the obtained maps of the urban environment. In this sense, vehicle localization was tested in an MCL that supports OGM and an MCL that supports GPOM. Precisely, for MCL based on GPOM, a new measurement likelihood based on multivariate normal probability density function is formulated. Experiments were performed in real urban environments. Maps were built using sparse laser data to verify there ronstruction of non-observed areas. The localization system was evaluated by comparing the results with a high precision GPS device. Results were also compared with localization based on OGM. / No contexto de veículos autônomos, a localização é um dos componentes fundamentais, pois possibilita tarefas como ultrapassagem, direção assistida e navegação autônoma. A presença de edifícios e o mau tempo interferem na recepção do sinal de GPS que consequentemente dificulta o uso de tal tecnologia para a localização de veículos dentro das cidades. Alternativamente, a localização com suporte aos mapas vem sendo empregada para estimar a posição sem a dependência do GPS. Nesta solução, a posição do veículo é dada pela região em que ocorre a melhor correspondência entre o mapa do ambiente e a leitura do sensor. Antes da criação dos mapas, características dos ambientes devem ser extraídas a partir das leituras dos sensores. Dessa forma, guias e sinalizações horizontais têm sido largamente utilizados para o mapeamento. Entretanto, métodos de mapeamento urbano geralmente necessitam de repetidas leituras do mesmo lugar para compensar as oclusões. A construção de representações precisas dos ambientes é essencial para uma adequada associação dos dados dos sensores como mapa durante a localização. De forma a evitar a necessidade de um processo manual para remover obstáculos que causam oclusão e áreas não observadas, propõe-se um método de localização de veículos com suporte aos mapas construídos a partir de observações parciais do ambiente. No sistema de localização proposto, os mapas são construídos a partir de guias e sinalizações horizontais extraídas a partir de leituras de um sensor multicamadas. As guias podem ser detectadas mesmo na presença de veículos que obstruem a percepção das ruas, por meio do uso de regressão robusta. Na detecção de sinalizações horizontais é empregado o método de limiarização por Otsu que analisa dados de reflexão infravermelho, o que torna o método insensível à variação de luminosidade. Dois tipos de mapas são empregados para a representação das guias e das sinalizações horizontais: mapa de grade de ocupação (OGM) e mapa de ocupação por processo Gaussiano (GPOM). O OGM é uma estrutura que representa o ambiente por meio de uma grade reticulada. OGPOM é uma representação contínua que possibilita a estimação de áreas não observadas. O método de localização por Monte Carlo (MCL) foi adaptado para suportar os mapas construídos. Dessa forma, a localização de veículos foi testada em MCL com suporte ao OGM e MCL com suporte ao GPOM. No caso do MCL baseado em GPOM, um novo modelo de verossimilhança baseado em função densidade probabilidade de distribuição multi-normal é proposto. Experimentos foram realizados em ambientes urbanos reais. Mapas do ambiente foram gerados a partir de dados de laser esparsos de forma a verificar a reconstrução de áreas não observadas. O sistema de localização foi avaliado por meio da comparação das posições estimadas comum GPS de alta precisão. Comparou-se também o MCL baseado em OGM com o MCL baseado em GPOM, de forma a verificar qual abordagem apresenta melhores resultados.

Abstract: Nowadays, traffic analysis are relying on data collected from various traffic sensors. Among the various traffic surveillance techniques, video surveillance systems are often used for monitoring and characterizing traffic load. In this thesis, we focused on two aspects of traffic analysis without using motion features in low frame-rate videos: Traffic density flow analysis and Vehicle detection and classification. Traffic density flow analysis}: Knowing in real time when the traffic is fluid or when it jams is a key information to help authorities re-route vehicles and reduce congestion. Accurate and timely traffic flow information is strongly needed by individual travelers, the business sectors and government agencies. In this part, we investigated the possibility of monitoring highway traffic based on videos whose frame rate is too low to accurately estimate motion features. As we are focusing on analyzing traffic images and low frame-rate videos, traffic density is defined as the percentage of road being occupied by vehicles. In our previous work, we validated that traffic status is highly correlated to its texture features and that Convolutional Neural Networks (CNN) has the superiority of extracting discriminative texture features. We proposed several CNN models to segment traffic images into three different classes (road, car and background), classify traffic images into different categories (empty, fluid, heavy, jam) and predict traffic density without using any motion features. In order to generalize the model trained on a specific dataset to analyze new traffic scenes, we also proposed a novel transfer learning framework to do model adaptation. Vehicle detection and classification: The detection of vehicles pictured by traffic cameras is often the very first step of video surveillance systems, such as vehicle counting, tracking and retrieval. In this part, we explore different deep learning methods applied to vehicle detection and classification. Firstly, realizing the importance of large dataset for traffic analysis, we built and released the largest traffic dataset (MIO-TCD) in the world for vehicle localization and classification in collaboration with colleagues from Miovision inc. (Waterloo, On). With this dataset, we organized the Traffic Surveillance Workshop and Challenge in conjunction with CVPR 2017. Secondly, we evaluated several state-of-the-art deep learning methods for the classification and localization task on the MIO-TCD dataset. In light of the results, we may conclude that state-of-the-art deep learning methods exhibit a capacity to localize and recognize vehicle from a single video frame. While with a deep analysis of the results, we also identify scenarios for which state-of-the-art methods are still failing and propose concrete ideas for future work. Lastly, as saliency detection aims to highlight the most relevant objects in an image (e.g. vehicles in traffic scenes), we proposed a multi-resolution 4*5 grid CNN model for the salient object detection. The model enables near real-time high performance saliency detection. We also extend this model to do traffic analysis, experiment results show that our model can precisely segment foreground vehicles in traffic scenes. / De nos jours, l’analyse de trafic routier est de plus en plus automatisée et s’appuie sur des données issues de senseurs en tout genre. Parmi les approches d’analyse de trafic routier figurent les méthodes à base de vidéo. Les méthodes à base de vidéo ont pour but d’identifier et de reconnaître les objets en mouvement (généralement des voitures et des piétons) et de comprendre leur dynamique. Un des défis parmi les plus difficile à résoudre est d’analyser des séquences vidéo dont le nombre d’images par seconde est très faible. Ce type de situation est pourtant fréquent considérant qu’il est très difficile (voir impossible) de transmettre et de stocker sur un serveur un très grand nombre d’images issues de plusieurs caméras. Dans ce cas, les méthodes issues de l’état de l’art échouent car un faible nombre d’images par seconde ne permet pas d’extraire les caractéristiques vidéos utilisées par ces méthodes tels le flux optique, la détection de mouvement et le suivi de véhicules. Au cours de cette thèse, nous nous sommes concentré sur l’analyse de trafic routier à partir de séquences vidéo contenant un très faible nombre d’images par seconde. Plus particulièrement, nous nous sommes concentrés sur les problème d’estimation de la densité du trafic routier et de la classification de véhicules. Pour ce faire, nous avons proposé différents modèles à base de réseaux de neurones profonds (plus particulièrement des réseaux à convolution) ainsi que de nouvelles bases de données permettant d’entraîner les dits modèles. Parmi ces bases de données figure « MIO-TCD », la plus grosse base de données annotées au monde faite pour l’analyse de trafic routier.

Nowadays, satellite-based localization is a well-established technical solution to support several navigation tasks in daily life. Besides the application inside of portable devices, satellite-based positioning is used for in-vehicle navigation systems as well. Moreover, due to its global coverage and the availability of inexpensive receiver hardware it is an appealing technology for numerous applications in the area of Intelligent Transportation Systems (ITSs). However, it has to be admitted that most of the aforementioned examples either rely on modest accuracy requirements or are not sensitive to temporary integrity violations. Although technical concepts of Advanced Driver Assistance Systems (ADASs) based on Global Navigation Satellite Systems (GNSSs) have been successfully demonstrated under open sky conditions, practice reveals that such systems suffer from degraded satellite signal quality when put into urban areas. Thus, the main research objective of this thesis is to provide a reliable vehicle positioning concept which can be used in urban areas without the aforementioned limitations. Therefore, an integrated probabilistic approach which preforms fault detection & exclusion, localization and multi-sensor data fusion within one unified Bayesian framework is proposed. From an algorithmic perspective, the presented concept is based on a probabilistic data association technique with explicit handling of outlier measurements as present in urban areas. By that approach, the accuracy, integrity and availability are improved at the same time, that is, a consistent positioning solution is provided. In addition, a comprehensive and in-depth analysis of typical errors in urban areas within the pseudorange domain is performed. Based on this analysis, probabilistic models are proposed and later on used to facilitate the positioning algorithm. Moreover, the presented concept clearly targets towards mass-market applications based on low-cost receivers and hence aims to replace costly sensors by smart algorithms. The benefits of these theoretical contributions are implemented and demonstrated on the example of a real-time vehicle positioning prototype as used inside of the European research project GAlileo Interactive driviNg (GAIN). This work describes all necessary parts of this system including GNSS signal processing, fault detection and multi-sensor data fusion within one processing chain. Finally, the performance and benefits of the proposed concept are examined and validated both with simulated and comprehensive real-world sensor data from numerous test drives.