O uso de sistemas de assistência ao motorista (DAS) baseados em visão tem contribuído consideravelmente na redução de acidentes e consequentemente no auxílio de uma melhor condução. Estes sistemas utilizam basicamente uma câmera de vídeo embarcada (normalmente fixada no para-brisa) com o propósito de extrair informações acerca da rodovia e ajudar o condutor num melhor processo de dirigibilidade. Pequenas distrações ou a perda de concentração podem ser suficientes para que um acidente ocorra. Este trabalho apresenta uma proposta para o desenvolvimento de algoritmos para extrair informações sobre a sinalização em rodovias. Mais precisamente, serão abordados algoritmos de calibração de câmera explorando a geometria da pista, de extração da marcação de pintura (sinalização horizontal) e detecção e identificação de placas de trânsito (sinalização vertical). Os resultados experimentais indicam que o método de calibração de câmera alcançou bons resultados na obtenção dos parâmetros extrínsecos com erros inferiores a 0:5 . O erro médio encontrado nos experimentos com relação a estimativa da altura da câmera foi em torno de 12 cm (erro relativo aproximado de 10%), permitindo explorar o uso da realidade aumentada como uma possível aplicação. A acurácia global para a detecção e reconhecimento da sinalização horizontal (marcas seccionadas, contínuas e mistas) foi acima de 96% perante uma diversidade de situações apresentadas, tais como: sombras, variação de iluminação, degradação do asfalto e pintura. O uso da câmera calibrada para a detecção da sinalização vertical contribui para delimitar o espaço de varredura da janela deslizante do detector, bem como realizar a procura por placas em uma única escala para cada região de busca, caracterizada pela distância ao veículo. Os resultados apresentados reportam uma taxa global de classificação de aproximadamente 99% para o sinal de proibido ultrapassar, considerando-se uma base de dados limitada a 962 amostras. / The use of driver assistance systems (DAS) based on computer vision has helped considerably in reducing accidents and consequently aid in better driving. These systems primarily use an embedded video camera (usually fixed on the windshield) for the purpose of extracting information about the highway and assisting the driver in a better handling process. Small distractions or loss of concentration may be sufficient for an accident to occur. This work presents the development of algorithms to extract information about traffic signs on highways. More specifically, this work will tackle a camera calibration algorithm that exploits the geometry of the road track, algorithms for the extraction of road marking paint (lane markings) and detection and identification of vertical traffic signs. Experimental results indicate that the proposed method for obtaining the extrinsic parameters achieve good results with errors of less than 0:5 . The average error in our experiments, related to the camera height, were around 12 cm (relative error around 10%). Global accuracy for the detection and classification of road lane markings (dashed, solid, dashed-solid, solid-dashed or double solid) were over 96%. Finally, our camera calibration algorithm was used to reduce the search region and to define the scale of a slidingwindow detector for vertical traffic signs. The use of the calibrated camera for the detection of traffic signs contributes to define the scanning area of the sliding window and perform a search for signs on a unique scale for each region of interest, determined by the distance to the vehicle. The results reported a global classification rate of approximately 99% for the no overtaking sign, considering a limited of 962 samples.

abstract: Driving is already a complex task that demands a varying level of cognitive and physical load. With the advancement in technology, the car has become a place for media consumption, a communications center and an interconnected workplace. The number of features in a car has also increased. As a result, the user interaction inside the car has become overcrowded and more complex. This has increased the amount of distraction while driving and has also increased the number of accidents due to distracted driving. This thesis focuses on the critical analysis of today’s in-car environment covering two main aspects, Multi Modal Interaction (MMI), and Advanced Driver Assistance Systems (ADAS), to minimize the distraction. It also provides deep market research on future trends in the smart car technology. After careful analysis, it was observed that an infotainment screen cluttered with lots of small icons, a center stack with a plethora of small buttons and a poor Voice Recognition (VR) results in high cognitive load, and these are the reasons for the increased driver distraction. Though the VR has become a standard technology, the current state of technology is focused on features oriented design and a sales driven approach. Most of the automotive manufacturers are focusing on making the VR better but attaining perfection in VR is not the answer as there are inherent challenges and limitations in respect to the in-car environment and cognitive load. Accordingly, the research proposed a novel in-car interaction design solution: Multi-Modal Interaction (MMI). The MMI is a new term when used in the context of vehicles, but it is widely used in human-human interaction. The approach offers a non-intrusive alternative to the driver to interact with the features in the car. With the focus on user-centered design, the MMI and ADAS can potentially help to reduce the distraction. To support the discussion, an experiment was conducted to benchmark a minimalist UI design. An engineering based method was used to test and measure distraction of four different UIs with varying numbers of icons and screen sizes. Lastly, in order to compete with the market, the basic features that are provided by all the other competitors cannot be eliminated, but the hard work can be done to improve the HCaI and to make driving safer. / Dissertation/Thesis / Date collected about reaction time in the experiment_Excel / Masters Thesis Computer Science 2015

abstract: The environmental impact of the fossil fuels has increased tremendously in the last decade. This impact is one of the most contributing factors of global warming. This research aims to reduce the amount of fuel consumed by vehicles through optimizing the control scheme for the future route information. Taking advantage of more degrees of freedom available within PHEV, HEV, and FCHEV “energy management” allows more margin to maximize efficiency in the propulsion systems. The application focuses on reducing the energy consumption in vehicles by acquiring information about the road grade. Road elevations are obtained by use of Geographic Information System (GIS) maps to optimize the controller. The optimization is then reflected on the powertrain of the vehicle.The approach uses a Model Predictive Control (MPC) algorithm that allows the energy management strategy to leverage road grade to prepare the vehicle for minimizing energy consumption during an uphill and potential energy harvesting during a downhill. The control algorithm will predict future energy/power requirements of the vehicle and optimize the performance by instructing the power split between the internal combustion engine (ICE) and the electric-drive system. Allowing for more efficient operation and higher performance of the PHEV, and HEV. Implementation of different strategies, such as MPC and Dynamic Programming (DP), is considered for optimizing energy management systems. These strategies are utilized to have a low processing time. This approach allows the optimization to be integrated with ADAS applications, using current technology for implementable real time applications. The Thesis presents multiple control strategies designed, implemented, and tested using real-world road elevation data from three different routes. Initial simulation based results show significant energy savings. The savings range between 11.84% and 25.5% for both Rule Based (RB) and DP strategies on the real world tested routes. Future work will take advantage of vehicle connectivity and ADAS systems to utilize Vehicle to Vehicle (V2V), Vehicle to Infrastructure (V2I), traffic information, and sensor fusion to further optimize the PHEV and HEV toward more energy efficient operation. / Dissertation/Thesis / Masters Thesis Mechanical Engineering 2016

Les véhicules dits intelligents actuellement développés par la plupart des constructeurs automobiles, ainsi que les véhicules autonomes nécessitent des informations sur le contexte dans lequel ils évoluent. Certaines de ces informations (par exemple la courbure de la route, la forme des intersections, les limitations de vitesses) sont fournies en temps réel par le système de navigation qui exploite les données de cartes routières numériques. Des défauts résultant de l’évolution du réseau routier ou d’imprécisions lors de la collecte de données peuvent être contenus dans ces cartes numériques et entraîner le dysfonctionnement des systèmes d’aide à la conduite. Les recherches menées dans cette thèse visent à rendre le véhicule capable d’évaluer, de manière autonome et en temps réel, l’intégrité des informations fournies par son système de navigation. Les véhicules de série sont désormais équipés d’un grand nombre de capteurs qui transmettent leurs mesures sur le réseau central interne du véhicule. Ces données sont donc facilement accessibles mais de faible précision. Le défi de cette thèse réside donc dans l’évaluation de l’intégrité des informations cartographiques malgré un faible degré de redondance et l’absence de données fiables. On s’adresse à deux types de défauts cartographiques : les défauts structurels et les défauts géométriques. Les défauts structurels concernent les connections entre les routes (intersections). Un cas particulier de défaut structurel est traité : la détection de ronds-points qui n’apparaissent pas dans la carte numérique. Ce défaut est essentiel car il est fréquent (surtout en Europe) et perturbe le fonctionnement des aides à la conduite. Les ronds-points sont détectés à partir de la forme typique de la trajectoire du véhicule lorsqu’il les traverse, puis sont mémorisés pour avertir les aides à la conduite aux prochains passages du véhicule sur la zone. Les imprécisions de représentation du tracé des routes dans la carte numérique sont quant à elles désignées comme défauts géométriques. Un formalisme mathématique est développé pour détecter ces défauts en comparant l’estimation de la position du véhicule d’après la carte à une autre estimation indépendante de la carte. Cette seconde estimation pouvant elle aussi être affectée par un défaut, les anciens trajetsdu véhicule sur la même zone sont utilisés. Un test statistique est finalement utilisé pour améliorer la méthode de détection de défauts géométriques dans des conditions de mesures bruitées. Toutes les méthodes développées dans le cadre de cette thèse sont évaluées à l’aide de données réelles. / Several Intelligent Vehicles capabilities from Advanced Driving Assistance Systems (ADAS) to Autonomous Driving functions depend on a priori information provided by navigation maps. Whilst these were intended for driver guidance as they store road network information, today they are even used in applications that control vehicle motion. In general, the vehicle position is projected onto the map to relate with links in the stored road network. However, maps might contain faults, leading to navigation and situation understanding errors. Therefore, the integrity of the map-matched estimates must be monitored to avoid failures that can lead to hazardous situations. The main focus of this research is the real-time autonomous evaluation of faults in navigation maps used in intelligent vehicles. Current passenger vehicles are equipped with proprioceptive sensors that allow estimating accurately the vehicle state over short periods of time rather than long trajectories. They include receiver for Global Navigation Satellite System (GNSS) and are also increasingly equipped with exteroceptive sensors like radar or smart camera systems. The challenge resides on evaluating the integrity of the navigation maps using vehicle on board sensors. Two types of map faults are considered: Structural Faults, addressing connectivity (e.g., intersections). Geometric Faults, addressing geographic location and road geometry (i.e. shape). Initially, a particular structural navigation map fault is addressed: the detection of roundabouts absent in the navigation map. This structural fault is problematic for ADAS and Autonomous Driving. The roundabouts are detected by classifying the shape of the vehicle trajectory. This is stored for use in ADAS and Autonomous Driving functions on future vehicle trips on the same area. Next, the geometry of the map is addressed. The main difficulties to do the autonomous integrity monitoring are the lack of reliable information and the low level of redundancy. This thesis introduces a mathematical framework based on the use of repeated vehicle trips to assess the integrity of map information. A sequential test is then developed to make it robust to noisy sensor data. The mathematical framework is demonstrated theoretically including the derivation of definitions and associated properties. Experiments using data acquired in real traffic conditions illustrate the performance of the proposed approaches.

O uso de sistemas de assistência ao motorista (DAS) baseados em visão tem contribuído consideravelmente na redução de acidentes e consequentemente no auxílio de uma melhor condução. Estes sistemas utilizam basicamente uma câmera de vídeo embarcada (normalmente fixada no para-brisa) com o propósito de extrair informações acerca da rodovia e ajudar o condutor num melhor processo de dirigibilidade. Pequenas distrações ou a perda de concentração podem ser suficientes para que um acidente ocorra. Este trabalho apresenta uma proposta para o desenvolvimento de algoritmos para extrair informações sobre a sinalização em rodovias. Mais precisamente, serão abordados algoritmos de calibração de câmera explorando a geometria da pista, de extração da marcação de pintura (sinalização horizontal) e detecção e identificação de placas de trânsito (sinalização vertical). Os resultados experimentais indicam que o método de calibração de câmera alcançou bons resultados na obtenção dos parâmetros extrínsecos com erros inferiores a 0:5 . O erro médio encontrado nos experimentos com relação a estimativa da altura da câmera foi em torno de 12 cm (erro relativo aproximado de 10%), permitindo explorar o uso da realidade aumentada como uma possível aplicação. A acurácia global para a detecção e reconhecimento da sinalização horizontal (marcas seccionadas, contínuas e mistas) foi acima de 96% perante uma diversidade de situações apresentadas, tais como: sombras, variação de iluminação, degradação do asfalto e pintura. O uso da câmera calibrada para a detecção da sinalização vertical contribui para delimitar o espaço de varredura da janela deslizante do detector, bem como realizar a procura por placas em uma única escala para cada região de busca, caracterizada pela distância ao veículo. Os resultados apresentados reportam uma taxa global de classificação de aproximadamente 99% para o sinal de proibido ultrapassar, considerando-se uma base de dados limitada a 962 amostras. / The use of driver assistance systems (DAS) based on computer vision has helped considerably in reducing accidents and consequently aid in better driving. These systems primarily use an embedded video camera (usually fixed on the windshield) for the purpose of extracting information about the highway and assisting the driver in a better handling process. Small distractions or loss of concentration may be sufficient for an accident to occur. This work presents the development of algorithms to extract information about traffic signs on highways. More specifically, this work will tackle a camera calibration algorithm that exploits the geometry of the road track, algorithms for the extraction of road marking paint (lane markings) and detection and identification of vertical traffic signs. Experimental results indicate that the proposed method for obtaining the extrinsic parameters achieve good results with errors of less than 0:5 . The average error in our experiments, related to the camera height, were around 12 cm (relative error around 10%). Global accuracy for the detection and classification of road lane markings (dashed, solid, dashed-solid, solid-dashed or double solid) were over 96%. Finally, our camera calibration algorithm was used to reduce the search region and to define the scale of a slidingwindow detector for vertical traffic signs. The use of the calibrated camera for the detection of traffic signs contributes to define the scanning area of the sliding window and perform a search for signs on a unique scale for each region of interest, determined by the distance to the vehicle. The results reported a global classification rate of approximately 99% for the no overtaking sign, considering a limited of 962 samples.

O uso de sistemas de assistência ao motorista (DAS) baseados em visão tem contribuído consideravelmente na redução de acidentes e consequentemente no auxílio de uma melhor condução. Estes sistemas utilizam basicamente uma câmera de vídeo embarcada (normalmente fixada no para-brisa) com o propósito de extrair informações acerca da rodovia e ajudar o condutor num melhor processo de dirigibilidade. Pequenas distrações ou a perda de concentração podem ser suficientes para que um acidente ocorra. Este trabalho apresenta uma proposta para o desenvolvimento de algoritmos para extrair informações sobre a sinalização em rodovias. Mais precisamente, serão abordados algoritmos de calibração de câmera explorando a geometria da pista, de extração da marcação de pintura (sinalização horizontal) e detecção e identificação de placas de trânsito (sinalização vertical). Os resultados experimentais indicam que o método de calibração de câmera alcançou bons resultados na obtenção dos parâmetros extrínsecos com erros inferiores a 0:5 . O erro médio encontrado nos experimentos com relação a estimativa da altura da câmera foi em torno de 12 cm (erro relativo aproximado de 10%), permitindo explorar o uso da realidade aumentada como uma possível aplicação. A acurácia global para a detecção e reconhecimento da sinalização horizontal (marcas seccionadas, contínuas e mistas) foi acima de 96% perante uma diversidade de situações apresentadas, tais como: sombras, variação de iluminação, degradação do asfalto e pintura. O uso da câmera calibrada para a detecção da sinalização vertical contribui para delimitar o espaço de varredura da janela deslizante do detector, bem como realizar a procura por placas em uma única escala para cada região de busca, caracterizada pela distância ao veículo. Os resultados apresentados reportam uma taxa global de classificação de aproximadamente 99% para o sinal de proibido ultrapassar, considerando-se uma base de dados limitada a 962 amostras. / The use of driver assistance systems (DAS) based on computer vision has helped considerably in reducing accidents and consequently aid in better driving. These systems primarily use an embedded video camera (usually fixed on the windshield) for the purpose of extracting information about the highway and assisting the driver in a better handling process. Small distractions or loss of concentration may be sufficient for an accident to occur. This work presents the development of algorithms to extract information about traffic signs on highways. More specifically, this work will tackle a camera calibration algorithm that exploits the geometry of the road track, algorithms for the extraction of road marking paint (lane markings) and detection and identification of vertical traffic signs. Experimental results indicate that the proposed method for obtaining the extrinsic parameters achieve good results with errors of less than 0:5 . The average error in our experiments, related to the camera height, were around 12 cm (relative error around 10%). Global accuracy for the detection and classification of road lane markings (dashed, solid, dashed-solid, solid-dashed or double solid) were over 96%. Finally, our camera calibration algorithm was used to reduce the search region and to define the scale of a slidingwindow detector for vertical traffic signs. The use of the calibrated camera for the detection of traffic signs contributes to define the scanning area of the sliding window and perform a search for signs on a unique scale for each region of interest, determined by the distance to the vehicle. The results reported a global classification rate of approximately 99% for the no overtaking sign, considering a limited of 962 samples.

The aim of this diploma thesis is the research of building polygons for the testing of autonomous vehicles, from the point of view of road technology and also designing aspects. In the thesis are mentioned 9 most important world test polygons and their description of design parameters. There are described particular stages of automation from foreign organizations which are concerned with research and development in the automotive industry. In addition, there are described basic advanced driver assistance systems and connectivity between vehicles and infrastructure. Conclusion also contains the assessment of existing aerodrome test areas for autonomous vehicles.

abstract: Object detection is an interesting computer vision area that is concerned with the detection of object instances belonging to specific classes of interest as well as the localization of these instances in images and/or videos. Object detection serves as a vital module in many computer vision based applications. This work focuses on the development of object detection methods that exhibit increased robustness to varying illuminations and image quality. In this work, two methods for robust object detection are presented. In the context of varying illumination, this work focuses on robust generic obstacle detection and collision warning in Advanced Driver Assistance Systems (ADAS) under varying illumination conditions. The highlight of the first method is the ability to detect all obstacles without prior knowledge and detect partially occluded obstacles including the obstacles that have not completely appeared in the frame (truncated obstacles). It is first shown that the angular distortion in the Inverse Perspective Mapping (IPM) domain belonging to obstacle edges varies as a function of their corresponding 2D location in the camera plane. This information is used to generate object proposals. A novel proposal assessment method based on fusing statistical properties from both the IPM image and the camera image to perform robust outlier elimination and false positive reduction is also proposed. In the context of image quality, this work focuses on robust multiple-class object detection using deep neural networks for images with varying quality. The use of Generative Adversarial Networks (GANs) is proposed in a novel generative framework to generate features that provide robustness for object detection on reduced quality images. The proposed GAN-based Detection of Objects (GAN-DO) framework is not restricted to any particular architecture and can be generalized to several deep neural network (DNN) based architectures. The resulting deep neural network maintains the exact architecture as the selected baseline model without adding to the model parameter complexity or inference speed. Performance results provided using GAN-DO on object detection datasets establish an improved robustness to varying image quality and a higher object detection and classification accuracy compared to the existing approaches. / Dissertation/Thesis / Doctoral Dissertation Electrical Engineering 2020

Navigation systems are one of the most commonly found electronic gadgets in modern vehicles nowadays. Alongside navigation units this technology is made readily available to individuals in everyday devices such as a mobile phone. Digital maps which come preloaded on these devices accommodate within them an extensive dataset of spatial information from around the globe which aids the driver in achieving a well guided driving experience. Apart from being essential for navigation this sensor information backs up other vehicular applications in making intelligent decisions. The quality of this information delivered is in direct relation to the underlying dataset used to produce these maps. Since we live in a highly dynamic environment with constantly changing geography, an effort is necessary to keep these maps updated with the most up to date information as frequently as possible. The digital map of interest in this study is OpenStreetMap, the underlying data of which is a combination of donated as well as crowdsourced information from the last 10 years. This extensive dataset helps in building of a detailed digital map of the world using well defined cartographic techniques. The information within OpenStreetMap is currently enhanced by a large group of volunteers who willing use donated satellite imagery, uploaded GPS tracks, field surveys etc. to correct and collect necessary data for a region of interest. Though this method helps in improving and increasing the quality and quantity of the OpenStreetMap dataset, it is very time consuming and requires a great deal of human effort. Through this thesis an effort is made to automatically enrich this dataset by preprocessing crowdsourced sensor data collected from the navigation system and driver assistance systems (Traffic Sign Recognition system and a Lane Detection System) of a driving vehicle. The kind of data that is algorithmically derived includes the calculation of the curvature of the underlying road, correction of speed limit values for individual road segments being driven and the identification of change in the geometry of existing roads due to closure of old ones or addition of new ones in the Nuremberg region of Bavaria, Germany. Except for a small percentage of speed limit information on roads segments, other information is currently not available in the OpenStreetMap database for use in safety and comfort related applications. The navigation system has the ability to deliver geographical data in form of GPS coordinates at a certain frequency. This set of GPS coordinates can grouped together to form a GPS track visualizing the actual path traversed by a driving vehicle. A large number of such GPS tracks repeatedly collected from different vehicles driving in a region of interest gives all GPS points which lie on a particular road. These points, after outlier elimination methods are used as a dataset to scientifically determine the underlying curvature of the road with the aid of curve fitting techniques. Additional information received from the lane detection system helps identify curves on a road for which the curvature must be calculated. The fusion of information from these sources helps to achieve curvature results with high accuracy. Traffic sign recognition system helps detect traffic signs while driving, the fusion of this data with geographical information from the navigation system at the instance of detection helps determine road segments for which the recognized speed limit values are valid. This thesis successfully demonstrates a method to automatically enrich OpenStreetMap data by crowdsourcing raw sensor data from multiple vehicles equipped with driver assistance systems. All OpenStreetMap attributes were 100% updated into the database and the results have proven the effectiveness our system architecture. The positive results obtained in combination with minimal errors promise a better future for assisted driving.

The demand for Advanced Driver Assistance System (ADAS) applications is increasing day by day and their development requires efficient prototyping and real time testing. ADTF (Automotive Data and Time Triggered Framework) is a software tool from Elektrobit which is used for Development, Validation and Visualization of Vision based applications, mainly for ADAS and Autonomous driving. With the help of ADTF tool, Image or Video data can be recorded and visualized and also the testing of data can be processed both on-line and off-line. The development of ADAS applications needs image and video processing and the algorithm has to be highly efficient and must satisfy Real-time requirements. The main objective of this research would be to integrate OpenCV library with ADTF cross platform. OpenCV libraries provide efficient image processing algorithms which can be used with ADTF for quick benchmarking and testing. An ADTF filter framework has been developed where the OpenCV algorithms can be directly used and the testing of the framework is carried out with .DAT and image files with a modular approach. CMake is also explained in this thesis to build the system with ease of use. The ADTF filters are developed in Microsoft Visual Studio 2010 in C++ and OpenMP API are used for Parallel programming approach.