The objective of this dissertation is to develop a Multi-Camera Lane Departure Warning (MCLDW) system and a framework to evaluate it. A Lane Departure Warning (LDW) system is a safety feature that is included in a few luxury automobiles. Using a single camera, it performs the task of informing the driver if a lane change is imminent. The core component of an LDW system is a lane detector, whose objective is to find lane markers on the road. Therefore, we start this dissertation by explaining the requirements of an ideal lane detector, and then present several algorithmic implementations that meet these requirements. After selecting the best implementation, we present the MCLDW methodology. Using a multi-camera setup, MCLDW system combines the detected lane marker information from each camera's view to estimate the immediate distance between the vehicle and the lane marker, and signals a warning if this distance is under a certain threshold. Next, we introduce a procedure to create ground truth and a database of videos which serve as the framework for evaluation. Ground truth is created using an efficient procedure called Time-Slicing that allows the user to quickly annotate the true locations of the lane markers in each frame of the videos. Subsequently, we describe the details of a database of driving videos that has been put together to help establish a benchmark for evaluating existing lane detectors and LDW systems. Finally, we conclude the dissertation by citing the contributions of the research and discussing the avenues for future work.

Durch die Entwicklung und Einführung moderner Fahrerassistenzsysteme soll der Komfort und die Sicherheit des Autofahrens erhöht werden. Das Fahrerassistenzsystem Speed Control verbindet die Ergebnisse der Forschungsarbeiten der letzten Jahre aus dem Bereich des Straßenentwurfs und der Fahrzeugtechnik. Dieses System warnt den Kraftfahrer vor sicherheitskritischen Stellen in der Linienführung von Außerortsstraßen. Es empfiehlt dem Fahrer eine sicher und komfortabel fahrbare Geschwindigkeit für den vorausliegenden Streckenabschnitt. Dafür wurden in Abhängigkeit des Fahrertyps Modelle zur Prognose und Beschreibung des Geschwindigkeits- und Beschleunigungsverhaltens entwickelt. Die Umgebungsbedingungen (Tag, nass) werden dabei mit beachtet. / By developing and implementing modern driver assistance systems the comfort and safety of driving shall be improved. The driver assistance system Speed Control combines the last year’s research work results in the fields of road design and automotive engineering. This system alerts the driver to safety critic spots in the alignment of roads. It recommends a safe and comfortable driving speed for the road segment ahead. Therefore driver type depending models to predict and describe the speed and acceleration behaviour were developed. Withal environmental conditions (day, wet) are regarded.

Originally presented as the author's Thesis--Technische Hochschule Aachen, 2008. / Includes bibliographical references.

Durch das I-FAS der TU Chemnitz wurde im Rahmen des AKTIV-Projektes eine Probandenstudie zur Akzeptanz von Systemausprägungen einer Aktiven Gefahrenbremsung (AGB) bei PKW durchgeführt. Unter Verwendung eines stehenden Hindernisses wurden sechs Systemausprägungen verglichen, die von den AGB-Partnern in zwei Versuchsträger implementiert wurden. Die sechs Systemausprägungen werden nahezu identisch bewertet, solange Probanden keine Vergleichsmöglichkeit zu anderen Systemausprägungen haben. Wenn es zu einem Fahrereingriff kommt, ist der Eingriffszeitpunkt des Fahrers unabhängig von der gefahrenen Systemausprägung.

O uso de sistemas de assistência ao motorista (DAS) baseados em visão tem contribuído consideravelmente na redução de acidentes e consequentemente no auxílio de uma melhor condução. Estes sistemas utilizam basicamente uma câmera de vídeo embarcada (normalmente fixada no para-brisa) com o propósito de extrair informações acerca da rodovia e ajudar o condutor num melhor processo de dirigibilidade. Pequenas distrações ou a perda de concentração podem ser suficientes para que um acidente ocorra. Este trabalho apresenta uma proposta para o desenvolvimento de algoritmos para extrair informações sobre a sinalização em rodovias. Mais precisamente, serão abordados algoritmos de calibração de câmera explorando a geometria da pista, de extração da marcação de pintura (sinalização horizontal) e detecção e identificação de placas de trânsito (sinalização vertical). Os resultados experimentais indicam que o método de calibração de câmera alcançou bons resultados na obtenção dos parâmetros extrínsecos com erros inferiores a 0:5 . O erro médio encontrado nos experimentos com relação a estimativa da altura da câmera foi em torno de 12 cm (erro relativo aproximado de 10%), permitindo explorar o uso da realidade aumentada como uma possível aplicação. A acurácia global para a detecção e reconhecimento da sinalização horizontal (marcas seccionadas, contínuas e mistas) foi acima de 96% perante uma diversidade de situações apresentadas, tais como: sombras, variação de iluminação, degradação do asfalto e pintura. O uso da câmera calibrada para a detecção da sinalização vertical contribui para delimitar o espaço de varredura da janela deslizante do detector, bem como realizar a procura por placas em uma única escala para cada região de busca, caracterizada pela distância ao veículo. Os resultados apresentados reportam uma taxa global de classificação de aproximadamente 99% para o sinal de proibido ultrapassar, considerando-se uma base de dados limitada a 962 amostras. / The use of driver assistance systems (DAS) based on computer vision has helped considerably in reducing accidents and consequently aid in better driving. These systems primarily use an embedded video camera (usually fixed on the windshield) for the purpose of extracting information about the highway and assisting the driver in a better handling process. Small distractions or loss of concentration may be sufficient for an accident to occur. This work presents the development of algorithms to extract information about traffic signs on highways. More specifically, this work will tackle a camera calibration algorithm that exploits the geometry of the road track, algorithms for the extraction of road marking paint (lane markings) and detection and identification of vertical traffic signs. Experimental results indicate that the proposed method for obtaining the extrinsic parameters achieve good results with errors of less than 0:5 . The average error in our experiments, related to the camera height, were around 12 cm (relative error around 10%). Global accuracy for the detection and classification of road lane markings (dashed, solid, dashed-solid, solid-dashed or double solid) were over 96%. Finally, our camera calibration algorithm was used to reduce the search region and to define the scale of a slidingwindow detector for vertical traffic signs. The use of the calibrated camera for the detection of traffic signs contributes to define the scanning area of the sliding window and perform a search for signs on a unique scale for each region of interest, determined by the distance to the vehicle. The results reported a global classification rate of approximately 99% for the no overtaking sign, considering a limited of 962 samples.

abstract: Driving is already a complex task that demands a varying level of cognitive and physical load. With the advancement in technology, the car has become a place for media consumption, a communications center and an interconnected workplace. The number of features in a car has also increased. As a result, the user interaction inside the car has become overcrowded and more complex. This has increased the amount of distraction while driving and has also increased the number of accidents due to distracted driving. This thesis focuses on the critical analysis of today’s in-car environment covering two main aspects, Multi Modal Interaction (MMI), and Advanced Driver Assistance Systems (ADAS), to minimize the distraction. It also provides deep market research on future trends in the smart car technology. After careful analysis, it was observed that an infotainment screen cluttered with lots of small icons, a center stack with a plethora of small buttons and a poor Voice Recognition (VR) results in high cognitive load, and these are the reasons for the increased driver distraction. Though the VR has become a standard technology, the current state of technology is focused on features oriented design and a sales driven approach. Most of the automotive manufacturers are focusing on making the VR better but attaining perfection in VR is not the answer as there are inherent challenges and limitations in respect to the in-car environment and cognitive load. Accordingly, the research proposed a novel in-car interaction design solution: Multi-Modal Interaction (MMI). The MMI is a new term when used in the context of vehicles, but it is widely used in human-human interaction. The approach offers a non-intrusive alternative to the driver to interact with the features in the car. With the focus on user-centered design, the MMI and ADAS can potentially help to reduce the distraction. To support the discussion, an experiment was conducted to benchmark a minimalist UI design. An engineering based method was used to test and measure distraction of four different UIs with varying numbers of icons and screen sizes. Lastly, in order to compete with the market, the basic features that are provided by all the other competitors cannot be eliminated, but the hard work can be done to improve the HCaI and to make driving safer. / Dissertation/Thesis / Date collected about reaction time in the experiment_Excel / Masters Thesis Computer Science 2015

abstract: The environmental impact of the fossil fuels has increased tremendously in the last decade. This impact is one of the most contributing factors of global warming. This research aims to reduce the amount of fuel consumed by vehicles through optimizing the control scheme for the future route information. Taking advantage of more degrees of freedom available within PHEV, HEV, and FCHEV “energy management” allows more margin to maximize efficiency in the propulsion systems. The application focuses on reducing the energy consumption in vehicles by acquiring information about the road grade. Road elevations are obtained by use of Geographic Information System (GIS) maps to optimize the controller. The optimization is then reflected on the powertrain of the vehicle.The approach uses a Model Predictive Control (MPC) algorithm that allows the energy management strategy to leverage road grade to prepare the vehicle for minimizing energy consumption during an uphill and potential energy harvesting during a downhill. The control algorithm will predict future energy/power requirements of the vehicle and optimize the performance by instructing the power split between the internal combustion engine (ICE) and the electric-drive system. Allowing for more efficient operation and higher performance of the PHEV, and HEV. Implementation of different strategies, such as MPC and Dynamic Programming (DP), is considered for optimizing energy management systems. These strategies are utilized to have a low processing time. This approach allows the optimization to be integrated with ADAS applications, using current technology for implementable real time applications. The Thesis presents multiple control strategies designed, implemented, and tested using real-world road elevation data from three different routes. Initial simulation based results show significant energy savings. The savings range between 11.84% and 25.5% for both Rule Based (RB) and DP strategies on the real world tested routes. Future work will take advantage of vehicle connectivity and ADAS systems to utilize Vehicle to Vehicle (V2V), Vehicle to Infrastructure (V2I), traffic information, and sensor fusion to further optimize the PHEV and HEV toward more energy efficient operation. / Dissertation/Thesis / Masters Thesis Mechanical Engineering 2016

Les véhicules dits intelligents actuellement développés par la plupart des constructeurs automobiles, ainsi que les véhicules autonomes nécessitent des informations sur le contexte dans lequel ils évoluent. Certaines de ces informations (par exemple la courbure de la route, la forme des intersections, les limitations de vitesses) sont fournies en temps réel par le système de navigation qui exploite les données de cartes routières numériques. Des défauts résultant de l’évolution du réseau routier ou d’imprécisions lors de la collecte de données peuvent être contenus dans ces cartes numériques et entraîner le dysfonctionnement des systèmes d’aide à la conduite. Les recherches menées dans cette thèse visent à rendre le véhicule capable d’évaluer, de manière autonome et en temps réel, l’intégrité des informations fournies par son système de navigation. Les véhicules de série sont désormais équipés d’un grand nombre de capteurs qui transmettent leurs mesures sur le réseau central interne du véhicule. Ces données sont donc facilement accessibles mais de faible précision. Le défi de cette thèse réside donc dans l’évaluation de l’intégrité des informations cartographiques malgré un faible degré de redondance et l’absence de données fiables. On s’adresse à deux types de défauts cartographiques : les défauts structurels et les défauts géométriques. Les défauts structurels concernent les connections entre les routes (intersections). Un cas particulier de défaut structurel est traité : la détection de ronds-points qui n’apparaissent pas dans la carte numérique. Ce défaut est essentiel car il est fréquent (surtout en Europe) et perturbe le fonctionnement des aides à la conduite. Les ronds-points sont détectés à partir de la forme typique de la trajectoire du véhicule lorsqu’il les traverse, puis sont mémorisés pour avertir les aides à la conduite aux prochains passages du véhicule sur la zone. Les imprécisions de représentation du tracé des routes dans la carte numérique sont quant à elles désignées comme défauts géométriques. Un formalisme mathématique est développé pour détecter ces défauts en comparant l’estimation de la position du véhicule d’après la carte à une autre estimation indépendante de la carte. Cette seconde estimation pouvant elle aussi être affectée par un défaut, les anciens trajetsdu véhicule sur la même zone sont utilisés. Un test statistique est finalement utilisé pour améliorer la méthode de détection de défauts géométriques dans des conditions de mesures bruitées. Toutes les méthodes développées dans le cadre de cette thèse sont évaluées à l’aide de données réelles. / Several Intelligent Vehicles capabilities from Advanced Driving Assistance Systems (ADAS) to Autonomous Driving functions depend on a priori information provided by navigation maps. Whilst these were intended for driver guidance as they store road network information, today they are even used in applications that control vehicle motion. In general, the vehicle position is projected onto the map to relate with links in the stored road network. However, maps might contain faults, leading to navigation and situation understanding errors. Therefore, the integrity of the map-matched estimates must be monitored to avoid failures that can lead to hazardous situations. The main focus of this research is the real-time autonomous evaluation of faults in navigation maps used in intelligent vehicles. Current passenger vehicles are equipped with proprioceptive sensors that allow estimating accurately the vehicle state over short periods of time rather than long trajectories. They include receiver for Global Navigation Satellite System (GNSS) and are also increasingly equipped with exteroceptive sensors like radar or smart camera systems. The challenge resides on evaluating the integrity of the navigation maps using vehicle on board sensors. Two types of map faults are considered: Structural Faults, addressing connectivity (e.g., intersections). Geometric Faults, addressing geographic location and road geometry (i.e. shape). Initially, a particular structural navigation map fault is addressed: the detection of roundabouts absent in the navigation map. This structural fault is problematic for ADAS and Autonomous Driving. The roundabouts are detected by classifying the shape of the vehicle trajectory. This is stored for use in ADAS and Autonomous Driving functions on future vehicle trips on the same area. Next, the geometry of the map is addressed. The main difficulties to do the autonomous integrity monitoring are the lack of reliable information and the low level of redundancy. This thesis introduces a mathematical framework based on the use of repeated vehicle trips to assess the integrity of map information. A sequential test is then developed to make it robust to noisy sensor data. The mathematical framework is demonstrated theoretically including the derivation of definitions and associated properties. Experiments using data acquired in real traffic conditions illustrate the performance of the proposed approaches.

O uso de sistemas de assistência ao motorista (DAS) baseados em visão tem contribuído consideravelmente na redução de acidentes e consequentemente no auxílio de uma melhor condução. Estes sistemas utilizam basicamente uma câmera de vídeo embarcada (normalmente fixada no para-brisa) com o propósito de extrair informações acerca da rodovia e ajudar o condutor num melhor processo de dirigibilidade. Pequenas distrações ou a perda de concentração podem ser suficientes para que um acidente ocorra. Este trabalho apresenta uma proposta para o desenvolvimento de algoritmos para extrair informações sobre a sinalização em rodovias. Mais precisamente, serão abordados algoritmos de calibração de câmera explorando a geometria da pista, de extração da marcação de pintura (sinalização horizontal) e detecção e identificação de placas de trânsito (sinalização vertical). Os resultados experimentais indicam que o método de calibração de câmera alcançou bons resultados na obtenção dos parâmetros extrínsecos com erros inferiores a 0:5 . O erro médio encontrado nos experimentos com relação a estimativa da altura da câmera foi em torno de 12 cm (erro relativo aproximado de 10%), permitindo explorar o uso da realidade aumentada como uma possível aplicação. A acurácia global para a detecção e reconhecimento da sinalização horizontal (marcas seccionadas, contínuas e mistas) foi acima de 96% perante uma diversidade de situações apresentadas, tais como: sombras, variação de iluminação, degradação do asfalto e pintura. O uso da câmera calibrada para a detecção da sinalização vertical contribui para delimitar o espaço de varredura da janela deslizante do detector, bem como realizar a procura por placas em uma única escala para cada região de busca, caracterizada pela distância ao veículo. Os resultados apresentados reportam uma taxa global de classificação de aproximadamente 99% para o sinal de proibido ultrapassar, considerando-se uma base de dados limitada a 962 amostras. / The use of driver assistance systems (DAS) based on computer vision has helped considerably in reducing accidents and consequently aid in better driving. These systems primarily use an embedded video camera (usually fixed on the windshield) for the purpose of extracting information about the highway and assisting the driver in a better handling process. Small distractions or loss of concentration may be sufficient for an accident to occur. This work presents the development of algorithms to extract information about traffic signs on highways. More specifically, this work will tackle a camera calibration algorithm that exploits the geometry of the road track, algorithms for the extraction of road marking paint (lane markings) and detection and identification of vertical traffic signs. Experimental results indicate that the proposed method for obtaining the extrinsic parameters achieve good results with errors of less than 0:5 . The average error in our experiments, related to the camera height, were around 12 cm (relative error around 10%). Global accuracy for the detection and classification of road lane markings (dashed, solid, dashed-solid, solid-dashed or double solid) were over 96%. Finally, our camera calibration algorithm was used to reduce the search region and to define the scale of a slidingwindow detector for vertical traffic signs. The use of the calibrated camera for the detection of traffic signs contributes to define the scanning area of the sliding window and perform a search for signs on a unique scale for each region of interest, determined by the distance to the vehicle. The results reported a global classification rate of approximately 99% for the no overtaking sign, considering a limited of 962 samples.

O uso de sistemas de assistência ao motorista (DAS) baseados em visão tem contribuído consideravelmente na redução de acidentes e consequentemente no auxílio de uma melhor condução. Estes sistemas utilizam basicamente uma câmera de vídeo embarcada (normalmente fixada no para-brisa) com o propósito de extrair informações acerca da rodovia e ajudar o condutor num melhor processo de dirigibilidade. Pequenas distrações ou a perda de concentração podem ser suficientes para que um acidente ocorra. Este trabalho apresenta uma proposta para o desenvolvimento de algoritmos para extrair informações sobre a sinalização em rodovias. Mais precisamente, serão abordados algoritmos de calibração de câmera explorando a geometria da pista, de extração da marcação de pintura (sinalização horizontal) e detecção e identificação de placas de trânsito (sinalização vertical). Os resultados experimentais indicam que o método de calibração de câmera alcançou bons resultados na obtenção dos parâmetros extrínsecos com erros inferiores a 0:5 . O erro médio encontrado nos experimentos com relação a estimativa da altura da câmera foi em torno de 12 cm (erro relativo aproximado de 10%), permitindo explorar o uso da realidade aumentada como uma possível aplicação. A acurácia global para a detecção e reconhecimento da sinalização horizontal (marcas seccionadas, contínuas e mistas) foi acima de 96% perante uma diversidade de situações apresentadas, tais como: sombras, variação de iluminação, degradação do asfalto e pintura. O uso da câmera calibrada para a detecção da sinalização vertical contribui para delimitar o espaço de varredura da janela deslizante do detector, bem como realizar a procura por placas em uma única escala para cada região de busca, caracterizada pela distância ao veículo. Os resultados apresentados reportam uma taxa global de classificação de aproximadamente 99% para o sinal de proibido ultrapassar, considerando-se uma base de dados limitada a 962 amostras. / The use of driver assistance systems (DAS) based on computer vision has helped considerably in reducing accidents and consequently aid in better driving. These systems primarily use an embedded video camera (usually fixed on the windshield) for the purpose of extracting information about the highway and assisting the driver in a better handling process. Small distractions or loss of concentration may be sufficient for an accident to occur. This work presents the development of algorithms to extract information about traffic signs on highways. More specifically, this work will tackle a camera calibration algorithm that exploits the geometry of the road track, algorithms for the extraction of road marking paint (lane markings) and detection and identification of vertical traffic signs. Experimental results indicate that the proposed method for obtaining the extrinsic parameters achieve good results with errors of less than 0:5 . The average error in our experiments, related to the camera height, were around 12 cm (relative error around 10%). Global accuracy for the detection and classification of road lane markings (dashed, solid, dashed-solid, solid-dashed or double solid) were over 96%. Finally, our camera calibration algorithm was used to reduce the search region and to define the scale of a slidingwindow detector for vertical traffic signs. The use of the calibrated camera for the detection of traffic signs contributes to define the scanning area of the sliding window and perform a search for signs on a unique scale for each region of interest, determined by the distance to the vehicle. The results reported a global classification rate of approximately 99% for the no overtaking sign, considering a limited of 962 samples.