Video/Image processing is a fundamental issue in computer science. It is widely used for a broad range of applications, such as weather prediction, computerized tomography (CT), artificial intelligence (AI), and etc. Video-based advanced driver assistance system (ADAS) attracts great attention in recent years, which aims at helping drivers to become more concentrated when driving and giving proper warnings if any danger is insight. Typical ADAS includes lane departure warning, traffic sign detection, pedestrian detection, and etc. Both basic and advanced video/image processing technologies are deployed in video-based driver assistance system. The key requirements of driver assistance system are rapid processing time and low power consumption. We consider Field Programmable Gate Array (FPGA) as the most appropriate embedded platform for ADAS. Owing to the parallel architecture, an FPGA is able to perform high-speed video processing such that it could issue warnings timely and provide drivers longer time to response. Besides, the cost and power consumption of modern FPGAs, particular small size FPGAs, are considerably efficient. Compared to the CPU implementation, the FPGA video/image processing achieves about tens of times speedup for video-based driver assistance system and other applications.

Indiana University-Purdue University Indianapolis (IUPUI) / An autonomous car is a self-driving vehicle, that operates without human intervention and has the capability of sensing the environment around it. To achieve this, the autonomous vehicle mostly depends on multiple Sensors, Actuators, Machine learning, complex algorithms and processors for software execution. Developed Software, at that point, processes all the information obtained from sensors, plans the path, and the instructions are passed to the vehicle’s actuators, which are capable of controlling acceleration, steering, and brake systems. The rules that are hard-coded, algorithms for detection of object and obstacle avoidance, and predictive modelling control algorithms assist the software with observing traffic guidelines and navigate the vehicle accordingly. Free driving is anything but a simple assignment, and to make independent driving game plans is an extraordinarily critical capacity in the current programming planning field. Engineers and Researchers have been keeping huge endeavors to develop safe and precise algorithms to be incorporated in autonomous vehicles. ROS is a flexible and perfect middle ware tool for robotic applications. ROS offers the necessary tools to effortlessly get the sensors information, process that information, and produce a suitable response to actuators of the vehicle. This thesis work plans to exhibit how ROS could be utilized as a middle- ware tool to make the vehicle move autonomously by examining the surroundings and taking decision. The main focus of this thesis is to develop a one-tenth scale of an autonomous Racecar equipped with Jetson Nano as the on-board computer, ROS based software architecture, sensors, and a PWM driver and implement ADAS features such as Emergency Brake system, Lane Detection and Lane change on the autonomous Race car vehicle. At last, by following the strategies introduced in this thesis work, it is possible to build and develop an autonomousvehicle that uses ROS framework.

With the development of Advanced Driving Assistance Systems (ADAS) vehicles have undergone better experience in field of safety, better driving and enhanced vehicle systems. Today these systems are one of the fastest growing in automotive domain. Physical parameters like map data, vehicle position and speed are crucial for the advancement of functionalities implemented for ADAS. All the navigation map databases are stored in proprietary format. So for the ADAS application to access this data an appropriate interface has to be defined. This is the main aim of Advance Driver Assistant Systems Interface Specifications (ADASIS) consortium. This new specification allows a coordinated effort of more than one industry to improve comfort and fuel efficiency. My research during the entire duration of my master thesis mainly focuses on two stages namely XML Comparator and CAN stream generation stages from ADASIS Test Environment that was developed in our company. In this test environment ADASIS Reconstructor of our company is tested against the parameters of Reference Reconstructor provided by ADASIS consortium. The main aim of this environment is to develop a Reconstructor which will adhere to all the specifications given in ADASIS Reconstructor. My implementation in this master thesis focuses on two stages of test environment setup which are XML Comparison and CAN Stream Generation Tool respectively. Prior to my working, these stages lacked in-depth research and usability features for further working.

Le confort et la sécurité de conduite sont les principaux critères de vente de l’industrie automobile actuelle. De nombreux projets de recherche sont mis en place afin de les améliorer et pour faire face aux mesures de législation et de contrôle mises en place pour réduire le nombre d’accidents routiers. Les mesures semblent efficaces : en France,par exemple, le nombre des accidents mortels diminue de 11% en moyenne chaque année.D’après de récentes études, 90% de ces accidents ont pour cause le facteur humain et il devient nécessaire de prendre en compte le conducteur pendant la phase de conception des systèmes de sécurité et d’aide à la conduite. Une assistance à la conduite basée sur le partage du contrôle du véhicule entre le conducteur et l’automate est un des axes de recherche privilégiés de l’industrie, notamment pour améliorer la sécurité.Il est maintenant devenu possible de récupérer une très grande quantité d’information sur l’environnement et de réaliser une interaction intelligente entre les différents acteurs du trafic. Les techniques existantes permettent même la conduite partagée entre le véhicule et le conducteur et, dans un horizon plus lointain, d’envisager un véhicule complètement autonome. Dans les situations de conduite automatisée, un algorithme adéquat est nécessaire pour remplacer le conducteur.L’intérêt principal de cette recherche se situe au niveau de l’interaction entre le conducteur et l’algorithme d’assistance ou de conduite automatisée. Il est indispensable de connaître et de comprendre le comportement du conducteur dans sa façon de conduire,de contrôler le véhicule et de prendre une décision. Par conséquent, un modèle adapté aux besoins est nécessaire. En plus de la nécessité de disposer d’un modèle suffisamment riche pour décrire le comportement de différents conducteurs dans les situations routières les plus fréquentes, il est indispensable de disposer d’une méthode de synthèse des systèmes d’assistance sur la base de ces modèles. / Driving comfort and safety are the main points of interest for the automotive industry. Many research projects were realized in order to improve them and to reduce the number of road accidents. The measures seem to be effective : in France, for example, the number of fatal accidents decreases by 11% on average each year. According to recent studies, 90% of these accidents are caused by the human factor. As a consequence, it becomes necessary to take the driver into account during the design of driving assistance systems. An assistance based on the control sharing between the driver and the automatic pilot is one of the main topics of research and a way to improve safety. It has now become possible to recover a very large amount of information on the environment and to achieve intelligent interaction between the various actors in the traffic. Existing technologies even allow imagining a completely autonomous driving in a more distant horizon. In such a situation, an adequate algorithm is required to replace the human driver.The main interest of this research is the interaction between the driver and the driver assistance algorithm. It is essential to know and to understand the behavior of the humanin his / her way to control the vehicle and to make a decision. Therefore, his model is necessary. Moreover, it is essential to develop a design method for such assistance systems on the basis of these driver models.

Advanced Driver Assistance System (ADAS) was widely learned nowadays. As crucial parts of ADAS, lane markings detection, as well as other objects detection, have become more popular than before. However, most methods implemented in such areas cannot perfectly balance the performance of accuracy versus efficiency, and the mainstream methods (e.g. Machine Learning) suffer from several limitations which can hardly break the wall between partial autonomous and fully autonomous driving. This thesis proposed a real-time lane marking detection framework for ADAS, which included 4-extreme points set descriptor and a rule-based cascade classifier. By analyzing the behavior of lane markings on the road surface, a characteristic of markings was discovered, i.e., standard markings can sustain their shape in the perpendicular plane of the driving direction. By employing this feature, a 4-extreme points set descriptor was applied to describe the shape of each marking first. Specifically, after processing Maximally Stable Extremal Region (MSER) and Hough transforms on a 2-D image, several contours of interest are obtained. A bounding box, with borders parallel to the image coordinate, intersected with each contour at 4 points in the edge, which was named 4-extreme points set. Afterward, to verify consistency of each contour and standard marking, some rules abstracted from construction manual are employed such as Area Filter, Colour Filter, Relative Location Filter, Convex Filter, etc. To reduce the errors caused by changes in driving direction, an enhanced module was then introduced. By tracking the vanishing point as well as other key points of the road net, a method for 3-D reconstruction, with respect to the optical axis between vanishing point and camera center, is possible. The principle of such algorithm was exhibited, and a description about how to obtain the depth information from this model was also provided. Among all of these processes, a key-point based classification method is the main contribution of this paper because of its function in eliminating the deformation of the object caused by inverse perspective mapping. Several experiments were conducted in highway and urban roads in Ottawa. The detection rate of the markings by the proposed algorithm reached an average accuracy rate of 96.77% while F1 Score (harmonic mean of precision and recall) also attained a rate of 90.57%. In summary, the proposed method exhibited a state-of-the-art performance and represents a significant advancement of understanding.

Denna rapport presenterar ett examensarbete på C-nivå där uppdragsgivaren har varit Hedemora Anpassning som bygger om och anpassar bilar för människor med olika funktionshinder. Företaget vill i framtiden ha möjlighet att erbjuda sina kunder ITS-tjänster. Målet med arbetet har varit att undersöka hur ITS skulle kunna stödja de funktionshindrade i deras bilkörning. Syftet delades upp i två delsyften, dels kartläggning av vilka tjänster som finns och är under utveckling inom området ITS – förarstöd och säkerhet samt hur de skulle kunna användas/anpassas för funktionshindrade dels att göra en undersökning av vilka behov de funktionshindrade har av ITS. De metoder som använts har varit kvalitativa intervjuer samt litteraturstudier. De senare har använts för att få inblick i ämnet ITS och tjänster som finns eller är under utveckling inom området. Intervjuerna gjordes med personer som endera arbetar med forskning inom ITS eller är verksamma inom området samt med funktionshindrade bilförare. En kartläggning av tjänster har skett och resultatet av intervjuerna pekar på att det finns ett behov av ITS-tjänster för funktionshindrade bilförare. Delvis finns behov av tjänster som redan finns i dag, till exempel navigationssystem. Det finns även behov av andra tjänster som är under utveckling. Gemensamt för dessa tjänster är att de i vissa fall skulle behöva anpassas till de funktionshindrade med röstanpassning eller andra specifika anpassningar.

No description available.

A demanda por novas tecnologias, melhoria de segurança e conforto para veículos urbanos cresceu consideravelmente nos últimos anos, motivando a indústria na criação de sistemas destinados ao apoio de motoristas (ADAS - Advanced Driver Assistance Systems). Este fato contribuiu para o desenvolvimento de diversos sistemas embarcados na área automobilística destacando-se, à prevenção de colisão a pedestres por veículos. Através do avanço em diversas pesquisas, começaram a circular pelas ruas veículos com sistemas anticolisão e com navegação autônoma. Contudo, para alcançar objetivos cada vez mais desafiadores, os projetistas precisam de ferramentas que permitam unir tecnologias e conhecimentos de áreas distintas de forma eficiente. Nesse contexto, há uma demanda para a construção de sistemas que aumentem o nível de abstração da modelagem de projetos para o processamento de imagens em sistemas embarcados e assim, possibilitando uma melhor exploração do espaço de projetos. A fim de contribuir para minimizar este problema, este trabalho de pesquisa demonstra o desenvolvimento de um framework para coprojeto de hardware e software específico para a construção de sistemas ADAS que utilizam visão computacional. O Framework visa facilitar o desenvolvimento dessas aplicações permitindo a exploração o espaço de projeto (DSE - Design Space Exploration), e assim contribuindo para um ganho de desempenho no desenvolvimento de sistemas embarcados quando comparados à construção totalmente de um modo manual. Uma das características deste projeto é a possibilidade da simulação da aplicação antes da síntese em um sistema reconfigurável. Os principais desafios deste sistema foram relacionados à construção do sistema de intercomunicação entre os diversos blocos de Propriedade Intelectual (IP) e os componentes de software, abstraindo do usuário final inúmeros detalhes de hardware, tais como gerenciamento de memória, interrupções, cache, tipos de dados (ponto flutuante, ponto fixo, inteiros) e etc, possibilitando um sistema mais amigável ao projetista. / The demand for new technologies, enhanced security and comfort for urban cars has grown considerably in recent years prompting the industry to create systems designed to support drivers (ADAS - Advanced Driver Assistance Systems). This fact contributed to the development of many embedded systems in the automotive area among them, the pedestrians collision avoidance. Through the advancement in various research, began circulating through the streets vehicles with anti-collision systems and autonomous navigation. However, to achieve ever more challenging goals, designers need tools to unite technology and expertise from different areas efficiently. In this context, there is a demand for building systems that increase the level of abstraction of models of image processing for use in embedded systems enabling better design space exploration. To help minimize this problem, this research demonstrates a develop a specific framework for hardware/software codesign to build ADAS systems using computer vision. The framework aims to facilitate the development of applications, allowing better explore the design space, and thus contribute to a performance gain in the development of embedded systems in relation to building entirely in hardware. One of the requirements of the project is the possibility of the simulation of an application before synthesis on a reconfigurable system. The main challenges of this system were related to the construction of the intercommunication system between the various Intellectual Property (IP) blocks and the software components, abstracting from the end user numerous hardware details, such as memory management, interruptions, cache, types (Floating point, fixed point, integers) and so on, enabling a more user-friendly system for the designer.

A demanda por novas tecnologias, melhoria de segurança e conforto para veículos urbanos cresceu consideravelmente nos últimos anos, motivando a indústria na criação de sistemas destinados ao apoio de motoristas (ADAS - Advanced Driver Assistance Systems). Este fato contribuiu para o desenvolvimento de diversos sistemas embarcados na área automobilística destacando-se, à prevenção de colisão a pedestres por veículos. Através do avanço em diversas pesquisas, começaram a circular pelas ruas veículos com sistemas anticolisão e com navegação autônoma. Contudo, para alcançar objetivos cada vez mais desafiadores, os projetistas precisam de ferramentas que permitam unir tecnologias e conhecimentos de áreas distintas de forma eficiente. Nesse contexto, há uma demanda para a construção de sistemas que aumentem o nível de abstração da modelagem de projetos para o processamento de imagens em sistemas embarcados e assim, possibilitando uma melhor exploração do espaço de projetos. A fim de contribuir para minimizar este problema, este trabalho de pesquisa demonstra o desenvolvimento de um framework para coprojeto de hardware e software específico para a construção de sistemas ADAS que utilizam visão computacional. O Framework visa facilitar o desenvolvimento dessas aplicações permitindo a exploração o espaço de projeto (DSE - Design Space Exploration), e assim contribuindo para um ganho de desempenho no desenvolvimento de sistemas embarcados quando comparados à construção totalmente de um modo manual. Uma das características deste projeto é a possibilidade da simulação da aplicação antes da síntese em um sistema reconfigurável. Os principais desafios deste sistema foram relacionados à construção do sistema de intercomunicação entre os diversos blocos de Propriedade Intelectual (IP) e os componentes de software, abstraindo do usuário final inúmeros detalhes de hardware, tais como gerenciamento de memória, interrupções, cache, tipos de dados (ponto flutuante, ponto fixo, inteiros) e etc, possibilitando um sistema mais amigável ao projetista. / The demand for new technologies, enhanced security and comfort for urban cars has grown considerably in recent years prompting the industry to create systems designed to support drivers (ADAS - Advanced Driver Assistance Systems). This fact contributed to the development of many embedded systems in the automotive area among them, the pedestrians collision avoidance. Through the advancement in various research, began circulating through the streets vehicles with anti-collision systems and autonomous navigation. However, to achieve ever more challenging goals, designers need tools to unite technology and expertise from different areas efficiently. In this context, there is a demand for building systems that increase the level of abstraction of models of image processing for use in embedded systems enabling better design space exploration. To help minimize this problem, this research demonstrates a develop a specific framework for hardware/software codesign to build ADAS systems using computer vision. The framework aims to facilitate the development of applications, allowing better explore the design space, and thus contribute to a performance gain in the development of embedded systems in relation to building entirely in hardware. One of the requirements of the project is the possibility of the simulation of an application before synthesis on a reconfigurable system. The main challenges of this system were related to the construction of the intercommunication system between the various Intellectual Property (IP) blocks and the software components, abstracting from the end user numerous hardware details, such as memory management, interruptions, cache, types (Floating point, fixed point, integers) and so on, enabling a more user-friendly system for the designer.

L'utilisation d'algorithmes de vision permettrait d'élargir le domaine d'application des systèmes d'aide à la conduite à d'autres situations telles que : les scènes urbaines ou les conditions météorologiques dégradées. À cette fin, trois nouvelles applications sont étudiées dans cette thèse pour la pluie et les feux tricolores. La pluie est la condition météorologique dégradée la plus fréquente. Nous comparons les modèles physiques et photométriques existants pour la pluie et les gouttes de pluie. Lors d'une conduite en temps de pluie de jour, les gouttes sur le pare-brise diminuent considérablement la visibilité du conducteur. Lorsqu'elles sont vue par une camera embarquée standard celles-ci apparaissent défocalisées. Ainsi, nous proposons de détecter ces gouttes hors-focus en utilisant soit une approche par manque de gradients soit par l'évaluation locale du flou. Lors d'une conduite de nuit sous la pluie, ce sont les phares qui paradoxalement diminuent la visibilité car leur lumière est réfléchie par les gouttes vers le conducteur. Nous appuyant sur la conception d'un simulateur physique, nous proposons un éclairage adaptatif qui illuminerait la scène sans éclairer les gouttes qui tombent. Les résultats de notre simulateur et le premier prototype construit montre que l'idée avancée pourrait efficacement améliorer la visibilité générale d'une scène. D'autre part, nous étudions la détection et le suivi de gouttes de pluie à grande vitesse. Les feux tricolores ont un rôle crucial dans la compréhension des scènes urbaines. Bien qu'il existe déjà des systèmes de détection de feux tricolores, les algorithmes actuels ne fonctionnent que dans des conditions simples. Ainsi, nous avons développé un algorithme de détection de feux tricolores qui utilise une détection en niveau de gris des spots lumineux et une classification par reconnaissance de modèle. L'approche ainsi conçue est assez flexible pour détecter différents types de feux tricolores même avec une camera à faible dynamique. Notre proposition a été évaluée sur des séquences acquises en France, Chine et Suisse. / Vision algorithms can be used to expand the working range of the assistance systems so as to deal with urban scenes or degraded weathers. To this end, three novel applications are investigated in this thesis for both rain and traffic lights. Rain is the most frequent degraded weather condition. We review the various physics and photometry models for rain and raindrops, and highlight some of the misuses. When driving in daytime the raindrops on the windscreen lower the driver visibility. For standard on-board camera these drops appear as unfocused. Hence, we investigate the detection of unfocused raindrops using blur maps or lack of gradients with photometry. For nightime driving in rain, the headlights paradoxically reduce the visibility due to light reflected off of raindrops back toward the driver. Relying on a physic-based simulator, we propose to build an illumination device that would illuminate the scene without shining the falling particles. The performance of the simulator and a proof-of-concept prototype sustain that our idea can efficiently improve the overall scene visibility. Fast reactive drops detection and tracking is also investigated.To deal with urban scenes, traffic lights play a key role. Though traffic light recognition was attempted in the past, the existing algorithms can't handle complex scenarios. Hence, we have developed a traffic light recognition algorithm that uses a grayscale spot light detection and a template matching classification. Our approach is modular and capable of detecting various kind of traffic lights even when using a low-dynamic camera. We have evaluated our algorithm on sequences from France, China and Switzerland.