Vehicle platooning is an advanced vehicle maneuver that allows for the simultaneous control of several vehicles traveling on the roadway cite{al2010experimental}. Automated platoons, when activated in tractor trailer convoys, have a high potential of increasing the fuel efficiency and improving the utilization of roadways by allowing more vehicles to share the road at the same time. The increased fuel efficiency translates to lower cost on goods and motivates a more environmentally friendly and sustainable economy. In order to achieve the promised fuel savings from vehicle platooning, the vehicles need to follow each other at shorter headways than in typical driving scenarios. The reduced separation distance between the lead and follow vehicle reduces visibility and the reaction time available for the follow vehicle; this renders most modern Active Driver Assist Systems (ADAS) ineffective since they are not designed for operation in such short headway conditions. The focus of this work is related to understanding and improving the failures of Lane Keep Assist (LKA) systems in the follow vehicles of a platoon. In this work, the source of lane detection degradation when using a monocular forward facing camera in short headway platooning is identified. Furthermore, a novel lane augmentation algorithm is proposed to improve the lane detection capability of follow vehicles in a platoon. The lane augmentation process utilizes a longitudinal transformation of lane parameters from the lead to the follow vehicles. The transformation utilizes an accurate understanding of the relative spatial position and orientation of the two vehicles. The transformation also requires a reliable communication system between the two vehicles such as a Vehicle-to-Vehicle (V2V) module. The work presented in this thesis develops theory, simulation and verification using real world data of the proposed cooperative lane augmentation. The results of this work indicate that it is possible to improve vehicle platooning performance by distributing the required sensing across multiple agents of the platoon. / Master of Science / Vehicle platooning is an advanced vehicle maneuver that allows for the simultaneous control of several vehicles traveling on the roadway cite{al2010experimental}. Automated platoons, when activated in tractor trailer convoys, have a high potential of increasing the fuel efficiency and improving the utilization of our roadways by allowing more vehicles to share the road at the same time. The increased fuel efficiency translates to lower cost on goods and motivates a more environmentally friendly and sustainable economy. In order to achieve the promised fuel savings from vehicle platooning, the vehicles need to follow each other at closer distances (headway) than in typical driving scenarios. The reduced separation distance between the lead and follow vehicle reduces visibility and the reaction time available for the follow vehicle; this renders most modern Active Driver Assist Systems (ADAS) ineffective since they are not designed for operation in such short headway conditions. The focus of this work is related to understanding and improving the failures of Lane Keep Assist (LKA) systems - the automated system used to keep the vehicle in the center of the lane - in the follow vehicles of a platoon. In the proposed scenario, the LKA uses a single forward facing camera to detect the lane lines ahead of the vehicle. The detected lanes serve as inputs to the lateral position (steering) controller in order to keep the vehicle in the center of the lane. In this work, the source of lane detection degradation in a follow vehicle of a short headway platoon is identified. Furthermore, a novel cooperative lane detection algorithm is proposed to improve the lane detection capability of the follow vehicles. The proposed algorithm utilizes lane information transformed from the lead to follow vehicle frame. The transformation utilizes the relative spatial position and orientation of the two vehicles. Additionally, a reliable communication protocol between the vehicles is required to transport the lane information. The work presented in this thesis develops theory, simulation and verification using real world data of the proposed algorithm. The results of this work indicate that lane keeping performance in a platoon can be improved using cooperative lane detection.

Based on .Net Framework4.0 development platform and Visual C# language, this thesis presents various methods of performing lane detection and preceding vehicle detection/tracking with code optimization and acceleration to reduce the execution time. The thesis consists of two major parts: vehicle detection and tracking. In the part of detection, driving lanes are identified first and then the preceding vehicles between the left lane and right lane are detected using the shadow information beneath vehicles. In vehicle tracking, three-pass search method is used to find the matched vehicles based on the detection results in the previous frames. According to our experiments, the preprocessing (including color-intensity conversion) takes a significant portion of total execution time. We propose different methods to optimize the code and speed up the software execution using pure C # pointers, OPENCV, and OPENCL etc. Experimental results show that the fastest detection/tracking speed can reach more than 30 frames per second (fps) using PC with i7-2600 3.4Ghz CPU. Except for OPENCV with execution rate of 18 fps, the rest of methods have up to 28 fps processing rate of almost the real-time speed. We also add the auxiliary vehicle information, such as preceding vehicle distance and vehicle offset warning.

As important components of intelligent transportation system, lane detection and tracking (LDT) and lane departure warning (LDW) systems have attracted great interest from the computer vision community over the past few years. Conversely, lane markings recognition (LMR) systems received surprisingly little attention. This thesis proposed a real-time lane assisting framework for intelligent vehicles, which consists of a comprehensive module and simplified module. To the best of our knowledge, this is the first parallel architecture that considers not only lane detection and tracking, but also lane marking recognition and departure warning. A lightweight version of the Hough transform, PPHT is used for both modules to detect lines. After detection stage, for the comprehensive module, a novel refinement scheme consisting of angle threshold and segment linking (ATSL) and trapezoidal refinement method (TRM) takes shape and texture information into account, which significantly improves the LDT performance. Also based on TRM, colour and edge informations are used to recognize lane marking colors (white and yellow) and shapes (solid and dashed). In the simplified module, refined MSER blobs dramatically simplifies the preprocessing and refinement stage, and enables the simplified module performs well on lane detection and tracking. Several experiments are conducted in highway and urban roads in Ottawa. The detection rate of the LDT system in comprehensive module average 95.9% and exceed 89.3% in poor conditions, while the recognition rate depends on the quality of lane paint and achieves an average accuracy of 93.1%. The simplified module has an average detection rate of 92.7% and exceeds 84.9% in poor conditions. Except the conventional experimental methods, a novel point cluster evaluation and pdf analysis method have been proposed to evaluate the performance of LDT systems, in terms of the stability, accuracy and similarity to Gaussian distribution.

Machine learning is a powerful method to handle the self-driving problem. Researchers use machine learning to construct a neural network and train it to drive the car. A self-driving car is a safety-critical system. However, the neural network is not necessarily reliable. The output of a neural network can be easily influenced by many factors, such as the quality of training data and the runtime environment. Also, it takes time for the neural network to generate the output. That is, the self-driving car may not respond in time. Such weaknesses will increase the risk of accidents. In this thesis, considering the safety of self-driving cars, we apply a delay-aware shielding mechanism to the neural network to protect the self-driving car. Our approach is an improvement based on previous research on runtime safety enforcement for general cyber-physical systems that did not consider the delay to generate the output. Our approach contains two steps. The first is to use formal language to specify the safety properties of the system. The second step is to synthesize the specifications into a delay-aware enforcer called the shield, which enforces the violated output to satisfy the specifications during the whole delay. We use a lane keeping system as a small but representative case study to evaluate our approach. We utilize an end-to-end neural network as a typical implementation of such a lane keeping system. Our shield supervises those outputs of the neural network and verifies the safety properties during the whole delay period with a prediction. The shield can correct it if a violation exists. We use a 1/16 scale truck and construct a curvy lane to test our approach. We conduct the experiments both on a simulator and a real road to evaluate the performance of our proposed safety mechanism. The result shows the effectiveness of our approach. We improve the safety of a self-driving car and we will consider more comprehensive driving scenarios and safety features in the future. / Master of Science / Self-driving cars is a hot topic nowadays. Machine learning is a popular method to achieve self-driving cars. Machine learning constructs a neural network, which imitates a human driver's behavior to drive the car. However, a neural network is not necessarily reliable. Many things can mislead the neural network into making wrong decisions, such as insufficient training data or a complex driving environment. Thus, we need to guarantee the safety of self-driving cars. We are inspired to use formal language to specify the safety properties of the self-driving system. A system should always follow those specifications. Then the specifications are synthesized into an enforcer called the shield. When the system's output violates the specifications, the shield will modify the output to satisfy the specifications. Nevertheless, there is a problem with state-of-the-art research on specifications. When the specifications are synthesized into a shield, it does not consider the delay to compute the output. As a result, the specifications may not be always satisfied during the period of the delay. To solve such a problem, we propose a delay-aware shielding mechanism to continually protect the self-driving system. We use a lane keeping system as a small self-driving case study. We evaluate the effectiveness of our approach both on the simulation platform and the hardware platform. The experiments show that the safety of our self-driving car is enhanced. We intend to study more comprehensive driving scenarios and safety features in the future.

In Intelligent Transportation Systems, Advanced Vehicle Control and Safety System are one of the most important researches around the world. AVCSS is a technique applied on vehicle and is composed by sensor, computer, communication, and control. In order to keep driver safe, the technique covered Collision Avoidance, Longitudinal Automated Control, Lateral Automated Control, Automated Parking, etc, and Collision Avoidance, Longitudinal Automated Control, Lateral Automated Control are most important. This thesis implemented the Lateral Automated Control by using a CCD camera to extract the road environment. And I presented and analyzed lane detection about structured road and unstructured road. The structured road stands for its obvious lane mark such as general road and freeway; and the unstructured road stands for its unobvious lane mark or without lane mark such as country road and campus road. Because of the characteristic of lane mark, the structured road is easier to detect, and there were less research about unstructured road around the world. So this thesis focused on the unstructured lane detection, and implemented multi-system on DSP (Digital Signal Process). Finally, we applied intelligent control system to vehicle and successfully guided the vehicle in structure and unstructured road.

No description available.

Computer vision and image processing systems are ubiquitous in automotive domain and manufacturing industry. Lane detection warning systems has been an elementary part of the modern automotive industry. Due to the recent progress in the computer vision and image processing methods, economical and flexible use of computer vision is now pervasive and computing with images is not just for the realm of the science, but also for the arts and social science and even for hobbyists. Image processing is a key technology in automotive industry, even now there is hardly a single manufacturing process that is thinkable without imaging. The applications of image processing and computer vision methods in embedded systems platform, is an ongoing research area since many years. OpenCV, an open-source computer vision library containing optimized algorithms and methods for designing and implementing applications based on video and image processing techniques. These method are organized in the form of modules for specific field including, user-graphic interface, machine learning, feature extraction etc [43]. Vision-based automotive application systems become an important mechanism for lane detection and warning systems to alert a driver about the road in localization of the vehicle [1]. In automotive electronic market, for lane detection problem, vision-based approaches has been designed and developed using different electronic hardware and software components including wireless sensor, camera module, Field-Programmable Gate Array (FPGA) based systems, GPU and digital signal processors (DSP) [13]. The software module consists on the top of real-time operating systems and hardware description programming language including Verilog, or VHDL. One of the most time critical task of vision based systems is to test system applications in real physical environment with wide variety of driving scenarios and validating the whole systems as per the automotive industry standards. For validating and testing the advanced driver assistance systems, there are some commercial tools available including Assist ADTF from Elektrobit, EB company [43]. In addition to the design and strict real-time requirements for advanced driver assistance systems applications based on electronic components and embedded platform, the complexity and characteristics of the implemented algorithms are two parameters that need to be taken into consideration choosing hardware and software component [13]. The development of vision-based automotive application, based on alone electronic and micro-controller is not a feasible solution approach [35] [13] and GPU based solution are attractive but has many other issues including power consumption. In this thesis project, image and video processing module is used from OpenCV library for road lane detection problems. In proposed lane detection methods, low-level image processing algorithms and methods are used to extract relevant information for lane detection problem by applying contrast analysis at pixel level intensity values. Furthermore, the work at hand presents different approaches for solving relevant partial problems in the domain of lane detection. The aim of the work is to apply contrast analysis based on low-level image processing methods to extract relevant lane model information from the grid of intensity values of pixel elements available in image frame. The approaches presented in this project work are based on contrast analysis of binary mask image frame extracted after applying range threshold. A set of points, available in an image frame, based lane feature models are used for detecting lanes on color image frame captured from video. For the performance measurement and evaluation, the proposed methods are tested on different systems setup, including Linux, Microsoft Windows, CodeBlocks, Visual Studio 2012 and Linux based Rasbian-Jessie operating systems running on Intel i3, AMD A8 APU, and embedded systems based (Raspberry Pi 2 Model B) ARM v7 processor respectively.

The interest for autonomous driving assistance, and in the end, self-driving cars, has increased vastly over the last decade. Automotive safety continues to be a priority for manufacturers, politicians and people alike. Visual-based systems aiding the drivers have lately been boosted by advances in computer vision and machine learning. In this thesis, we evaluate the concept of an end-to-end machine learning solution for detecting and classifying road lane markings, and compare it to a more classical semantic segmentation solution. The analysis is based on the frame-by-frame scenario, and shows that our proposed end-to-end system has clear advantages when it comes detecting the existence of lanes and producing a consistent, lane-like output, especially in adverse conditions such as weak lane markings. Our proposed method allows the system to predict its own confidence, thereby allowing the system to suppress its own output when it is not deemed safe enough. The thesis finishes with proposed future work needed to achieve optimal performance and create a system ready for deployment in an active safety product.

Autonomous driving is an area of increasing investment for researchers and auto manufacturers. Integration has already begun for self-driving cars in urban environments. An essential aspect of navigation in these areas is the ability to sense and follow lane markers. This thesis focuses on the development of a vision-based control platform using lane detection to control a full-sized electric vehicle with only a monocular camera. An open-source, integrated solution is presented for automation of a stock vehicle. Aspects of reverse engineering, system identification, and low-level control of the vehicle are discussed. This work also details methods for lane detection and the design of a non-linear vision-based control strategy.

In modern cars, advanced driver assistance systems are used to aid the driver and increase the automobile safety. An example of such a system is the night vision system designed to detect and warn for pedestrians in danger of being hit by the car. To determine if a warning should be given when a pedestrian is detected, the system requires a prediction of the future path of the car for up to four seconds ahead in time. In this master's thesis, a new path prediction algorithm based on satellite positioning and a digital map database has been developed. The algorithm uses an extended Kalman filter to get an accurate estimate of the current position and heading direction of the car. The estimate is then matched to a position in the map database and the possible future paths of the vehicle are predicted using the road network. The performance of the path prediction algorithm has been evaluated on recorded night vision sequences corresponding to 15 hours of driving. The results show that map-based path prediction algorithms are superior to dead-reckoning methods for longer time horizons. It has also been investigated whether vision-based lane detection and tracking can be used to improve the path prediction. A prediction method using lane markings has been implemented and evaluated on recorded sequences. Based on the results, the conclusion is that lane detection can be used to support a path prediction system when lane markings are clearly visible.