Autonomous vehicles or self-driving vehicles is an emerging concept, supported by the latest technological achievements. In the last decade, the interest has increased tremendously, leading to extensive research on the subject of automation. Today almost all major automotive manufacturers are looking at the adaptation of self-driving vehicles and the benefits they may possess. Potential benefits such as decreasing the number of traffic accidents and enhancing the traffic flow make it an interesting prospect for combating traffic issues related to urbanisation. As a ripple effect, the 5G ride initiative was taken to investigate the performance of a level 4 autonomous vehicle in the environment of Stockholm, Sweden. The vehicle features technologies such as 5G, Edge-Node, AI, SGX, vehicular protocols and a remote traffic tower. To understand the impact of the vehicle, this thesis sought to evaluate the 5G ride in regard to current sustainable indicators and measurements. The focal point was set to the United Nations Sustainable Development Goal 11.2. A model for evaluating the sustainability of technological solutions was applied to the 5G ride. It takes both the negative and positive outcomes, as well as the perception of time and the magnitude of the impact into account when assessing a technological solution. Conclusively, the results reveal several promising factors for sustainable development. However, this comes at the expense of the current system standards and its resources. / Autonoma fordon eller självkörande-fordon är ett framväxande koncept, möjliggjort av de senaste teknologiska framgångarna. Under det senaste årtiondet har intresset för självkörande fordon ökat oerhört mycket, vilket har lett till omfattande studier inom området. I dag undersöker nästan alla stora biltillverkare möjligheten till att adaptera självkörande teknik samt undersöka vilka förmågor de besitter. Detta inkluderar minskningen av trafikolyckor samt möjligheten till ett mer effektivt trafikflöde på vägar. Utifrån detta anses autonoma fordon vara ett intressant prospekt för att motarbeta trafikrelaterade problem som uppstått som följd av urbaniseringen. Som en kedjeeffekt utav detta togs ett initiativ som heter 5G ride, där ett självkörande fordon med nivå 4 färdigheter testades i Stockholms miljö. Fordonet är utrustat med tekniker som 5G, Edge-Node, AI, SGX, fordons protokoll samt ett avlägset trafiktorn. För att bygga upp en förståelse för fordonets påverkningsmöjligheter har vår studie utvärderat 5G-riden utifrån samhällsenliga hållbarhetsindikatorer. Den centrala utgångspunkten har varit United Nations hållbarhetsmål 11.2. En hållbarhetsmodell för att evaluera tekniska lösningar applicerades på 5G-riden. Modellen tar både hänsyn till positiva och negativa faktorer, tidsperspektiv samt magnituden utav impakten, när en teknisk lösning utvärderas. Sammanfattningsvis kunde flera lovande indikatorer identifieras, dock på bekostnad utav det nuvarande systemet och dess resurser.

<p>Object detection is an increasingly popular tool used in many fields, especially in the<br> development of autonomous vehicles. The task of object detections involves the localization<br> of objects in an image, constructing a bounding box to determine the presence and loca-<br> tion of the object, and classifying each object into its appropriate class. Object detection<br> applications are commonly implemented using convolutional neural networks along with the<br> construction of feature pyramid networks to extract data.<br> Another commonly used technique in the automotive industry is sensor fusion. Each<br> automotive sensor – camera, radar, and lidar – have their own advantages and disadvantages.<br> Fusing two or more sensors together and using the combined information is a popular method<br> of balancing the strengths and weakness of each independent sensor. Together, using sensor<br> fusion within an object detection network has been found to be an effective method of<br> obtaining accurate models. Accurate detections and classifications of images is a vital step<br> in the development of autonomous vehicles or self-driving cars.<br> Many studies have proposed methods to improve neural networks or object detection<br> networks. Some of these techniques involve data augmentation and hyperparameter opti-<br> mization. This thesis achieves the goal of improving a camera and radar fusion network by<br> implementing various techniques within these areas. Additionally, a novel idea of integrating<br> a third sensor, the lidar, into an existing camera and radar fusion network is explored in this<br> research work.<br> The models were trained on the Nuscenes dataset, one of the biggest automotive datasets<br> available today. Using the concepts of augmentation, hyperparameter optimization, sensor<br> fusion, and annotation filters, the CRF-Net was trained to achieve an accuracy score that<br> was 69.13% higher than the baseline</p>

For safe mobility, an autonomous vehicle must perceive the surroundings accurately. There are many perception tasks associated with understanding the local environment such as object detection, localization, and lane analysis. Object detection, in particular, plays a vital role in determining an object’s location and classifying it correctly and is one of the challenging tasks in the self-driving research area. Before employing an object detection module in autonomous vehicle testing, an organization needs to have a precise analysis of the module. Hence, it becomes crucial for a company to have an evaluation framework to evaluate an object detection algorithm’s performance. This thesis develops a comprehensive framework for evaluating and analyzing object detection algorithms, both 2D (camera images based) and 3D (LiDAR point cloud-based). The pipeline developed in this thesis provides the ability to evaluate multiple models with ease, signified by the key performance metrics, Average Precision, F-score, and Mean Average Precision. 40-point interpolation method is used to calculate the Average Precision. / För säker rörlighet måste ett autonomt fordon uppfatta omgivningen exakt. Det finns många uppfattningsuppgifter associerade med att förstå den lokala miljön, såsom objektdetektering, lokalisering och filanalys. I synnerhet objektdetektering spelar en viktig roll för att bestämma ett objekts plats och klassificera det korrekt och är en av de utmanande uppgifterna inom det självdrivande forskningsområdet. Innan en anställd detekteringsmodul används i autonoma fordonsprovningar måste en organisation ha en exakt analys av modulen. Därför blir det avgörande för ett företag att ha en utvärderingsram för att utvärdera en objektdetekteringsalgoritms prestanda. Denna avhandling utvecklar ett omfattande ramverk för utvärdering och analys av objektdetekteringsalgoritmer, både 2 D (kamerabilder baserade) och 3 D (LiDAR-punktmolnbaserade). Rörledningen som utvecklats i denna avhandling ger möjlighet att enkelt utvärdera flera modeller, betecknad med nyckelprestandamätvärdena, Genomsnittlig precision, F-poäng och genomsnittlig genomsnittlig precision. 40-punkts interpoleringsmetod används för att beräkna medelprecisionen.

The paper proposes the management of bus disruption (e.g. fleet failure) and maintain a resilient transportation system through a synergy between shared autonomous vehicles and the existing public transport system based on the organizational structure and demand characteristics. The methodology is applied to the region of Rennes (France) and its surroundings.

Nowadays, deep neural networks based software have been widely applied in many areas including safety-critical areas such as traffic control, medical diagnosis and malware detection, etc. However, the software engineering techniques, which are supposed to guarantee the functionality, safety as well as fairness, are not well studied. For example, some serious crashes of DNN based autonomous cars have been reported. These crashes could have been avoided if these DNN based software were well tested. Traditional software testing, debugging or repairing techniques do not work well on DNN based software because there is no control flow, data flow or AST(Abstract Syntax Tree) in deep neural networks. Proposing software engineering techniques targeted on DNN based software are imperative. In this thesis, we first introduced the development of SE(Software Engineering) for AI(Artificial Intelligence) area and how our works have influenced the advancement of this new area. Then we summarized related works and some important concepts in SE for AI area. Finally, we discussed four important works of ours. Our first project DeepTest is one of the first few papers proposing systematic software testing techniques for DNN based software. We proposed neuron coverage guided image synthesis techniques for DNN based autonomous cars and leveraged domain specific metamorphic relation to generate oracle for new generated test cases to automatically test DNN based software. We applied DeepTest to testing three top performing self-driving car models in Udacity self-driving car challenge and our tool has identified thousands of erroneous behaviors that may lead to potential fatal crash. In DeepTest project, we found that the natural variation such as spatial transformations or rain/fog effects have led to problematic corner cases for DNN based self-driving cars. In the follow-up project DeepRobust, we studied per-point robustness of deep neural network under natural variation. We found that for a DNN model, some specific weak points are more likely to cause erroneous outputs than others under natural variation. We proposed a white-box approach and a black-box approach to identify these weak data points. We implemented and evaluated our approaches on 9 DNN based image classifiers and 3 DNN based self-driving car models. Our approaches can successfully detect weak points with good precision and recall for both DNN based image classifiers and self-driving cars. Most of existing works in SE for AI area including our DeepTest and DeepRobust focus on instance-wise errors, which are single inputs that result in a DNN model's erroneous outputs. Different from instance-wise errors, group-level errors reflect a DNN model's weak performance on differentiating among certain classes or inconsistent performance across classes. This type of errors is very concerning since it has been found to be related to many real-world notorious errors without malicious attackers. In our third project DeepInspect, we first introduced the group-level errors for DNN based software and categorized them into confusion errors and bias errors based on real-world reports. Then we proposed neuron coverage based distance metric to detect group-level errors for DNN based software without requiring labels. We applied DeepInspect to testing 8 pretrained DNN models trained in 6 popular image classification datasets, including three adversarial trained models. We showed that DeepInspect can successfully detect group-level violations for both single-label and multi-label classification models with high precision. As a follow-up and more challenging research project, we proposed five WR(weighted regularization) techniques to repair group-level errors for DNN based software. These five different weighted regularization techniques function at different stages of retraining or inference of DNNs including input phase, layer phase, loss phase and output phase. We compared and evaluated these five different WR techniques in both single-label and multi-label classifications including five combinations of four DNN architectures on four datasets. We showed that WR can effectively fix confusion and bias errors and these methods all have their pros, cons and applicable scenario. All our four projects discussed in this thesis have solved important problems in ensuring the functionality, safety as well as fairness for DNN based software and had significant influence in the advancement of SE for AI area.

No description available.

Object detection has been a fundamental part of many emerging technologies, such as autonomous vehicles, robotics, and security. As deep learning is the main reason behind the leap of performance in object detection, it has mostly been associated with a post-processing step of non-maximum suppression (NMS) to reduce the number of resulting bounding boxes output from the network to, ideally, one box per object. As non-maximum suppression blindly suppress the overlap with a pre-defined threshold, it introduces the problem of suppressing false negatives in crowded scenes by choosing a high threshold, or vice versa. This problem is critical, especially in the autonomous vehicle industry, as this concerns the safety of passengers. The problem of the machine understanding whether these bounding boxes belong to the same object or two near-by objects is still not directly solvable. Although a lot of previous research tried to invent a new box-reduction method, every method has its own drawbacks while solving the problem. That is why, until now, many researchers are still using non‐maximum suppression. In this research, a literature review was carried out to determine the best NMS alternatives. Then, an approach for box reduction based on determinantal point process (DPP) was implemented. Furthermore, an evaluation pipeline was introduced for experimental analysis for the differences between NMS and DPP. Although NMS shows a better performance in terms of precision and recall, DPP chooses better fitting bounding boxes. / Objektdetektering har varit en grundläggande del av många nya tekniker, t.ex. autonoma fordon, robotik och säkerhet. Eftersom djupinlärning är den främsta orsaken till den stora prestandaskillnaden vid objektsdetektering har den oftast varit förknippad med ett efterbehandlingssteg med icke-maximal undertryckning (NMS) för att minska antalet resulterande avgränsande rutor som produceras av nätverket till, idealt sett, en ruta per objekt. Eftersom icke-maximal undertryckning blint undertrycker överlappningen med ett fördefinierat tröskelvärde, uppstår problemet med att undertrycka falskt negativa resultat i överfulla scener genom att välja ett högt tröskelvärde, eller tvärtom. Detta problem är kritiskt, särskilt inom industrin för autonoma fordon, eftersom det gäller passagerarnas säkerhet. Problemet med att maskinen ska förstå om dessa avgränsande rutor tillhör samma objekt eller två närliggande objekt är fortfarande inte direkt lösbart. Även om man i tidigare forskning har försökt hitta en ny metod för att reducera boxar, har varje metod sina egna nackdelar när den löser problemet. Det är därför som många forskare fram till nu fortfarande använder sig av icke-maximalt undertryckande. I denna forskning gjordes en litteraturstudie för att fastställa de bästa NMS-alternativen. Därefter implementerades en metod för boxförminskning baserad på determinant punktprocess (DPP). Dessutom infördes en utvärderingsledning för experimentell analys av skillnaderna mellan NMS och DPP. Även om NMS visar en bättre prestanda när det gäller precision och återkallande, väljer DPP bättre passande avgränsande lådor.

The development of autonomous driving in recent years has been in full swing and one of the aspects that Autonomous Vehicles (AVs) should always focus on is safety. Although the corresponding technology has gradually matured, and AVs have performed well in a large number of tests, people are still uncertain whether AVs can cope with all possible situations. This world is complex and ever-changing, experiencing countless disturbances every moment, and according to The Butterfly Effect, even the most insignificant small disturbance may set off a huge storm in the near future. If AVs really enter people’s daily lives, they will inevitably encounter many unexpected situations that have never been experienced before. Thus how to ensure that AVs can handle these well has become the most important issue at the moment. It is necessary to give the Automated Driving System (ADS) sufficient challenges during training and testing for acceptable safety and stability. However, dangerous and extreme driving scenarios in the real world are very rare, and it is also very expensive for such a test to be carried out in reality. Therefore, artificially creating a series of critical driving scenarios then training and testing the ADS in a simulation environment has become the current mainstream solution. This thesis project builds a complete framework for the automatic generation, simulation, and analysis of safety-critical driving scenarios. First, the specified scenarios and features are sequentially extracted from the naturalistic driving dataset through pre-defined rules; then a Density Estimation Model is adopted to learn the features, trying to find the distribution of the specified scenarios; after the distribution is obtained, synthetic driving scenarios can be obtained by sampling. Finally, visualize these synthetic scenarios via simulation for safety assessment and data analysis. / Utvecklingen av självkörande fordon har varit i full gång de senaste åre och en av aspekterna som självkörande alltid bör fokusera på är säkerheten. Även om motsvarande teknik gradvis har mognat, och självkörande har presterat bra i ett stort antal tester, är människor fortfarande osäkra på om självkörande klarar av alla möjliga situationer. Den här världen är komplex och ständigt föränderlig, upplever otaliga störningar varje ögonblick, och enligt The Butterfly Effect kan även den mest obetydliga lilla störningen sätta igång en enorm storm inom en snar framtid. Om självkörande verkligen kommer in i människors dagliga liv kommer de oundvikligen att möta många oväntade situationer som aldrig har upplevts tidigare. Så hur man säkerställer att självkörande kan hantera dessa väl har blivit den viktigaste frågan för tillfället. Det är nödvändigt att ge självkörande tillräckliga utmaningar underträning och testning för acceptabel säkerhet och stabilitet. Men farliga och extrema körscenarier i den verkliga världen är mycket sällsynta, och det är också mycket dyrt att genomföra ett sådant test i verkligheten. Att på konstgjord väg skapa en serie kritiska körscenarier och sedan träna och testa det automatiserade körsystemet i en simuleringsmiljö har därför blivit den nuvarande vanliga lösningen. Detta examensarbete bygger ett komplett ramverk för automatisk generering, simulering och analys av säkerhetskritiska körscenarier. Först extraheras de specificerade scenarierna och funktionerna sekventiellt från den naturalistiska kördatauppsättningen genom fördefinierade regler; sedan antas en densitetsuppskattningsmodell för att lära sig funktionerna och försöka hitta fördelningen av de specificerade scenarierna; efter att fördelningen erhållits kan syntetiska körscenarier erhållas genom provtagning. Slutligen, visualisera dessa syntetiska scenarier via simulering för säkerhetsbedömning och dataanalys.

Intersection Navigation plays a significant role in autonomous vehicle operation. This paper focuses on enhancing autonomous vehicle intersection navigation through advanced computer vision and Vehicle-to-Infrastructure (V2I) communication systems. The research unfolds in two phases. In the first phase, an approach utilizing YOLOv8s is proposed for precise traffic light detection and recognition, trained on the Small-Scale Traffic Light Dataset (S2TLD). The second phase establishes seamless connectivity between autonomous vehicles and traffic lights in a simulated Mississippi State University Autonomous Vehicle Simulation (MAVS) environment resembling a small city with multiple intersections. This V2I system enables the transmission of Signal Phase and Timing (SPaT) messages to vehicles, providing information on current traffic light phases and time until the next phase change which enables the vehicles to adjust their speed and behavior in real-time. The simulation demonstrates accurate traffic light detection, with vehicles receiving SPaT messages, showcasing the system’s effectiveness in a multi-intersection scenario.

Mobility is a topic of great interest in research and engineering since critical aspects such as safety, traffic efficiency, and environmental sustainability still represent wide open challenges for researchers and engineers. In this thesis, at first, we address the cooperative driving safety problem both from a centralized and decentralized perspective. Then we address the problem of optimal energy management of hybrid vehicles to improve environmental sustainability, and finally, we develop an intersection management systems for Connected Autonomous Vehicle to maximize the traffic efficiency at an intersection. To address the first two topics, we define a common framework. Both the cooperative safety and the energy management for Hybrid Electric Vehicle requires to model the driver behavior. In the first case, we are interested in evaluating the safety of the driver’s intentions, while in the second case, we are interested in predicting the future velocity profile to optimize energy management in a fixed time horizon. The framework is the Co-Driver, which is, in short, a bio-inspired agent able both to model and to imitate a human driver. It is based on a layered control structure based on the generation of atomic human-like longitudinal maneuvers that compete with each other like affordances. To address driving safety, the Co-Driver behaves like a safe driver, and its behavior is compared to the actual driver to understand if he/she is acting safely and providing warnings if not. In the energy management problem, the Co-Driver aims at imitating the driver to predict the future velocity. The Co-Driver generates a set of possible maneuvers and selects one of them, imitating the action selection process of the driver. At first, we address the problem of safety by developing and investigating a framework for Advanced Driving Assistance Systems (ADAS) built on the Co-Driver. We developed and investigated this framework in an innovative context of new intelligent road infrastructure, where vehicles and roads communicate. The infrastructure that allows the roads to interact with vehicles and the environment is the topic of a research project called SAFESTRIP. This project is about deploying innovative sensors and communication devices on the road that communicate with all vehicles. Including vehicles that are equipped with Vehicle-To-Everything (V2X) technology and vehicles that are not, using an interface (HMI) on smart-phones. Co-Driver-based ADAS systems exploit connections between vehicles and (smart) roads provided by SAFESTRIP to cover several safety-critical use cases: pedestrian protection, wrong-way vehicles on-ramps, work-zones on roads and intersections. The ADAS provide personalized warning messages that account for the adaptive driver behavior to maximize the acceptance of the system. The ability of the framework to predict human drivers’ intention is exploited in a second application to improve environmental sustainability. We employ it to feed with the estimated speed profile a novel online Model Predictive Control (MPC) approach for Hybrid Electric Vehicles, introducing a state-of-the-art electrochemical model of the battery. Such control aims at preserving battery life and fuel consumption through equivalent costs. We validated the approach with actual driving data used to simulate vehicles and the power-train dynamics. At last, we address the traffic efficiency problem in the context of autonomous vehicles crossing an intersection. We propose an intersection management system for Connected Autonomous Vehicles based on a bi-level optimization framework. The motion planning of the vehicle is provided by a simplified optimal control problem, while we formulate the intersection management problem (in terms of order and timing) as a Mixed Integer Non-Linear Programming. The latter approximates a linear problem with a powerful piecewise linearization technique. Therefore, thanks to this technique, we can bound the error and employ commercial solvers to solve the problem (fast enough). Finally, this framework is validated in simulation and compared with the "Fist-Arrived First-Served" approach to show the impact of the proposed algorithm.