Platooning is a method where a chain of vehiclesdrive with small inter-vehicular distances. The many benefitsof autonomous platooning includes improved fuel economy,less congestion and safer transportation. To create a safe andfunctional platoon the operational software needs to be able tohandle various road surfaces without the risk of a crash. Thisreport is aiming to improve the safety of a platoon by includingcommunication of data between vehicles in the chain. Specificallythe focus has been on transferring information about the tireslip, to model a cooperative adaptive cruise control (C-ACC)and combine the two. A system was designed using the dynamicsfor a quarter-car model and then connected to a controller and aplatoon of four vehicles. Simulations of when the leading vehiclebraked hard on two different road surfaces with and withoutthe slip awareness was conducted. The tire slip awareness in thecontroller consisted of proportional control on the error and alow-pass filter. The simulations showed that the inclusion of thetire slip in the controller improved the platooning performance,in the sense that the inter-vehicle distance could be contained.It was also shown the controller could be tuned so that the slipratios were limited. / Konvojkörning är en metod där en kedjaav fordon åker med små interna distanser. De många fördelarnamed förarlösa konvojer inkluderar förbättrad bränsleförbukning, mindre trafik och säkrare transportering. För atten säker och funktionell konvoj ska kunna skapas krävs detatt mjukvaran kan handskas med varierande vägunderlag utanrisk att krocka. Den här rapporten siktar på att förbättrasäkerheten i konvojkörning genom att överföra data till andrafordon i konvojkedjan. Speciellt har fokuset legat på överförainformation om däcksliring, att modellera en kooperative adaptivfarthållare (C-ACC) och sedan kombinera de två. Ett systemdesignades genom att använda dynamiken av en fjärdedelsbil och sen ansluta modellen till en konvoj med fyra fordon.Simulationer av när det ledande fordonet tvärbromsade på olikavägunderlag med och utan däcksliringsinfromation genomfördes.Däckslirnings i regulatorn bestod av proportionerlig kontroll påfelet och ett lågpassfilter. Simulationerna visade att inkluderingenav däcksliringsinformation i regulatorn förbättrar konvojensprestanda, på så sätt att de interna distanserna kan hanteras.Det kunde också påvisas att kontrollern kunde kalibreras så attslirningen begränsades. / Kandidatexjobb i elektroteknik 2022, KTH, Stockholm

Multitask Learning (MTL) was conceived as an approach to improve thegeneralization ability of machine learning models. When applied to neu-ral networks, multitask models take advantage of sharing resources forreducing the total inference time, memory footprint and model size. Wepropose MTL as a way to speed up deep learning models for applicationsin which multiple tasks need to be solved simultaneously, which is par-ticularly useful in embedded, real-time systems such as the ones foundin autonomous cars or UAVs.In order to study this approach, we apply MTL to a Computer Vi-sion problem in which both Object Detection and Semantic Segmenta-tion tasks are solved based on the Single Shot Multibox Detector andFully Convolutional Networks with skip connections respectively, usinga ResNet-50 as the base network. We train multitask models for twodifferent datasets, Pascal VOC, which is used to validate the decisionsmade, and a combination of datasets with aerial view images capturedfrom UAVs.Finally, we analyse the challenges that appear during the process of train-ing multitask networks and try to overcome them. However, these hinderthe capacity of our multitask models to reach the performance of the bestsingle-task models trained without the limitations imposed by applyingMTL. Nevertheless, multitask networks benefit from sharing resourcesand are 1.6x faster, lighter and use less memory compared to deployingthe single-task models in parallel, which turns essential when runningthem on a Jetson TX1 SoC as the parallel approach does not fit intomemory. We conclude that MTL has the potential to give superior per-formance as far as the object detection and semantic segmentation tasksare concerned in exchange of a more complex training process that re-quires overcoming challenges not present in the training of single-taskmodels.

Autonomous driving is the concept of a vehicle that operates in traffic without instructions from a driver. A major challenge for such a system is to provide a comprehensive, accurate and compact scene model based on information from sensors. For such a model to be comprehensive it must provide 3D position and semantics on relevant surroundings to enable a safe traffic behavior. Such a model creates a foundation for autonomous driving to make substantiated driving decisions. The model must be compact to enable efficient processing, allowing driving decisions to be made in real time. In this thesis rectangular objects (The Stixelworld) are used to represent the surroundings of a vehicle and provide a scene model. LIDAR and semantic segmentation are fused in the computation of these rectangles. This method indicates that a dense and compact scene model can be provided also from sparse LIDAR data by use of semantic segmentation. / Fullt självkörande fordon behöver inte förare. Ett sådant fordon behöver en precis, detaljerad och kompakt modell av omgivningen baserad på sensordata. Med detaljerad avses att modellen innefattar all information nödvändig för ett trafiksäkert beteende. Med kompakt avses att en snabb bearbetning kan göras av modellen så att fordonet i realtid kan fatta beslut och manövrera i trafiken. I denna uppsats tillämpas en metod där man med rektangulära objekt skapar en modell av omgivningen. Dessa beräknas från LIDAR och semantisk segmentering. Arbetet indikerar att med hjälp av semantisk segmentering kan en tät, detaljerad och kompakt modell göras även från glesa LIDAR-data.

Motion planning consists of finding a feasible path of an object between an initial state and a goal state, and commonly constitutes a sub-system of a larger autonomous system. Motion planners that utilize sampling-based algorithms create an implicit representation of the search space via sampling said search space. Autonomous systems that rely on real-time motion planning benefit from the ability of these algorithms to quickly compute paths that are optimal or near optimal. For sampling-based motion planning algorithms, the sampling strategy greatly affects the convergence speed of finding these paths, i.e., how the sampling distribution is shaped within the search space. In baseline approaches, the samples may be drawn with uniform probability over this space. This thesis project explores a learning-based approach that can utilize experience from previous successful motion plans to provide useful information in novel planning scenarios, as a means of improvement over conventional motion planning methods. Specifically, the focus has been on learning the sampling distributions in both the state space and the control space of an autonomous ground vehicle. The innovatory parts of this work consist of (i) learning the control space sampling distributions, and (ii) learning said distributions for a tractor-trailer system. At the core of the method is an artificial neural network consisting of a conditional variational autoencoder. This artificial neural network is capable of learning suitable sampling distributions in both the state space and control space of a vehicle in different planning scenarios. The method is tested in four different environments and for two kinds of vehicles. Evaluation is partly done by comparison of results with a conventional motion planning algorithm. These evaluations indicates that the artificial neural network can produce valuable information in novel planning scenarios. Future work, primarily on how the artificial neural network may be applied to motion planning algorithms, is necessary to draw further conclusions. / Rörelseplanering består av att hitta en genomförbar bana för ett objekt mellan ett initialtillstånd och ett måltillstånd, och utgör vanligtvis ett delsystem av ett större autonomt system. Rörelseplanerare som använder provtagningssbaserade algoritmer skapar en implicit representation av sökutrymmet via provtagning av sökutrymmet. Autonoma system som förlitar sig på rörelseplanering i realtid drar nytta av dessa algoritmers förmåga att snabbt beräkna banor som är optimala eller nästan optimala. För provtagningssbaserade rörelseplaneringsalgoritmer påverkar provtagningsstrategin i hög grad konvergenshastigheten för att hitta dessa vägar, dvs. hur provtagningsfördelningen är formad inom sökutrymmet. I standardmetoder kan stickproven dras med jämn sannolikhet över detta utrymme. Detta examensarbete utforskar en lärande-baserat metod som kan utnyttja erfarenheter från tidigare lyckade rörelseplaner för att tillhandahålla användbar information i nya planeringsscenarier, som ett medel för förbättring jämfört med konventionella rörelseplaneringsmetoder. Specifikt har fokus legat på att lära sig provtagningssfördelningarna i både tillståndsrummet och styrsignals-rummet för ett autonomt markfordon. De nyskapande delarna av detta arbete består av att (i) lära sig kontrollutrymmessamplingsfördelningarna, och (ii) inlärning av nämnda provtagningsfördelningarna för ett traktor-släpsystem. Kärnan i metoden är ett artificiellt neuralt nätverk bestående av en conditional variational autoencoder. Detta artificiella neurala nätverk är kapabelt att lära sig lämpliga provtagningsfördelningar i både tillståndsrummet och kontrollrummet för ett fordon i olika planeringsscenarier. Metoden testas i fyra olika miljöer och för två olika av fordon. Utvärdering görs delvis genom jämförelse av resultat med en konventionell rörelseplaneringsalgoritm. Dessa utvärderingar tyder på att det artificiella neurala nätverket kan producera värdefull information i nya planeringsscenarier. Mer forskning, i första hand med hur det artificiella neurala nätverket kan tillämpas på rörelseplaneringsalgoritmer, är nödvändigt för att dra ytterligare slutsatser.

Precise and robust automotive localization is a must for autonomous vehicles. Radar is a cheap and robust sensor, and this project aimed to find a method to use automotive radar to localize globally. By using radar data to build occupancy grids based on other state-of-the-art radar localization methods, and applying image correlation techniques, a localization precision of below 20 cm could be achieved, delivering poses at frequency higher than 0.5 Hz along with a characterization of the uncertainty. By using an improved sensor model for the occupancy grid mapping, filtering the radar data, and using image correlation in the Fourier domain. The presented results are better than the state-of-the-art radar localization methods, both in terms of precision and frequency, however not in terms of heading estimation. The work provides a foundation for future investigations and improvements of radar as a sensor for localization. / Exakt och robust fordonslokalisering är ett måste för framtidens autonoma fordon. Radar är billig och robust sensor, och detta projekt utfördes i syfte att hitta en metod för att använda bilradar för att lokalisera globalt. Genom att använda radardata för att bygga occupancyg grids baserade på de senaste bästa radarlokaliseringsmetoder och tillämpa bildkorrelationstekniker, kunde en lokaliseringsprecision bättre än 20 cm uppnås, vilket ger positioner vid frekvens högre än 0,5 Hz tillsammans osäkerhetens karaktärisering. Genom att använda en förbättrad sensormodell för kartläggning av occupancy grids, filtrera radardata och använda bildkorrelation i Fourier- domänen. De presenterade resultaten är bättre än de senaste metoderna för radarlokalisering, både när det gäller precision och frekvens, men inte när det gäller riktning. Arbetet utgör en grund för framtida undersökningar av radar som en sensor för lokalisering.

Reducing the size of a neural network whilst maintaining a comparable performance is an important problem to be solved since the constrictions on resources of small devices make it impossible to deploy large models in numerous real-life scenarios. A prominent example is autonomous driving, where computer vision tasks such as object detection and semantic segmentation need to be performed in real time by mobile devices. In this thesis, the knowledge and spherical knowledge distillation techniques are utilized to train a small model (PSPNet50) under the supervision of a large model (PSPNet101) in order to perform semantic segmentation of urban scenes. The importance of the distillation hyperparameters is studied first, namely the influence of the temperature and the weights of the loss function on the performance of the distilled model, showing no decisive advantage over the individual training of the student. Thereafter, distillation is performed utilizing a pretrained student, revealing a good improvement in performance. Contrary to expectations, the pretrained student benefits from a high learning rate when training resumes under distillation, especially in the spherical knowledge distillation case, displaying a superior and more stable performance when compared to the regular knowledge distillation setting. These findings are validated by several experiments conducted using the Cityscapes dataset. The best distilled model achieves 87.287% pixel accuracy and a 42.0% mean Intersection-Over-Union value (mIoU) on the validation set, higher than the 86.356% pixel accuracy and 39.6% mIoU obtained by the baseline student. On the test set, the official evaluation obtained by submission to the Cityscapes website yields 42.213% mIoU for the distilled model and 41.085% for the baseline student. / Att minska storleken på ett neuralt nätverk med bibehållen prestanda är ett viktigt problem som måste lösas, eftersom de begränsade resurserna i små enheter gör det omöjligt att använda stora modeller i många verkliga situationer. Ett framträdande exempel är autonom körning, där datorseende uppgifter som objektsdetektering och semantisk segmentering måste utföras i realtid av mobila enheter. I den här avhandlingen används tekniker för destillation av kunskap och sfärisk kunskap för att träna en liten modell (PSPNet50) under övervakning av en stor modell (PSPNet101) för att utföra semantisk segmentering av stadsscener. Betydelsen av hyperparametrarna för destillation studeras först, nämligen temperaturens och förlustfunktionens vikter för den destillerade modellens prestanda, vilket inte visar någon avgörande fördel jämfört med individuell träning av eleven. Därefter utförs destillation med hjälp av en utbildad elev, vilket visar på en god förbättring av prestanda. Tvärtemot förväntningarna har den utbildade eleven en hög inlärningshastighet när utbildningen återupptas under destillation, särskilt i fallet med sfärisk kunskapsdestillation, vilket ger en överlägsen och stabilare prestanda jämfört med den vanliga kunskapsdestillationssituationen. Dessa resultat bekräftas av flera experiment som utförts med hjälp av datasetet Cityscapes. Den bästa destillerade modellen uppnår 87.287% pixelprecision och ett 42.0% medelvärde för skärning över union (mIoU) på valideringsuppsättningen, vilket är högre än de 86.356% pixelprecision och 39.6% mIoU som uppnåddes av grundstudenten. I testuppsättningen ger den officiella utvärderingen som gjordes på webbplatsen Cityscapes 42.213% mIoU för den destillerade modellen och 41.085% för grundstudenten.

Vehicle Action Intention Prediction plays a more and more crucial role in automated driving and traffic safety. It allows automated vehicles to comprehend the other road participants’ current actions, and foresee the upcoming actions, which can significantly reduce the likelihood of traffic accidents, so as to enhance overall road safety. Meanwhile, by anticipating other vehicles’ movements on the road, the ego vehicle can plan its velocity and trajectory in advance, and make more smooth and finer adjustments during the whole driving process, contributing to a more safe and efficient traffic. Furthermore, the intention prediction enables vehicles to respond proactively rather than reactively in traditional ADAS (Advanced Driver Assistance Systems), such as AEB (Automatic Emergency Braking), which facilitates a more preventive and early intervention approach to traffic safety. In normal conditions, traffic behavior is controlled by traffic rules. This thesis explores vehicle behavior using intention prediction models in scenarios where there are no traffic rules. At hand, we have a unique dataset containing vehicle trajectories, collected from 2 cameras installed overhead on a 1-lane narrowing street, where the vehicles need to negotiate their right of way. After pre-processing these data to form specific input structures, we use different classifier models including both traditional methods and deep learning methods to make vehicle action intention predictions. The data was organized in 3-second windows and contained vehicle position and distance to the center of the intersection along with the speed of both vehicles. We compared and evaluated the model performances and found that MLPs (Multi-Layer Perceptrons) and LSTM (Long Short Term Memory) yield the best performance. Furthermore, a feature selection method and features’ importance analysis are also applied to explore which variables influence the model most in order to explain the internal principle of the prediction model. It was found that close to the narrowing street the first and last samples of the position and distance in the time window and the last sample of the speed of both vehicles were found to influence the model performance the most. Further away from the narrowing street, the first and last samples of the position of the vehicle have a higher influence on the model.

The development of autonomous vehicles is progressing rapidly through extensive efforts by the automotive industry and researchers. One of the key factors for the adoption of autonomous driving technology is motion comfort and the ability to engage in non-driving tasks such as reading, socialising, and relaxing without experiencing motion sickness while travelling. Therefore, for the full success of autonomous vehicles, it is necessary to learn how to design and control the vehicles to mitigate motion sickness for the passengers.  This thesis aims to investigate methods for prediction of motion sickness in autonomous vehicles and how to mitigate it using vehicle dynamics based solutions, with an emphasis on trajectory planning. As a first step, a review and evaluation of existing motion sickness prediction methods were performed. The review highlighted the importance of accurate motion sickness assessment in the early phases of autonomous vehicle design. Two chosen methods (ISO 2631-based and sensory conflict theory-based) were evaluated to estimate individual motion sickness feelings using measured data and subjective assessment ratings from field tests. It can be concluded that the methods can be adjusted to predict individual motion sickness feelings, as shown by the comparison with the experimental data. To continue the work, a review of vehicle dynamics based motion sickness mitigation methods for autonomous vehicles was performed. Several chassis control strategies in literature like active suspension, rear-wheel steering and torque distribution have demonstrated the potential help to reduce motion sickness. Another effective approach to mitigate motion sickness in autonomous vehicles is to regulate vehicle speed and path using trajectory planning which was chosen to be further investigated. The trajectory planning was constructed as an optimisation problem where there is a trade-off between motion sickness and manoeuvre time. The impact of the trajectory planning algorithm to reduce motion sickness was analysed by simulating two different vehicle models in specific test manoeuvres. The results indicate that driving style has a significant influence on motion sickness and trajectory planning algorithms should be carefully designed to find a good balance between journey time and motion sickness. The research presented in this thesis contributes to the development of methodologies for predicting and mitigating motion sickness in autonomous vehicles, helping to achieve the goal of ensuring their overall success. / Utvecklingen av autonoma fordon går snabbt framåt tack vare omfattande insatser från fordonsindustrin och forskare. En av de viktigaste faktorerna för införandet av teknik för autonom körning är åkkomfort och möjligheten att ägna sig åt andra saker än körning, som att läsa, umgås och koppla av, utan att drabbas av åksjuka under resan. För att autonoma fordon ska lyckas fullt ut är det därför nödvändigt att förstå hur man utformar och styr fordonen för att minska risken för att passagerarna drabbas av åksjuka.  Denna licentiatuppsats syftar till att undersöka hur åksjuka kan förutsägas i vägfordon och hur den kan reduceras med hjälp av fordonsdynamikbaserade lösningar, med tonvikt på trajektorieplanering. Som ett första steg genomfördes en granskning och utvärdering av befintliga metoder för åksjukeprediktion. Granskningen belyste vikten av en korrekt bedömning av åksjuka i de tidiga faserna av autonom fordonsdesign. Två valda metoder (ISO 2631-baserad och sensorisk konfliktbaserad) utvärderades för att uppskatta individuell åksjuka med hjälp av uppmätta data och subjektiva bedömningar från fälttester. Slutsatsen är att metoderna kan justeras för att förutsäga individuell åksjuka, vilket framgår av jämförelsen med experimentella data. För att fortsätta arbetet gjordes en genomgång av fordonsdynamikbaserade metoder för att minska åksjuka i autonoma fordon. Flera chassireglerstrategier i litteraturen, såsom aktiv fjädring, bakhjulsstyrning och drivmomentfördelning, har visat sig kunna bidra till att minska åksjuka. En annan effektiv metod för att minska åksjuka i autonoma fordon är att reglera fordonets hastighet och bana med hjälp av trajektorieplanering, vilket valdes att undersökas ytterligare. Trajektorieplaneringen konstruerades som ett optimeringsproblem där det finns en avvägning mellan åksjuka och manövertid. Effekten av trajektorieplaneringsalgoritmen för att minska åksjuka analyserades genom att simulera två olika fordonsmodeller i specifika testmanövrar. Resultaten indikerar att körstil har en betydande inverkan på åksjuka och att algoritmer för trajektorieplanering bör utformas noggrant för att hitta en bra balans mellan restid och åksjuka. Forskningen som presenteras i denna uppsats bidrar till utvecklingen av metoder för att förutsäga och mildra åksjuka i autonoma fordon, vilket hjälper till att uppnå målet att säkerställa deras framgång.

Autonomous driving could bring profound benefits for our society. The benefits range from economic and safety benefits due to the reduction of the number of traffic accidents, to environmental gains due to reduced traffic congestion. However, the utopian future of self-driving vehicles is yet to come. To this end, we propose machine learning methods to address three pivotal aspects of autonomous driving: visual privacy, 3D depth perception, and trajectory prediction modeling. We begin by exploring the crucial issue of visual privacy within person-aware visual systems. We propose the use of depth-information to protect privacy in person-aware visual systems while preserving important foreground subjects and scene structures. We aim to preserve the identity of foreground subjects while hiding superfluous details in the background that may contain sensitive information. In particular, for an input color and depth image pair, we first create a sensitivity map which favors background regions (where privacy should be preserved) and low depth-gradient pixels (which often relates a lot to scene structure but little to identity). We then combine this per-pixel sensitivity map with an inhomogeneous image obscuration process for privacy protection. We tested the proposed method using data involving different scenarios including various illumination conditions, various number of subjects, different context, etc. The experiments demonstrate the quality of preserving the identity of humans and edges obtained from the depth information while obscuring privacy intrusive information in the background. Next, we focus on the label layout problem: AR technologies can overlay virtual annotations directly onto the real-world view of a self-driving vehicle (SDV). Autonomous vehicles operate in dynamic environments, due to the complexity of the traffic scene and the interactions between the participating agents. Overlaying virtual annotations directly onto the real-world view of a SDV, can provide additional context, such as highlighting important information or projecting the future trajectories of other participants. Designing a layout of labels that does not violate domain-specific design requirements, while at the same time satisfying aesthetic and functional principles of good design, can be a daunting task even for skilled visual designers. Presenting the annotations in 3D object space instead of projection space, allows for the preservation of spatial and depth cues. This results in stable layouts in dynamic environments, since the annotations are anchored in 3D space. In this domain, we make two major contributions. First, we propose a technique for managing the layout and rendering of annotations in Virtual/Augmented Reality scenarios by manipulating the annotations directly in 3D space. For this, we make use of Artificial Potential Fields and use 3D geometric constraints to adapt them in 3D space. Second, we introduce PartLabeling: an open source platform in the form of a web application that acts as a much-needed generic framework allowing to easily add labeling algorithms and 3D models. This serves as a catalyst for researchers in this field to make their algorithms and implementations publicly available, as well as ensure research reproducibility. The PartLabeling framework relies on a dataset that we generate as a subset of the original PartNet dataset consisting of models suitable for the label management task. The dataset consists of 1,000 3D models with part annotations. Finally, we focus on the trajectory prediction task in the context of autonomous driving. Predicting the trajectories of multiple participating agents in the context of autonomous driving is a challenging problem due to the complexity of the traffic scene and the interactions between the agents. Autonomous vehicles need to effectively anticipate the behavior of other movingparticipants in the traffic scene (human pedestrians, cyclists, animals, other moving vehicles). The task of modeling human driver behavior, as well as the interactions between the traffic participants must be addressed to enable a safe and optimized autonomous vehicle systems. There are many factors that traffic participants take into consideration in order to safely interact with other traffic participants. Human drivers have sophisticated interaction strategies that come naturally to them. Given the highly interactive nature of traffic scenarios, representing the interactions between multiple participating agents in a traffic scene in the form of a graph structure is a natural conclusion. In order to leverage the influences between multiple agents in a traffic scene, we structure the scene as a graph whose nodes represent the traffic participants. The node features are each agent’s surrounding context encoded as a raster image. For this purpose, we leveragel R-GCN (Relational Graph-Convolutional Netowrks). Then, we propose a novel Cross-Modal Attention Network (CMAN) to encourage interactions between two modalities: 1) the latent features of an ego-agent’s raster image and 2) the latent features of the surrounding agents’ influences on the ego-agent, in a manner that allows these two modalities to complement each other. / Computer and Information Science

Longitudinal velocity control, or adaptive cruise control (ACC), is a common advanced driving feature aimed at assisting the driver and reducing fatigue. It maintains the velocity of a vehicle and ensures a safe distance from the preceding vehicle. Many models for ACC are available, such as Proportional, Integral, and Derivative (PID) and Model Predictive Control (MPC). However, conventional models have some limitations as they are designed for simplified driving scenarios. Artificial intelligence (AI) and machine learning (ML) have made robust navigation and decision-making possible in complex environments. Recent approaches, such as reinforcement learning (RL), have demonstrated remarkable performance in terms of faster processing and effective navigation through unknown environments. This dissertation explores an RL approach, deep deterministic policy gradient (DDPG), for longitudinal velocity control. The baseline DDPG model has been modified in two different ways. In the first method, an attention mechanism has been applied to the neural network (NN) of the DDPG model. Integrating the attention mechanism into the DDPG model helps in decreasing focus on less important features and enhances overall model effectiveness. In the second method, the inputs of the actor and critic networks of DDPG are replaced with outputs of the self-supervised network. The self-supervised learning process allows the model to accurately predict future states from current states and actions. A custom reward function has been designed for the RL algorithm considering overall safety, efficiency, and comfort. The proposed models have been trained with human car-following data, and evaluated on multiple datasets, including publicly available data, simulated data, and sensor data collected from real-world environments. The analyses demonstrate that the new architectures can maintain strong robustness across various datasets and outperform the current state-of-the-art models.