No description available.

No description available.

Deep learning has proven a powerful tool for image analysis during the past two decades. With the rise of high resolution overhead imagery, an opportunity for automatic geospatial 3D-recreation has presented itself. This master thesis researches the possibil- ity of 3D-recreation through deep learning based image analysis of overhead imagery. The goal is a model capable of making predictions for three different tasks: heightmaps, bound- ary proximity heatmaps and semantic segmentations. A new neural network is designed with the novel feature of supplying the predictions from one task to another with the goal of improving performance. A number of strategies to ensure the model generalizes to un- seen data are employed. The model is trained using satellite and aerial imagery from a variety of cities on the planet. The model is meticulously evaluated by using four common performance metrics. For datasets with no ground truth data, the results were assessed visually. This thesis concludes that it is possible to create a deep learning network capa- ble of making predictions for the three tasks with varying success, performing best for heightmaps and worst for semantic segmentation. It was observed that supplying estima- tions from one task to another can both improve and decrease performance. Analysis into what features in an image is important for the three tasks was clear in some images, unclear in others. Lastly, validation proved that a number of random transformations during the training process helped the model generalize to unseen data. / <p>Examensarbetet är utfört vid Institutionen för teknik och naturvetenskap (ITN) vid Tekniska fakulteten, Linköpings universitet</p>

Understanding a black-box model is a major problem in domains that relies on model predictions in critical tasks. If solved, can help to evaluate the trustworthiness of a model. This thesis proposes a user-centric approach to black-box interpretability. It addresses the problem in semantic segmentation setting with an example of humanitarian remote sensing application for building detection. The question that drives this work was, Can existing methods for explaining black-box classifiers be used for a deep learning semantic segmentation model? We approached this problem with exploratory qualitative research involving a case study and human evaluation. The study showed that it is possible to explain a segmentation model with adapted methods for classifiers but not without a cost. The specificity of the model is likely to be lost in the process. The sole process could include introducing artificial classes or fragmenting image into super-pixels. Other approaches are necessary to mitigate identified drawback. The main contribution of this work is an interactive visualisation approach for exploring learned latent space via a deep segmenter, named U-Net, evaluated with a user study involving 45 respondents. We developed an artefact (accessible online) to evaluate the approach with the survey. It presents an example of this approach with a real-world satellite image dataset. In the evaluation study, the majority of users had a computer science background (80%), including a large percentage of users with machine learning specialisation (44.4% of all respondents). The model distinguishes rurality vs urbanization (58% of users). External quantitative comparison of building densities of each city concerning the location in the latent space confirmed the later. The representation of the model was found faithful to the underlying model (62% of users). Preliminary results show the utility of the pursued approach in the application domain. Limited possibility to present complex model visually requires further investigation. / Att förstå en svartboxmodell är ett stort problem inom domäner som förlitar sig på modellprognoser i kritiska uppgifter. Om det löses, kan det hjälpa till att utvärdera en modells pålitlighet. Den här avhandlingen föreslår en användarcentrisk strategi för svartboxtolkbarhet. Den tar upp problemet i semantisk segmentering med ett exempel på humanitär fjärranalysapplikation för byggnadsdetektering. Frågan som driver detta arbete var: Kan befintliga metoder för att förklara svartruta klassificerare användas för en djup semantisk segmenteringsmodell? Vi närmade oss detta problem med utforskande kvalitativ forskning som involverade en fallstudie och mänsklig utvärdering. Studien visade att det är möjligt att förklara en segmenteringsmodell med anpassade metoder för klassificerare men inte utan kostnad. Modellens specificitet kommer sannolikt att gå förlorad i processen. Den enda processen kan inkludera införande av konstgjorda klasser eller fragmentering av bild i superpixlar. Andra tillvägagångssätt är nödvändiga för att mildra identifierad nackdel. Huvudbidraget i detta arbete är en interaktiv visualiseringsmetod för att utforska lärt latent utrymme via en djup segmenter, benämnd U-Net, utvärderad med en användarstudie med 45 svarande. Vi utvecklade en artefakt (tillgänglig online) för att utvärdera tillvägagångssättet med undersökningen. Den presenterar ett exempel på denna metod med en verklig satellitbilddatasats. I utvärderingsstudien hade majoriteten av användarna en datavetenskaplig bakgrund (80%), inklusive en stor andel användare med specialisering av maskininlärning (44,4 % av alla svarande). Modellen skiljer ruralitet och urbanisering (58 % av användarna). Den externa kvantitativa jämförelsen av byggnadstätheten i varje stad angående platsen i det latenta utrymmet bekräftade det senare. Representationen av modellen visade sig vara trogen mot den underliggande modellen (62% av användarna). Preliminära resultat visar användbarheten av den eftersträvade metoden inom applikationsdomänen. Begränsad möjlighet att presentera komplexa modeller visuellt kräver ytterligare utredning.

The subject of self-driving car technology has drawn growing interest in recent years. Many companies, such as Baidu and Tesla, have already introduced automatic driving techniques in their newest cars when driving in a specific area. However, there are still many challenges ahead toward fully autonomous driving cars. Tesla has caused several severe accidents when using autonomous driving functions, which makes the public doubt self-driving car technology. Therefore, it is necessary to use the simulator environment to help verify and perfect algorithms for the perception, planning, and decision-making of autonomous vehicles before implementation in real-world cars. This project aims to build a benchmark for implementing the whole self-driving car system in software. There are three main components including perception, planning, and control in the entire autonomous driving system. This thesis focuses on two sub-tasks 2D object detection and semantic segmentation in the perception part. All of the experiments will be tested in a simulator environment called The CAR Learning to Act(Carla), which is an open-source platform for autonomous car research. Carla simulator is developed based on the game engine(Unreal4). It has a server-client system, which provides a flexible python API. 2D object detection uses the You only look once(Yolov4) algorithm that contains the tricks of the latest deep learning techniques from the aspect of network structure and data augmentation to strengthen the network’s ability to learn the object. Yolov4 achieves higher accuracy and short inference time when comparing with the other popular object detection algorithms. Semantic segmentation uses Efficient networks for Computer Vision(ESPnetv2). It is a light-weight and power-efficient network, which achieves the same performance as other semantic segmentation algorithms by using fewer network parameters and FLOPS. In this project, Yolov4 and ESPnetv2 are implemented into the Carla simulator. Two modules work together to help the autonomous car understand the world. The minimal distance awareness application is implemented into the Carla simulator to detect the distance to the ahead vehicles. This application can be used as a basic function to avoid the collision. Experiments are tested by using a single Nvidia GPU(RTX2060) in Ubuntu 18.0 system. / Ämnet självkörande bilteknik har väckt intresse de senaste åren. Många företag, som Baidu och Tesla, har redan infört automatiska körtekniker i sina nyaste bilar när de kör i ett specifikt område. Det finns dock fortfarande många utmaningar inför fullt autonoma bilar. Detta projekt syftar till att bygga ett riktmärke för att implementera hela det självkörande bilsystemet i programvara. Det finns tre huvudkomponenter inklusive uppfattning, planering och kontroll i hela det autonoma körsystemet. Denna avhandling fokuserar på två underuppgifter 2D-objekt detektering och semantisk segmentering i uppfattningsdelen. Alla experiment kommer att testas i en simulatormiljö som heter The CAR Learning to Act (Carla), som är en öppen källkodsplattform  för autonom bilforskning. Du ser bara en gång (Yolov4) och effektiva nätverk för datorvision (ESPnetv2) implementeras i detta projekt för att uppnå Funktioner för objektdetektering och semantisk segmentering. Den minimala distans medvetenhets applikationen implementeras i Carla-simulatorn för att upptäcka avståndet till de främre bilarna. Denna applikation kan användas som en grundläggande funktion för att undvika kollisionen.

Network mapping is used to extract the coordinates of a network's components in an image. Furthermore, machine learning algorithms have demonstrated their efficacy in advancing the field of network mapping across various domains, including mapping of road networks and blood vessel networks. However, accurately mapping of road networks still remains a challenge due to difficulties in identification and separation of roads in the presence of occlusion caused by trees, as well as complex environments, such as parking lots and complex intersections. Additionally, the segmentation of blood vessels networks, such as the ones in the retina, is also not trivial due to their complex shape and thin appearance. Therefore, the aim for this thesis was to investigate two deep learning approaches to improve mapping of networks, namely by refining existing road network probability maps, and by estimating road network orientations. Additionally, the thesis explores the possibility of using a machine learning model trained on road network probability maps to refine retina network segmentations. In the first approach, U-Net models with a binary output channel were implemented to refine existing probability maps of networks. In the second approach, ResNet models with a regression output were implemented to estimate the orientation of roads within a network. The models for refining road network probability maps were evaluated using F1-score and MCC-score, while the models for estimating road network orientation were evaluated based on angle loss, angle difference, F1-score, and MCC-score.  The results for refining road segmentations yielded an increase of 0.102 MCC-score compared to the baseline (0.701). However, when applying the segmentation refinement model to retina images, the output from the model achieved merely 0.226 in MCC-score. Nevertheless, the model demonstrated the capability to identify and refine the segmentation of large blood vessels. Additionally, the estimation of road network orientation achieved an average error of 10.50 degrees. It successfully distinguished roads from the background, achieving an MCC-score of 0.805. In conclusion, this thesis shows that a deep learning-based approach for road segmentation refinement is beneficial, especially in cases where occlusions are present. However, the refinement of retina image segmentations using a model trained on roads and tested on retina images produced unsatisfactory results, likely due to differences in scale between road width and vessel size. Further experiments with adjustments in image scales are likely needed to achieve better results. Moreover, the orientation model demonstrated promising results in estimating the orientation of road pixels and effectively differentiating between road and non-road pixels.

Multitask Learning (MTL) was conceived as an approach to improve thegeneralization ability of machine learning models. When applied to neu-ral networks, multitask models take advantage of sharing resources forreducing the total inference time, memory footprint and model size. Wepropose MTL as a way to speed up deep learning models for applicationsin which multiple tasks need to be solved simultaneously, which is par-ticularly useful in embedded, real-time systems such as the ones foundin autonomous cars or UAVs.In order to study this approach, we apply MTL to a Computer Vi-sion problem in which both Object Detection and Semantic Segmenta-tion tasks are solved based on the Single Shot Multibox Detector andFully Convolutional Networks with skip connections respectively, usinga ResNet-50 as the base network. We train multitask models for twodifferent datasets, Pascal VOC, which is used to validate the decisionsmade, and a combination of datasets with aerial view images capturedfrom UAVs.Finally, we analyse the challenges that appear during the process of train-ing multitask networks and try to overcome them. However, these hinderthe capacity of our multitask models to reach the performance of the bestsingle-task models trained without the limitations imposed by applyingMTL. Nevertheless, multitask networks benefit from sharing resourcesand are 1.6x faster, lighter and use less memory compared to deployingthe single-task models in parallel, which turns essential when runningthem on a Jetson TX1 SoC as the parallel approach does not fit intomemory. We conclude that MTL has the potential to give superior per-formance as far as the object detection and semantic segmentation tasksare concerned in exchange of a more complex training process that re-quires overcoming challenges not present in the training of single-taskmodels.

Autonomous driving is the concept of a vehicle that operates in traffic without instructions from a driver. A major challenge for such a system is to provide a comprehensive, accurate and compact scene model based on information from sensors. For such a model to be comprehensive it must provide 3D position and semantics on relevant surroundings to enable a safe traffic behavior. Such a model creates a foundation for autonomous driving to make substantiated driving decisions. The model must be compact to enable efficient processing, allowing driving decisions to be made in real time. In this thesis rectangular objects (The Stixelworld) are used to represent the surroundings of a vehicle and provide a scene model. LIDAR and semantic segmentation are fused in the computation of these rectangles. This method indicates that a dense and compact scene model can be provided also from sparse LIDAR data by use of semantic segmentation. / Fullt självkörande fordon behöver inte förare. Ett sådant fordon behöver en precis, detaljerad och kompakt modell av omgivningen baserad på sensordata. Med detaljerad avses att modellen innefattar all information nödvändig för ett trafiksäkert beteende. Med kompakt avses att en snabb bearbetning kan göras av modellen så att fordonet i realtid kan fatta beslut och manövrera i trafiken. I denna uppsats tillämpas en metod där man med rektangulära objekt skapar en modell av omgivningen. Dessa beräknas från LIDAR och semantisk segmentering. Arbetet indikerar att med hjälp av semantisk segmentering kan en tät, detaljerad och kompakt modell göras även från glesa LIDAR-data.

This thesis presents a deep learning tool able to identify ice in radar images fromthe sea-ice environment of the Twhaites glacier outlet. The project is motivatedby the threatening situation of the Thwaites glacier that has been increasingits mass loss rate during the last decade. This is of concern considering thelarge mass of ice held by the glacier, that in case of melting, could increasethe mean sea level by more than +65 cm [1]. The algorithm generated alongthis work is intended to help in the generation of navigation charts and identificationof icebergs in future stages of the project, outside of the scope of this thesis.The data used for this task are ICEYE’s X-band radar images from the Thwaitessea-ice environment, the target area to be studied. The corresponding groundtruth for each of the samples has been manually generated identifying the iceand icebergs present in each image. Additional data processing includes tiling,to increment the number of samples, and augmentation, done by horizontal andvertical flips of a random number of tiles.The proposed tool performs semantic segmentation on radar images classifyingthe class "Ice". It is developed by a deep learning Convolutional Neural Network(CNN) model, trained with prepared ICEYE’s radar images. The model reachesvalues of F1 metric higher than 89% in the images of the target area (Thwaitessea-ice environment) and is able to generalize to different regions of Antarctica,reaching values of F1 = 80 %. A potential alternative version of the algorithm isproposed and discussed. This alternative score F1 values higher than F1 &gt; 95 %for images of the target environment and F1 = 87 % for the image of the differentregion. However, it must not be confirmed as the final algorithm due to the needfor further verification.

[pt] A segmentação semântica das estruturas anatômicas em imagens de mamografia desempenha um papel significativo no apoio da análise médica. Esta tarefa pode ser abordada com o uso de um modelo de aprendizado de máquina, que deve ser capaz de identificar e delinear corretamente as estruturas de interesse tais como papila, tecido fibroglandular, músculo peitoral e tecido gorduroso. No entanto, a segmentação de estruturas pequenas como papila e peitoral é frequentemente um desafio. Sendo o maior desafio o reconhecimento ou deteção do músculo peitoral na vista craniocaudal (CC), devido ao seu tamanho variável, possíveis ausências e sobreposição de tecido fibroglandular. Para enfrentar esse desafio, este trabalho propõe uma abordagem centrada em dados para melhorar o desempenho do modelo de segmentação na papila mamária e no músculo peitoral. Especificamente, aprimorando os dados de treinamento e as anotações em duas etapas. A primeira etapa é baseada em modificações nas anotações. Foram desenvolvidos algoritmos para buscar automaticamente anotações fora do comum dependendo da sua forma. Com estas anotações encontradas, foi feita uma revisão e correção manual. A segunda etapa envolve um downsampling do conjunto de dados, reduzindo as amostras de imagens do conjunto de treinamento. Foram analisados os casos de falsos positivos e falsos negativos, identificando as imagens que fornecem informações confusas, para posteriormente removê-las do conjunto. Em seguida, foram treinados modelos usando os dados de cada etapa e foram obtidas as métricas de classificação para o músculo peitoral em vista CC e o IoU para cada estrutura nas vistas CC e MLO (Mediolateral Oblíqua). Os resultados do treinamento mostram uma melhora progressiva na identificação e segmentação do músculo peitoral em vista CC e uma melhora na papila em vista MLO, mantendo as métricas para as demais estruturas. / [en] The semantic segmentation of anatomical structures in mammography images plays a significant role in supporting medical analysis. This task can be approached using a machine learning model, which must be capable of identifying and accurately delineating the structures. However, segmentation of small structures such as nipple and pectoral is often challenging. Especially in there cognition or detection of the pectoral muscle in the craniocaudal (CC) view,due to its variable size, possible absences and overlapping of fibroglandular tissue.To tackle this challenge, this work proposes a data-centric approach to improvethe segmentation model s performance on the mammary papilla and pectoral muscle. Specifically, enhancing the training data and annotations in two stages.The first stage is based on modifications to the annotations. Algorithms were developed to automatically search for uncommon annotations dependingon their shape. Once these annotations were found, a manual review and correction were performed.The second stage involves downsampling the dataset, reducing the image samples in the training set. Cases of false positives and false negatives were analyzed, identifying images that provide confusing information, which were subsequently removed from the set. Next, models were trained using the data from each stage, and classification metrics were obtained for the pectoral muscle in the CC view and IoU for each structure in CC and MLO (mediolateral oblique) views. The training results show a progressive improvement in the identification and segmentation of the pectoral muscle in the CC view and an enhancement in the mammary papilla in the MLO view, while maintaining segmentation metricsfor the other structures.