In recent years, the EU has observed a decrease in the stocks of certain fish species due to unrestricted fishing. To combat the problem, many fisheries are investigating how to automatically estimate the catch size and composition using sensors onboard the vessels. Yet, measuring the size of fish in marine imagery is a difficult task. The images generally suffer from complex conditions caused by cluttered fish, motion blur and dirty sensors. In this thesis, we propose a novel method for automatic measurement of fish size that can enable measuring both visible and occluded fish. We use a Mask R-CNN to segment the visible regions of the fish, and then fill in the shape of the occluded fish using a U-Net. We train the U-Net to perform shape completion in a semi-supervised manner, by simulating occlusions on an open-source fish dataset. Different to previous shape completion work, we teach the U-Net when to fill in the shape and not by including a small portion of fully visible fish in the input training data. Our results show that our proposed method succeeds to fill in the shape of the synthetically occluded fish as well as of some of the cluttered fish in real marine imagery. We achieve an mIoU score of 93.9 % on 1 000 synthetic test images and present qualitative results on real images captured onboard a fishing vessel. The qualitative results show that the U-Net can fill in the shapes of lightly occluded fish, but struggles when the tail fin is hidden and only parts of the fish body is visible. This task is difficult even for a human, and the performance could perhaps be increased by including the fish appearance in the shape completion task. The simulation-to-reality gap could perhaps also be reduced by finetuning the U-Net on some real occlusions, which could increase the performance on the heavy occlusions in the real marine imagery.

Magnetic resonance imaging (MRI) is a commonly used non-invasive imaging technique that provides high resolution images of soft tissue. One problem with MRI is that it is sensitive to signal quality issues. The issues can arise for various reasons, for example by metal located either inside or outside of the body. Another common signal quality issue is caused by the patient being partly placed outside field of view of the MRI scanner.   This thesis aims to investigate the possibility to automatically detect these signal quality issues using deep neural networks. More specifically, two different 3D CNN network types were studied, a classification-based approach and a reconstruction-based approach. The datasets used consist of MRI volumes from UK Biobank which have been processed and manually annotated by operators at AMRA Medical. For the classification method four different network architectures were explored utilising supervised learning with multi-label classification. The classification method was evaluated using accuracy and label-based evaluation metrics, such as macro-precision, macro-recall and macro-F1. The reconstruction method was based on anomaly detection using an autoencoder which was trained to reconstruct volumes without any artefacts. A mean squared prediction error was calculated for the reconstructed volume and compared against a threshold in order to classify a volume with or without artefacts. The idea was that volumes containing artefacts should be more difficult to reconstruct and thus, result in a higher prediction error. The reconstruction method was evaluated using accuracy, precision, recall and F1-score.  The results show that the classification method has overall higher performance than the reconstruction method. The achieved accuracy for the classification method was 98.0% for metal artefacts and 97.5% for outside field of view artefacts. The best architecture for the classification method proved to be DenseNet201. The reconstruction method worked for metal artefacts with an achieved accuracy of 75.7%. Furthermore, it was concluded that reconstruction method did not work for detection of outside field of view artefacts.    The results from the classification method indicate that there is a possibility to automatically detect artefacts with deep neural networks. However, it is needed to further improve the method in order to completely replace a manual quality control step before using the volumes for calculation of biomarkers.

In recent years, the usage of 3D deep learning techniques has seen a surge,mainly driven by advancements in autonomous driving and medical applications.This thesis investigates the applicability of existing state-of-the-art 3Ddeep learning network architectures to dense voxel grids from single photoncounting 3D LiDAR. This work also examine the choice of loss function asa means of dealing with extreme data imbalance, in order to segment peopleand vehicles in outdoor forest scenes. Due to data similarities with volumetricmedical data, such as computer tomography scans, this thesis investigates ifa model for 3D deep learning used for medical applications, the commonlyused 3D U-Net, can be used for photon counting data. The results showthat segmentation of people and vehicles is possible in this type of data butthat performance depends on the segmentation task, light conditions, and theloss function. For people segmentation the final models are able to predictall targets, but with a significant amount of false positives, something that islikely caused by similar LiDAR responses between people and tree trunks.For vehicle detection, the results are more inconsistent and varies greatlybetween different loss functions as well as the position and orientation of thevehicles. Overall, we consider the 3D U-Net model a successful proof-ofconceptregarding the applicability of 3D deep learning techniques to this kindof data. / Under de senaste åren har användningen för djupinlärningstekniker för 3Dsett en kraftig ökning, främst driven av framsteg inom autonoma fordon ochmedicinska tillämpningar. Denna avhandling undersöker befintliga modernadjupinlärningsnätverk för 3D i täta voxelgriddar från fotonräknande 3D LiDARför att segmentera människor och fordon i skogsscener. Vidare undersöksvalet av målfunktion som ett sätt att hantera extrem dataobalans. På grundav datalikheter med volymetriska medicinska data, såsom datortomografi,kommer denna avhandling att undersöka om en modell för 3D-djupinlärningsom används för medicinska applikationer, nämligen 3D U-Net, kan användasför fotonräknande data. Resultaten visar att segmentering av människor ochfordon är möjligt men att prestanda varier avsevärt med segmenteringsuppgiften,ljusförhållanden, och målfunktioner. För segmentering av människorkan de slutgiltiga modellerna segmentera alla mål men med en betydandemängd falska utslag, något som sannolikt orsakas av liknande LiDAR-svarmellan människor och trädstammar. För segmentering av fordon är resultatenmer oberäkneliga och varierar kraftigt mellan olika målfunktioner såväl somfordonens position och orientering. Sammantaget anser vi att 3D U-Netmodellenvisar på en framgångsrik konceptvalidering när det gäller tillämpningav djupinlärningstekniker för 3D på denna typ av data.

Remote Sensing (RS) technology provides valuable information about Earth by enabling an overview of the planet from above, making it a much-needed resource for many applications. Given the abundance of RS data and continued urbanisation, there is a need for efficient approaches to leverage RS data and its unique characteristics for the assessment and management of urban areas. Consequently, employing Deep Learning (DL) for RS applications has attracted much attention over the past few years. In this thesis, novel datasets consisting of satellite RS images over urban areas in Sweden were compiled from Sentinel-2 multispectral, Sentinel-1 Synthetic Aperture Radar (SAR) and Urban Atlas 2018 Land Use/Land Cover (LULC) data. Then, DL was applied for multiband and multiclass semantic image segmentation of LULC. The contributions of complementary spectral, temporal and SAR data and spectral indices to LULC classification performance compared to using only Sentinel-2 data with red, green and blue spectral bands were investigated by implementing DL models based on the fully convolutional network-based architecture, U-Net, and performing data fusion. Promising results were achieved with 25 possible LULC classes. Furthermore, almost all DL models at an overall model level and all DL models at an individual class level for most LULC classes benefited from complementary satellite RS data with varying degrees of classification improvement. Additionally, practical knowledge and insights were gained from evaluating the results and are presented regarding satellite RS data characteristics and semantic segmentation of LULC in urban areas. The obtained results are helpful for practitioners and researchers applying or intending to apply DL for semantic segmentation of LULC in general and specifically in Swedish urban environments.

The deep learning-based automatic segmentation methods have developed rapidly in recent years to give a promising performance in the medical image segmentation tasks, which provide clinical medicine with an accurate and fast computer-aided diagnosis method. Generative adversarial networks and their extended frameworks have achieved encouraging results on image-to-image translation problems. In this report, the proposed hybrid network combined cycle-consistent adversarial networks, which transformed contrast-enhanced images from computed tomography angiography to the conventional low-contrast CT scans, with the segmentation network and trained them simultaneously in an end-to-end manner. The trained segmentation network was tested on the non-contrast-enhanced CT images. The synthetic process and the segmentation process were also implemented in a two-stage manner. The two-stage process achieved a higher Dice similarity coefficient than the baseline U-Net did on test data, but the proposed hybrid network did not outperform the baseline due to the field of view difference between the two training data sets.

In light of the emergence of social media, and its abundance of facial imagery, facial recognition finds itself useful from an Open Source Intelligence standpoint. Images uploaded on social media are likely to be filtered, which can destroy or modify biometric features. This study looks at the recognition effort of identifying individuals based on their facial image after filters have been applied to the image. The social media image filters studied occlude parts of the nose and eyes, with a particular interest in filters occluding the eye region. Our proposed method uses a Residual Neural Network Model to extract features from images, with recognition of individuals based on distance measures, based on the extracted features. Classification of individuals is also further done by the use of a Linear Support Vector Machine and XGBoost classifier. In attempts to increase the recognition performance for images completely occluded in the eye region, we present a method to reconstruct this information by using a variation of a U-Net, and from the classification perspective, we also train the classifier on filtered images to increase the performance of recognition. Our experimental results showed good recognition of individuals when filters were not occluding important landmarks, especially around the eye region. Our proposed solution shows an ability to mitigate the occlusion done by filters through either reconstruction or training on manipulated images, in some cases, with an increase in the classifier’s accuracy of approximately 17% points with only reconstruction, 16% points when the classifier trained on filtered data, and  24% points when both were used at the same time. When training on filtered images, we observe an average increase in performance, across all datasets, of 9.7% points.

Identification and localization of FDG-avid lymphoma lesions in PET-CT image volumes is of high importance for the diagnosis and monitoring of treatment progress in lymphoma patients. This process is tedious, time-consuming, and error-prone, due to large image volumes and the heterogeneity of lesions. Thus, a fully automatic method for lymphoma detection is desirable. The AutoPET challenge dataset contains 145 full-body FDG-PET-CT images of lymphoma patients with pixel-level segmentation of lesions. The Retina U-Net utilizes semantic segmentation maps for object detection through simultaneous segmentation and detection. More recently, transformer-based methods became increasingly popular due to their good performance. Here, TransRUnet is proposed, a 2D deep neural network capable of segmentation and object detection, combining the Retina U-Net with a Feature Pyramid Transformer. Firstly, a Retina U-Net was trained as a Baseline on 2D axial slices of 116 patient volumes from the AutoPET dataset, achieving an mAP of 0.377 and a DSC of 0.737 on the 29 test patients. Secondly, the TransRUnet was trained on the same patients, achieving an mAP and DSC of 0.285 and 0.732, respectively. Performance comparison based on mAP and DSC did not show significant differences (p = 0.596 and p = 0.940, for mAP and DSC, respectively) between the Retina U-Net and the TransRUnet. Furthermore, a substantial difference in FROC between the two models could not be observed. The ground truth data should be preprocessed to reduce noise in the training data or a 3D generalization of the TransRUnet should be used to improve the detection performance. / Att i PET-CT-bildvolymer identifiera och lokalisera lymfomlesioner med hög FDG-aviditet är av stor betydelse för diagnos och övervakning av behandlingseffekt hos lymfompatienter. Denna process är omständlig, tidskrävande och felbenägen på grund av stora bildvolymer och heterogeniteten hos lesionerna. Därför är det önskvärt med en helautomatisk metod för lymfomdetektion. AutoPET Challenge-datasetet innehåller 145 FDG-PET-CT-bilder av lymfom-patienter med segmentering av lesioner på pixelnivå. Retina U-Net använder semantiska segmenteringskartor för objektsdetektering genom samtidig segmentering och detektering. På senare tid har transformatorbaserade metoder blivit alltmer populära på grund av sina goda prestanda. Här föreslås TransRUnet, ett djupgående neuralt 2D-nätverk som kan segmentera och upptäcka objekt och som kombinerar Retina U-Net med en Feature Pyramid Transformer. I första steget tränades ett Retina U-Net som baslinje på 2D axialskivor av 116 patientvolymer från AutoPET-dataset, och uppnådde en mAP på 0,377 och en DSC på 0,737 på de 29 testpatienterna. I nästa steg tränades TransRUnet på samma patienter och uppnådde en mAP och DSC på 0,285 respektive 0,732. Jämförelse av prestanda baserat på mAP och DSC visade inga signifikanta skillnader (p = 0,596 och p = 0,940 för mAP respektive DSC) mellan Retina U-Net och TransRUnet. Dessutom kunde ingen väsentlig skillnad i FROC mellan de två modellerna observeras. Ground truth-data bör förbehandlas för att minska bruset i träningsdata eller också bör en 3D-generalisering av TransRUnet användas för att förbättra detektionsprestanda.

[pt] A partir de melhorias incrementais em uma conhecida rede neural convolucional (U-Net), diferentes técnicas são avaliadas quanto às suas performances na tarefa de segmentação semântica em imagens sísmicas. Mais especificamente, procura-se a identificação e delineamento de estruturas salinas no subsolo, o que é de grande relevância na indústria de óleo e gás para a exploração de petróleo em camadas pré-sal, por exemplo. Além disso, os desafios apresentados no tratamento destas imagens sísmicas se assemelham em muito aos encontrados em tarefas de áreas médicas como identificação de tumores e segmentação de tecidos, o que torna o estudo da tarefa em questão ainda mais valioso. Este trabalho pretende sugerir uma metodologia adequada de abordagem à tarefa e produzir redes neurais capazes de segmentar imagens sísmicas com bons resultados dentro das métricas utilizadas. Para alcançar estes objetivos, diferentes estruturas de redes, transferência de aprendizado e técnicas de aumentação de dados são testadas em dois datasets com diferentes níveis de complexidade. / [en] Through incremental improvements in a well-known convolutional neural network (U-Net), different techniques are evaluated regarding their performance on the task of semantic segmentation of seismic images. More specifically, the objective is the better identification and outline of subsurface salt structures, which is a task of great relevance for the oil and gas industry in the exploration of pre-salt layers, for example. Besides that application, the challenges imposed by the treatment of seismic images also resemble those found in medical fields like tumor detection and tissue segmentation, which makes the study of this task even more valuable. This work seeks to suggest a suitable methodology for the task and to yield neural networks that are capable of performing semantic segmentation of seismic images with good results regarding specific metrics. For that purpose, different network structures, transfer learning and data augmentation techniques are applied in two datasets with different levels of complexity.

Recent improvements in deep learning architectures, combined with the strength of modern computing hardware such as graphics processing units, has lead to significant results in the field of image analysis. In this thesis work, locally connected architectures are employed to reduce noise in flash X-ray diffraction images. The layers in these architectures use convolutional kernels, but without shared weights. This combines the benefits of lower model memory footprint in convolutional networks with the higher model capacity of fully connected networks. Since the camera used to capture the diffraction images has pixelwise unique characteristics, and thus lacks equivariance, this compromise can be beneficial. The background images of this thesis work were generated with an active laser but without injected samples. Artificial diffraction patterns were then added to these background images allowing for training U-Net architectures to separate them. Architecture A achieved a performance of 0.187 on the test set, roughly translating to 35 fewer photon errors than a model similar to state of the art. After smoothing the photon errors this performance increased to 0.285, since the U-Net architectures managed to remove flares where state of the art could not. This could be taken as a proof of concept that locally connected networks are able to separate diffraction from background in flash X-Ray imaging.

Táto práce se zabývá použitím hlubokého učení neuronových sítí ke zvýšení rozlišení obrázků, které obsahují obličeje. Tato metoda najde uplatnění v různých oblastech, zejména v bezpečnosti, například, při bezpečnostním incidentu, kdy policie potřebuje identifikovat podezřelého z nahraného videa ze sledovací kamery. Cílem této práce je navrhnout minimálně dvě architektury neuronových sítí, které budou pracovat se sekvencí snímků, a porovnat je s metodami zpracování jediného snímku. Pro tento účel je také vytvořena nová trénovací množina, obsahující sekvenci snímku obličeje. Metody zpracování jednoho snímku jsou natrénované na nové množině. Dále jsou navrženy nové metody zvětšení obrázků na základě sekvence snímků. Tyto metody jsou založené na U-Net modelu, který je úspěšný v segmentaci, ale také v superrozlišení. Pro zlepšení architektury byly použity reziduální bloky a jejich modifikace, a navíc také percepční ztrátová funkce, která dovoluje vyhnout se rozmazání a získání více detailů. První čast této práce je věnovana popisu neuronových sítí a některých architektur, jejichž modifikace mohou být použity v superrozlišení. Druhá část se poté zabývá popisem metod pro zvýšení rozlišení obrazu pomocí jednoho snímku, několika snímků a videa. Ve třetí části jsou popsány navržené metody a experimenty a v poslední části porovnaná metod založených na jednom snímku a několika snímcích. Navržené metody jsou schopny získat více detailů v obraze, ale mohou produkovat artefakty. Ty lze ale poté eliminovat pomocí filtru, například Gaussova. Nové metody méně selhávají při detekci obličejů, a to je podstatné u identifikace člověka v případě incidentu.