Submitted by Luciana Ferreira (lucgeral@gmail.com) on 2018-11-21T11:05:44Z No. of bitstreams: 2 Dissertação - Vinícius Araújo Santos - 2018.pdf: 14601054 bytes, checksum: e02a8bcd3cdc93bf2bf202c3933b3f27 (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) / Approved for entry into archive by Luciana Ferreira (lucgeral@gmail.com) on 2018-11-21T11:06:26Z (GMT) No. of bitstreams: 2 Dissertação - Vinícius Araújo Santos - 2018.pdf: 14601054 bytes, checksum: e02a8bcd3cdc93bf2bf202c3933b3f27 (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) / Made available in DSpace on 2018-11-21T11:06:26Z (GMT). No. of bitstreams: 2 Dissertação - Vinícius Araújo Santos - 2018.pdf: 14601054 bytes, checksum: e02a8bcd3cdc93bf2bf202c3933b3f27 (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) Previous issue date: 2018-10-11 / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - CAPES / Visual Odometry is an important process in image based navigation of robots. The standard methods of this field rely on the good feature matching between frames where feature detection on images stands as a well adressed problem within Computer Vision. Such techniques are subject to illumination problems, noise and poor feature localization accuracy. Thus, 3D information on a scene may mitigate the uncertainty of the features on images. Deep Learning techniques show great results when dealing with common difficulties of VO such as low illumination conditions and bad feature selection. While Visual Odometry and Deep Learning have been connected previously, no techniques applying Siamese Convolutional Networks on depth infomation given by disparity maps have been acknowledged as far as this work’s researches went. This work aims to fill this gap by applying Deep Learning to estimate egomotion through disparity maps on an Siamese architeture. The SiameseVO-Depth architeture is compared to state of the art techniques on OV by using the KITTI Vision Benchmark Suite. The results reveal that the chosen methodology succeeded on the estimation of Visual Odometry although it doesn’t outperform the state-of-the-art techniques. This work presents fewer steps in relation to standard VO techniques for it consists of an end-to-end solution and demonstrates a new approach of Deep Learning applied to Visual Odometry. / Odometria Visual é um importante processo na navegação de robôs baseada em imagens. Os métodos clássicos deste tema dependem de boas correspondências de características feitas entre imagens sendo que a detecção de características em imagens é um tema amplamente discutido no campo de Visão Computacional. Estas técnicas estão sujeitas a problemas de iluminação, presença de ruído e baixa de acurácia de localização. Nesse contexto, a informação tridimensional de uma cena pode ser uma forma de mitigar as incertezas sobre as características em imagens. Técnicas de Deep Learning têm demonstrado bons resultados lidando com problemas comuns em técnicas de OV como insuficiente iluminação e erros na seleção de características. Ainda que já existam trabalhos que relacionam Odometria Visual e Deep Learning, não foram encontradas técnicas que utilizem Redes Convolucionais Siamesas com sucesso utilizando informações de profundidade de mapas de disparidade durante esta pesquisa. Este trabalho visa preencher esta lacuna aplicando Deep Learning na estimativa do movimento por de mapas de disparidade em uma arquitetura Siamesa. A arquitetura SiameseVO-Depth proposta neste trabalho é comparada à técnicas do estado da arte em OV utilizando a base de dados KITTI Vision Benchmark Suite. Os resultados demonstram que através da metodologia proposta é possível a estimativa dos valores de uma Odometria Visual ainda que o desempenho não supere técnicas consideradas estado da arte. O trabalho proposto possui menos etapas em comparação com técnicas clássicas de OV por apresentar-se como uma solução fim-a-fim e apresenta nova abordagem no campo de Deep Learning aplicado à Odometria Visual.

La reconnaissance des opinions d'un locuteur dans une interaction orale est une étape cruciale pour améliorer la communication entre un humain et un agent virtuel. Dans cette thèse, nous nous situons dans une problématique de traitement automatique de la parole (TAP) sur les phénomènes d'opinions dans des interactions orales spontanées naturelles. L'analyse d'opinion est une tâche peu souvent abordée en TAP qui se concentrait jusqu'à peu sur les émotions à l'aide du contenu vocal et non verbal. De plus, la plupart des systèmes récents existants n'utilisent pas le contexte interactionnel afin d'analyser les opinions du locuteur. Dans cette thèse, nous nous penchons sur ces sujet. Nous nous situons dans le cadre de la détection automatique en utilisant des modèles d’apprentissage statistiques. Après une étude sur la modélisation de la dynamique de l'opinion par un modèle à états latents à l’intérieur d'un monologue, nous étudions la manière d’intégrer le contexte interactionnel dialogique, et enfin d'intégrer l'audio au texte avec différents types de fusion. Nous avons travaillé sur une base de données de Vlogs au niveau d'un sentiment global, puis sur une base de données d'interactions dyadiques multimodales composée de conversations ouvertes, au niveau du tour de parole et de la paire de tours de parole. Pour finir, nous avons fait annoté une base de données en opinion car les base de données existantes n'étaient pas satisfaisantes vis-à-vis de la tâche abordée, et ne permettaient pas une comparaison claire avec d'autres systèmes à l'état de l'art.A l'aube du changement important porté par l’avènement des méthodes neuronales, nous étudions différents types de représentations: les anciennes représentations construites à la main, rigides mais précises, et les nouvelles représentations apprises de manière statistique, générales et sémantiques. Nous étudions différentes segmentations permettant de prendre en compte le caractère asynchrone de la multi-modalité. Dernièrement, nous utilisons un modèle d'apprentissage à états latents qui peut s'adapter à une base de données de taille restreinte, pour la tâche atypique qu'est l'analyse d'opinion, et nous montrons qu'il permet à la fois une adaptation des descripteurs du domaine écrit au domaine oral, et servir de couche d'attention via son pouvoir de clusterisation. La fusion multimodale complexe n'étant pas bien gérée par le classifieur utilisé, et l'audio étant moins impactant sur l'opinion que le texte, nous étudions différentes méthodes de sélection de paramètres pour résoudre ces problèmes. / 2588/5000Recognizing a speaker's opinions in an oral interaction is a crucial step in improving communication between a human and a virtual agent. In this thesis, we find ourselves in a problematic of automatic speech processing (APT) on opinion phenomena in natural spontaneous oral interactions. Opinion analysis is a task that is not often addressed in TAP that focused until recently on emotions using voice and non-verbal content. In addition, most existing legacy systems do not use the interactional context to analyze the speaker's opinions. In this thesis, we focus on these topics.We are in the context of automatic detection using statistical learning models. A study on modeling the dynamics of opinion by a model with latent states within a monologue, we study how to integrate the context interactional dialogical, and finally to integrate audio to text with different types of fusion. We worked on a basic Vlogs data at a global sense, and on the basis of multimodal data dyadic interactions composed of open conversations, at the turn of speech and word pair of towers. Finally, we annotated database in opinion because existing database were not satisfactory vis-à-vis the task addressed, and did not allow a clear comparison with other systems in the state art.At the dawn of significant change brought by the advent of neural methods, we study different types of representations: the ancient representations built by hand, rigid, but precise, and new representations learned statistically, and general semantics. We study different segmentations to take into account the asynchronous nature of multi-modality. Recently, we are using a latent state learning model that can adapt to a small database, for the atypical task of opinion analysis, and we show that it allows both an adaptation of the descriptors of the written domain to the oral domain, and serve as an attention layer via its clustering power. Complex multimodal fusion is not well managed by the classifier used, and audio being less impacting on opinion than text, we study different methods of parameter selection to solve these problems.

Die Anwendung von Deep Learning in der manuellen Montage birgt großes Potenzial, Montagezeiten zu reduzieren und Montagefehler zu vermeiden. Indem der Montageablauf mithilfe einer Kamera erfasst und die aufgezeichneten Bilder durch einen Objekterkennungsalgorithmus analysiert werden, lassen sich Position, Lage und Art der montierten Bauteile bestimmen. Daraus lassen sich wiederum Informationen über Arbeitsschritte, Montagefehler oder den aktuellen Zustand des Produkts ableiten, sodass die Mitarbeiter bei der Montage durch entsprechende Anweisungen unterstützt werden können. Es stellt sich jedoch die Frage, inwieweit gegenwärtige Produkte für den Einsatz von Deep Learning geeignet sind. Nur wenn die zu montierenden Bauteile sicher erkannt werden, ist der Einsatz in der manuellen Montage sinnvoll. Bestehende Gestaltungsrichtlinien adressieren diesen Aspekt bislang nicht. Im Forschungsprojekt wurde daher untersucht, welche Eigenschaften Produkte aufweisen sollten, um eine optimale Objekterkennung zu ermöglichen. Dazu wurden Hypothesen zu positiven und negativen Bauteileigenschaften hinsichtlich der Erkennungsgenauigkeit formuliert und in praktischen Versuchen überprüft. Dabei konnte gezeigt werden, dass alle untersuchten Objekte durch den eingesetzten Objekterkennungsalgorithmus sehr gut detektiert werden. Aus den vorliegenden Forschungsergebnissen lassen sich daher keine Einschränkungen in der Produktgestaltung ableiten.

Cardinality estimation is a fundamental task in database query processing and optimization. Unfortunately, the accuracy of traditional estimation techniques is poor resulting in non-optimal query execution plans. With the recent expansion of machine learning into the field of data management, there is the general notion that data analysis, especially neural networks, can lead to better estimation accuracy. Up to now, all proposed neural network approaches for the cardinality estimation follow a global approach considering the whole database schema at once. These global models are prone to sparse data at training leading to misestimates for queries which were not represented in the sample space used for generating training queries. To overcome this issue, we introduce a novel local-oriented approach in this paper, therefore the local context is a specific sub-part of the schema. As we will show, this leads to better representation of data correlation and thus better estimation accuracy. Compared to global approaches, our novel approach achieves an improvement by two orders of magnitude in accuracy and by a factor of four in training time performance for local models.

<p>Spiking Neural Networks (SNNs) have recently attracted significant research interest as the third generation of artificial neural networks that can enable low-power event-driven data analytics. We propose ANN-SNN conversion using “soft re-set” spiking neuron model, referred to as Residual Membrane Potential (RMP) spiking neuron, which retains the “resid- ual” membrane potential above threshold at the firing instants. In addition, we propose a time-based coding scheme, named Temporal-Switch-Coding (TSC), and a corresponding TSC spiking neuron model. Each input image pixel is presented using two spikes with opposite polarity and the timing between the two spiking instants is proportional to the pixel intensity. We demonstrate near loss-less ANN-SNN conversion using RMP neurons for VGG-16, ResNet-20, and ResNet-34 SNNs on challenging datasets including CIFAR-10, CIFAR-100, and ImageNet. With the help of TSC coding, it achieves 7-14.5× less inference latency, and 30-60× fewer addition operations and memory accesses per inference across datasets compared to the state of the art (SOTA) SNN models. In the second part of the thesis, we propose a new type of recurrent neural network (RNN) architecture, named Os- cillatory Fourier Neural Network (O-FNN). We demonstrate that O-FNN is mathematically equivalent to a simplified form of Discrete Fourier Transform applied onto periodical activa- tion. In particular, the computationally intensive back-propagation through time in training is eliminated, leading to faster training while achieving the SOTA inference accuracy in a diverse group of sequential tasks. For instance, applying the proposed model to sentiment analysis on IMDB review dataset reaches 89.4% test accuracy within 5 epochs, accompanied by over 35x reduction in the model size compared to Long Short-Term Memory (LSTM). The proposed novel RNN architecture is well poised for intelligent sequential processing in resource constrained hardware.</p>

Computer vision in the area of medical imaging has rapidly improved during recent years as a consequence of developments in deep learning and explainability algorithms. In addition, imaging in nuclear medicine is becoming increasingly sophisticated, with the emergence of targeted radiotherapies that enable treatment and imaging on a molecular level (“theranostics”) where radiolabeled targeted molecules are directly injected into the bloodstream. Based on our recent work, we present two use-cases in nuclear medicine as follows: first, the impact of automated organ segmentation required for personalized dosimetry in patients with neuroendocrine tumors and second, purely data-driven identification and verification of brain regions for diagnosis of Parkinson’s disease. Convolutional neural network was used for automated organ segmentation on computed tomography images. The segmented organs were used for calculation of the energy deposited into the organ-at-risk for patients treated with a radiopharmaceutical. Our method resulted in faster and cheaper dosimetry and only differed by 7% from dosimetry performed by two medical physicists. The identification of brain regions, however was analyzed on dopamine-transporter single positron emission tomography images using convolutional neural network and explainability, i.e., layer-wise relevance propagation algorithm. Our findings confirm that the extra-striatal brain regions, i.e., insula, amygdala, ventromedial prefrontal cortex, thalamus, anterior temporal cortex, superior frontal lobe, and pons contribute to the interpretation of images beyond the striatal regions. In current common diagnostic practice, however, only the striatum is the reference region, while extra-striatal regions are neglected. We further demonstrate that deep learning-based diagnosis combined with explainability algorithm can be recommended to support interpretation of this image modality in clinical routine for parkinsonian syndromes, with a total computation time of three seconds which is compatible with busy clinical workflow. Overall, this thesis shows for the first time that deep learning with explainability can achieve results competitive with human performance and generate novel hypotheses, thus paving the way towards improved diagnosis and treatment in nuclear medicine.

This thesis will cover the topic of cyber security in vehicles. Current vehicles contain many computers which communicate over a controller area network. This network has many vulnerabilities which can be leveraged by attackers. To combat these attackers, intrusion detection systems have been implemented. The latest research has mostly focused on the use of deep learning techniques for these intrusion detection systems. However, these deep learning techniques are not foolproof and possess their own security vulnerabilities. One such vulnerability comes in the form of adversarial samples. These are attacks that are manipulated to evade detection by these intrusion detection systems. In this thesis, the aim is to show that the known vulnerabilities of deep learning techniques are also present in the current state-of-the-art intrusion detection systems. The presence of these vulnerabilities shows that these deep learning based systems are still to immature to be deployed in actual vehicles. Since if an attacker is able to use these weaknesses to circumvent the intrusion detection system, they can still control many parts of the vehicles such as the windows, the brakes and even the engine. Current research regarding deep learning weaknesses has mainly focused on the image recognition domain. Relatively little research has investigated the influence of these weaknesses for intrusion detection, especially on vehicle networks. To show these weaknesses, firstly two baseline deep learning intrusion detection systems were created. Additionally, two state-of-the-art systems from recent research papers were recreated. Afterwards, adversarial samples were generated using the fast gradient-sign method on one of the baseline systems. These adversarial samples were then used to show the drop in performance of all systems. The thesis shows that the adversarial samples negatively impact the two baseline models and one state-of-the-art model. The state-of-the-art model’s drop in performance goes as high as 60% in the f1-score. Additionally, some of the adversarial samples need as little as 2 bits to be changed in order to evade the intrusion detection systems. / Detta examensarbete kommer att täcka ämnet cybersäkerhet i fordon. Nuvarande fordon innehåller många datorer som kommunicerar över ett så kallat controller area network. Detta nätverk har många sårbarheter som kan utnyttjas av angripare. För att bekämpa dessa angripare har intrångsdetekteringssystem implementerats. Den senaste forskningen har mestadels fokuserat på användningen av djupinlärningstekniker för dessa intrångsdetekteringssystem. Dessa djupinlärningstekniker är dock inte idiotsäkra och har sina egna säkerhetsbrister. En sådan sårbarhet kommer i form av kontradiktoriska prover. Dessa är attacker som manipuleras för att undvika upptäckt av dessa intrångsdetekteringssystem. I det här examensarbetet kommer vi att försöka visa att de kända sårbarheterna hos tekniker för djupinlärning också finns i de nuvarande toppmoderna systemen för intrångsdetektering. Förekomsten av dessa sårbarheter visar att dessa djupinlärningsbaserade system fortfarande är för omogna för att kunna användas i verkliga fordon. Eftersom om en angripare kan använda dessa svagheter för att kringgå intrångsdetekteringssystemet, kan de fortfarande kontrollera många delar av fordonet som rutorna, bromsarna och till och med motorn. Aktuell forskning om svagheter i djupinlärning har främst fokuserat på bildigenkänningsdomänen. Relativt lite forskning har undersökt inverkan av dessa svagheter för intrångsdetektering, särskilt på fordonsnätverk. För att visa dessa svagheter skapades först två baslinjesystem för djupinlärning intrångsdetektering. Dessutom återskapades två toppmoderna system från nya forskningsartiklar. Efteråt genererades motstridiga prover med hjälp av den snabba gradient-teckenmetoden på ett av baslinjesystemen. Dessa kontradiktoriska prover användes sedan för att visa nedgången i prestanda för alla system. Avhandlingen visar att de kontradiktoriska proverna negativt påverkar de två baslinjemodellerna och en toppmodern modell. Den toppmoderna modellens minskning av prestanda går så högt som 60% i f1-poängen. Dessutom behöver några av de kontradiktoriska samplen så lite som 2 bitar att ändras för att undvika intrångsdetekteringssystem.

Articular chondral lesions in the knee joint can be diagnosed at an early stage using MRI. Segmenting and visualizing lesions and the overall joint structure allows improved communication between the radiologist and referring physician. It can also be of help when determining diagnosis or conducting surgical planning. Although there are a variety of studies proving good results of segmentation of larger structures such as bone and cartilage in the knee, there are no studies available researching segmentation of articular cartilage lesions. Automating the segmentation will save time and money since manual segmentation is very time-consuming. In this thesis, a U-Net based convolutional neural network is used to perform automatic segmentation of chondral lesions located on the distal part of the femur, in the knee joint. Using two different techniques, batch normalization and dropout, a network was trained and tested using MRI sequences collected from Episurf Medical's database. The network was then evaluated using a segmentation approach and a detection approach. For the segmentation approach, the highest achieved dice coefficient and sensitivity of 0.4059 ± 0.1833 and 0.4591 ± 0.2387, was obtained using batch normalization and 260 training subjects, consisting of MRI sequence and corresponding masks. Using a detection approach, the predicted output could correctly identify 81.8% of the chondral lesions in the MRI sequences. Although there is a need for improvement of technique and datasets used in this thesis, the achieved results show prerequisites for future improvement and possible implementation. / Skador i knäledens brosk kan diagnostiseras i ett tidigt stadie med hjälp av MR. Segmentering och visualisering av skadorna, samt ledens struktur i helhet, bidrar till en förbättrad kommunikation mellan radiolog och remitterande läkare. Det kan också underlätta för att ställa diagnos eller utföra operationsplanering. I dagsläget finns flertalet studier som påvisar goda resultat för segmentering av större strukturer, t.ex. ben och brosk. Det finns dock få studier som studerar segmentering av skador i ledbrosk. Genom att automatisera segmenteringsprocessen kan både tid och pengar sparas. Detta eftersom att manuell segmentering är mycket tidskrävande. I detta arbete kommer ett U-Net baserat convolutional neural network att användas för att utföra automatisk segmentering av skador på distala femur i knäleden. Nätverket kommer att tränas med två olika tekniker, batch normalization och dropout. Nätverket kommer att tränas med data som är hämtad från Episurf Medicals databas och består av MR sekvenser. Nätverket kommer att tränas och utvärderas med hjälp av två metoder, en segmenteringsmetod och detekteringsmetod. Den högsta uppnådda dice koefficienten och sensitiviteten vid utvärderingen av segmenteringsmetoden uppmätte 0,4059 ± 0,1833 och 0,4591 ± 0,2387. Den upnåddes med hjälp av batch normalization och 260 MR sekvenser för träning och testning. För detektionsmetoden kunde programmet identifiera 81,8% av skadorna synliga på MR sekvenserna. Även om tekniken och datan som används behöver optimeras, så visar det uppnådda resultatet på bra förutsättningar för fortsatta studier och i framtiden möjligen även implementering av tekniken.

Named Entity Recognition (NER) is a critical task in Natural Language Processing (NLP), and recent advancements in language model pre-training have significantly improved its performance. However, this improvement is not universally applicable due to a lack of large pre-training datasets or computational budget for smaller languages. This study explores the viability of fine-tuning an English and a multilingual model on a Swedish NER task, compared to a model trained solely on Swedish. Our methods involved training these models and measuring their performance using the F1-score metric. Despite fine-tuning, the Swedish model outperformed both the English and multilingual models by 3.0 and 9.0 percentage points, respectively. The performance gap between the English and Swedish models during fine-tuning decreased from 19.8 to 9.0 percentage points. This suggests that while the Swedish model achieved the best performance, fine-tuning can substantially enhance the performance of English and multilingual models for Swedish NER tasks. / Inom området för Natural Language Processing (NLP) är identifiering av namngivna entiteter (NER) en viktig problemtyp. Tack vare senaste tidens framsteg inom förtränade språkmodeller har modellernas prestanda på problemtypen ökat kraftigt. Denna förbättring kan dock inte tillämpas överallt på grund av en brist på omfattande dataset för förträning eller tillräcklig datorkraft för mindre språk. I denna studie undersöks potentialen av fine-tuning på både en engelsk, en svensk och en flerspråkig modell för en svensk NER-uppgift. Dessa modeller tränades och deras effektivitet bedömdes genom att använda F1-score som mått på prestanda. Även med fine-tuning var den svenska modellen bättre än både den engelska och flerspråkiga modellen, med en skillnad på 3,0 respektive 9,0 procentenheter i F1-score. Skillnaden i prestandan mellan den engelska och svenska modellen minskade från 19,8 till 9,0 procentenheter efter fine-tuning. Detta indikerar att även om den svenska modellen var mest framgångsrik, kan fine-tuning av engelska och flerspråkiga modeller betydligt förbättra prestandan för svenska NER-uppgifter.

Emerging sensors and intelligent traffic technologies provide extensive data sets in a traffic network. However, realizing the full potential of such data sets for a unique representation of real-world states is challenging due to data accuracy, noise, and temporal-spatial resolution. Data assimilation is a known group of methodological approaches that exploit physics-informed traffic models and data observations to perform short-term predictions of the traffic state in freeway environments. At the same time, neural networks capture high non-linearities, similar to those presented in traffic networks. Despite numerous works applying different variants of Kalman filters, the possibility of traffic state estimation with deep-learning-based methodologies is only partially explored in the literature. We present a deep-learning modeling approach to perform traffic state estimation on large freeway networks. The proposed framework is trained on local observations from static and moving sensors and identifies differences between well-trusted data and model outputs. The detected patterns are then used throughout the network, even where there are no available observations to estimate fundamental traffic quantities. The preliminary results of the work highlight the potential of deep learning for traffic state estimation.