Image segmentation through neural networks and deep learning have, in the recent decade, become a successful tool for automated decision-making. For Luossavaara-Kiirunavaara Aktiebolag (LKAB), this means identifying the amount of slag inside a furnace through computer vision.  There are many prominent convolutional neural network architectures in the literature, and this thesis explores two: a modified U-Net and the PSPNet. The architectures were combined with three loss functions and three class weighting schemes resulting in 18 model configurations that were evaluated and compared. This thesis also explores transfer learning techniques for neural networks tasked with identifying slag in images from inside a furnace. The benefit of transfer learning is that the network can learn to find features from already labeled data of another context. Finally, the thesis explored how temporal information could be utilised by adding an LSTM layer to a model taking pairs of images as input, instead of one. The results show (1) that the PSPNet outperformed the U-Net for all tested configurations in all relevant metrics, (2) that the model is able to find more complex features while converging quicker by using transfer learning, and (3) that utilising temporal information reduced the variance of the predictions, and that the modified PSPNet using an LSTM layer showed promise in handling images with outlying characteristics.

Reinforcement learning has proven itself very useful in certain areas, such as games. However, the approach has been seen as quite limited. Reinforcement-based learning has for instance not been commonly used for classification tasks as it is receiving feedback on how well it did for an action performed on a specific input. This slows the performance convergence rate as compared to other classification approaches which has the input and the corresponding output to train on. Nevertheless, this thesis aims to investigate whether reinforcement-based learning could successfully be employed on a classification task. Moreover, as sensor fusion is an expanding field which can for instance assist autonomous vehicles in understanding its surroundings, it is also interesting to see how sensor fusion, i.e., fusion between lidar and RGB images, could increase the performance in a classification task. In this thesis, a reinforcement-based learning approach is compared to a semi-classical approach. As an example of a reinforcement learning model, a deep Q-learning network was chosen, and a support vector machine classifier built on top of a deep neural network, was chosen as an example of a semi-classical model. In this work, these frameworks are compared with and without sensor fusion to see whether fusion improves their performance. Experiments show that the evaluated reinforcement-based learning approach underperforms in terms of metrics but mainly due to its slow learning process, in comparison to the semi-classical approach. However, on the other hand using reinforcement-based learning to carry out a classification task could still in some cases be advantageous, as it still performs fairly well in terms of the metrics presented in this work, e.g. F1-score, or for instance imbalanced datasets. As for the impact of sensor fusion, a notable improvement can be seen, e.g. when training the deep Q-learning model for 50 episodes, the F1-score increased with 0.1329; especially, when taking into account that the most of the lidar data used in the fusion is lost since this work projects the 3D lidar data onto the same 2D plane as the RGB images.

Computer vision in the area of medical imaging has rapidly improved during recent years as a consequence of developments in deep learning and explainability algorithms. In addition, imaging in nuclear medicine is becoming increasingly sophisticated, with the emergence of targeted radiotherapies that enable treatment and imaging on a molecular level (“theranostics”) where radiolabeled targeted molecules are directly injected into the bloodstream. Based on our recent work, we present two use-cases in nuclear medicine as follows: first, the impact of automated organ segmentation required for personalized dosimetry in patients with neuroendocrine tumors and second, purely data-driven identification and verification of brain regions for diagnosis of Parkinson’s disease. Convolutional neural network was used for automated organ segmentation on computed tomography images. The segmented organs were used for calculation of the energy deposited into the organ-at-risk for patients treated with a radiopharmaceutical. Our method resulted in faster and cheaper dosimetry and only differed by 7% from dosimetry performed by two medical physicists. The identification of brain regions, however was analyzed on dopamine-transporter single positron emission tomography images using convolutional neural network and explainability, i.e., layer-wise relevance propagation algorithm. Our findings confirm that the extra-striatal brain regions, i.e., insula, amygdala, ventromedial prefrontal cortex, thalamus, anterior temporal cortex, superior frontal lobe, and pons contribute to the interpretation of images beyond the striatal regions. In current common diagnostic practice, however, only the striatum is the reference region, while extra-striatal regions are neglected. We further demonstrate that deep learning-based diagnosis combined with explainability algorithm can be recommended to support interpretation of this image modality in clinical routine for parkinsonian syndromes, with a total computation time of three seconds which is compatible with busy clinical workflow. Overall, this thesis shows for the first time that deep learning with explainability can achieve results competitive with human performance and generate novel hypotheses, thus paving the way towards improved diagnosis and treatment in nuclear medicine.

[ES] Durante los últimos años, los repositorios multimedia en línea se han convertido en fuentes clave de conocimiento gracias al auge de Internet, especialmente en el área de la educación. Instituciones educativas de todo el mundo han dedicado muchos recursos en la búsqueda de nuevos métodos de enseñanza, tanto para mejorar la asimilación de nuevos conocimientos, como para poder llegar a una audiencia más amplia. Como resultado, hoy en día disponemos de diferentes repositorios con clases grabadas que siven como herramientas complementarias en la enseñanza, o incluso pueden asentar una nueva base en la enseñanza a distancia. Sin embargo, deben cumplir con una serie de requisitos para que la experiencia sea totalmente satisfactoria y es aquí donde la transcripción de los materiales juega un papel fundamental. La transcripción posibilita una búsqueda precisa de los materiales en los que el alumno está interesado, se abre la puerta a la traducción automática, a funciones de recomendación, a la generación de resumenes de las charlas y además, el poder hacer llegar el contenido a personas con discapacidades auditivas. No obstante, la generación de estas transcripciones puede resultar muy costosa. Con todo esto en mente, la presente tesis tiene como objetivo proporcionar nuevas herramientas y técnicas que faciliten la transcripción de estos repositorios. En particular, abordamos el desarrollo de un conjunto de herramientas de reconocimiento de automático del habla, con énfasis en las técnicas de aprendizaje profundo que contribuyen a proporcionar transcripciones precisas en casos de estudio reales. Además, se presentan diferentes participaciones en competiciones internacionales donde se demuestra la competitividad del software comparada con otras soluciones. Por otra parte, en aras de mejorar los sistemas de reconocimiento, se propone una nueva técnica de adaptación de estos sistemas al interlocutor basada en el uso Medidas de Confianza. Esto además motivó el desarrollo de técnicas para la mejora en la estimación de este tipo de medidas por medio de Redes Neuronales Recurrentes. Todas las contribuciones presentadas se han probado en diferentes repositorios educativos. De hecho, el toolkit transLectures-UPV es parte de un conjunto de herramientas que sirve para generar transcripciones de clases en diferentes universidades e instituciones españolas y europeas. / [CAT] Durant els últims anys, els repositoris multimèdia en línia s'han convertit en fonts clau de coneixement gràcies a l'expansió d'Internet, especialment en l'àrea de l'educació. Institucions educatives de tot el món han dedicat molts recursos en la recerca de nous mètodes d'ensenyament, tant per millorar l'assimilació de nous coneixements, com per poder arribar a una audiència més àmplia. Com a resultat, avui dia disposem de diferents repositoris amb classes gravades que serveixen com a eines complementàries en l'ensenyament, o fins i tot poden assentar una nova base a l'ensenyament a distància. No obstant això, han de complir amb una sèrie de requisits perquè la experiència siga totalment satisfactòria i és ací on la transcripció dels materials juga un paper fonamental. La transcripció possibilita una recerca precisa dels materials en els quals l'alumne està interessat, s'obri la porta a la traducció automàtica, a funcions de recomanació, a la generació de resums de les xerrades i el poder fer arribar el contingut a persones amb discapacitats auditives. No obstant, la generació d'aquestes transcripcions pot resultar molt costosa. Amb això en ment, la present tesi té com a objectiu proporcionar noves eines i tècniques que faciliten la transcripció d'aquests repositoris. En particular, abordem el desenvolupament d'un conjunt d'eines de reconeixement automàtic de la parla, amb èmfasi en les tècniques d'aprenentatge profund que contribueixen a proporcionar transcripcions precises en casos d'estudi reals. A més, es presenten diferents participacions en competicions internacionals on es demostra la competitivitat del programari comparada amb altres solucions. D'altra banda, per tal de millorar els sistemes de reconeixement, es proposa una nova tècnica d'adaptació d'aquests sistemes a l'interlocutor basada en l'ús de Mesures de Confiança. A més, això va motivar el desenvolupament de tècniques per a la millora en l'estimació d'aquest tipus de mesures per mitjà de Xarxes Neuronals Recurrents. Totes les contribucions presentades s'han provat en diferents repositoris educatius. De fet, el toolkit transLectures-UPV és part d'un conjunt d'eines que serveix per generar transcripcions de classes en diferents universitats i institucions espanyoles i europees. / [EN] During the last years, on-line multimedia repositories have become key knowledge assets thanks to the rise of Internet and especially in the area of education. Educational institutions around the world have devoted big efforts to explore different teaching methods, to improve the transmission of knowledge and to reach a wider audience. As a result, online video lecture repositories are now available and serve as complementary tools that can boost the learning experience to better assimilate new concepts. In order to guarantee the success of these repositories the transcription of each lecture plays a very important role because it constitutes the first step towards the availability of many other features. This transcription allows the searchability of learning materials, enables the translation into another languages, provides recommendation functions, gives the possibility to provide content summaries, guarantees the access to people with hearing disabilities, etc. However, the transcription of these videos is expensive in terms of time and human cost. To this purpose, this thesis aims at providing new tools and techniques that ease the transcription of these repositories. In particular, we address the development of a complete Automatic Speech Recognition Toolkit with an special focus on the Deep Learning techniques that contribute to provide accurate transcriptions in real-world scenarios. This toolkit is tested against many other in different international competitions showing comparable transcription quality. Moreover, a new technique to improve the recognition accuracy has been proposed which makes use of Confidence Measures, and constitutes the spark that motivated the proposal of new Confidence Measures techniques that helped to further improve the transcription quality. To this end, a new speaker-adapted confidence measure approach was proposed for models based on Recurrent Neural Networks. The contributions proposed herein have been tested in real-life scenarios in different educational repositories. In fact, the transLectures-UPV toolkit is part of a set of tools for providing video lecture transcriptions in many different Spanish and European universities and institutions. / Agua Teba, MÁD. (2019). CONTRIBUTIONS TO EFFICIENT AUTOMATIC TRANSCRIPTION OF VIDEO LECTURES [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/130198 / TESIS

No description available.

No description available.

Krebs ist die zweithäufigste Todesursache weltweit und zählt damit zu den größten Plagen der Menschheit. Jährlich sterben Menschen an den Folgen bösartiger Tumore und stellen Ärzte vor scheinbar unlösbare Aufgaben. Um Krebsgeschwüre effizient bekämpfen oder sogar vollständig beseitigen zu können, ist es enorm wichtig diese früh genug zu diagnostizieren. Oft stellt jedoch genau das in der Praxis ein großes Problem dar und Tumore werden erst dann als solche erkannt, wenn das Zellwachstum schon sehr weit fortgeschritten ist. Eine große Chance für die frühzeitige Erkennung von Krebs bieten unterdessen Deep Learning Algorithmen. Die vorliegende Seminararbeit stellt diese Verfahren und ihre Anwendung in der Krebsdiagnostik vor. Es wird hierbei genauer auf Convolutional Neural Networks eingegangen, die besonders gut geeignet für die Analyse von Gewebebildern sind und unter anderem auch im System von Google's DeepMind zum Einsatz kommen. Die Arbeit analysiert Chancen und Risiken des Einsatzes von Deep Neural Networks bei der Diagnose von bösartigen Tumoren und verschafft dem Leser damit einen ganzheitlichen Überblick über die Anwendung von Deep Neural Networks im Bereich der Onkologie.:1 Einleitung 2 Vom Neuronalen Netz zum Deep Learning Algorithmus 2.1 Grundlagen Künstlicher Neuronaler Netze 2.1.1 Allgemeiner Aufbau 2.1.2 Das Neuron als Grundbaustein 2.1.3 Lernen in neuronalen Netzen 2.1.4 Loss Function und Optimizer 2.2 Convolutional Neural Networks 2.2.1 Convolutional Layer 2.2.2 Pooling Layer 2.2.3 Fully Connected Layer 2.2.4 Lernen und Aktivierung in CNN’s 3 DeepMind als Deep Learning Multitalent 3.1 Bisherige Erfolge 3.2 DeepMind Health 4 Chancen und Risiken in der Krebsdiagnostik 4.1 Aktueller Stand der Brustkrebsdiagnostik 4.2 Chancen von Deep Learning Algorithmen 4.3 Ethische Risiken 4.3.1 False Positives 4.3.2 False Negatives 4.4 Fazit der Risikoanalyse 5 Ausblick

In relation to the increasing interest in implementing smart cities, deployment of widespread wireless sensor networks (WSNs) has become a current hot topic. Among the application’s greatest challenges, there is still progress to be made concerning energy consumption and quality of service. Consequently, this project aims to explore a series of feasible solutions to improve the WSN energy efficiency for data aggregation by the WSN. This by strategically adjusting the position of the receiving base station and the packet rate of the WSN nodes. Additionally, the low-energy adaptive clustering hierarchy (LEACH) protocol is coupled with the WSN state of charge (SoC). For this thesis, a WSN was defined as a two dimensional area which contains sensor nodes and a mobile sink, i.e. a movable base station. Subsequent to the rigorous analyses of the WSN data clustering principles and system-wide dynamics, two different developing strategies, model-based and data-driven designs, were employed to develop two corresponding control approaches, model predictive control and reinforcement learning, on WSN energy management. To test their performance, a simulation environment was thus developed in Python, including the extended LEACH protocol. The amount of data transmitted by an energy unit is adopted as the index to estimate the control performance. The simulation results show that the model based controller was able to aggregate over 22% more bits than only using the LEACH protocol. Whilst the data driven controller had a worse performance than the LEACH network but showed potential for smaller sized WSNs containing a fewer amount of nodes. Nonetheless, the extension of the LEACH protocol did not give rise to obvious improvement on energy efficiency due to a wide range of differing results. / I samband med det ökande intresset för att implementera så kallade smart cities, har användningen av utbredda trådlösa sensor nätverk (WSN) blivit ett intresseområde. Bland applikationens största utmaningar, finns det fortfarande förbättringar med avseende på energiförbrukning och servicekvalité. Därmed så inriktar sig detta projekt på att utforska en mängd möjliga lösningar för att förbättra energieffektiviteten för dataaggregation inom WSN. Detta gjordes genom att strategiskt justera positionen av den mottagande basstationen samt paketfrekvensen för varje nod. Dessutom påbyggdes low-energy adaptive clustering hierarchy (LEACH) protokollet med WSN:ets laddningstillstånd. För detta examensarbete definierades ett WSN som ett två dimensionellt plan som innehåller sensor noder och en mobil basstation, d.v.s. en basstation som går att flytta. Efter rigorös analys av klustringsmetoder samt dynamiken av ett WSN, utvecklades två kontrollmetoder som bygger på olika kontrollstrategier. Dessa var en modelbaserad MPC kontroller och en datadriven reinforcement learning kontroller som implementerades för att förbättra energieffektiviteten i WSN. För att testa prestandan på dom två kontrollmetoderna, utvecklades en simulations platform baserat på Python, tillsamans med påbyggnaden av LEACH protokollet. Mängden data skickat per energienhet användes som index för att approximera kontrollprestandan. Simuleringsresultaten visar att den modellbaserade kontrollern kunde öka antalet skickade datapacket med 22% jämfört med när LEACH protokollet användes. Medans den datadrivna kontrollern hade en sämre prestanda jämfört med när enbart LEACH protokollet användes men den visade potential för WSN med en mindre storlek. Påbyggnaden av LEACH protokollet gav ingen tydlig ökning med avseende på energieffektiviteten p.g.a. en mängd avvikande resultat.

This thesis will cover the topic of cyber security in vehicles. Current vehicles contain many computers which communicate over a controller area network. This network has many vulnerabilities which can be leveraged by attackers. To combat these attackers, intrusion detection systems have been implemented. The latest research has mostly focused on the use of deep learning techniques for these intrusion detection systems. However, these deep learning techniques are not foolproof and possess their own security vulnerabilities. One such vulnerability comes in the form of adversarial samples. These are attacks that are manipulated to evade detection by these intrusion detection systems. In this thesis, the aim is to show that the known vulnerabilities of deep learning techniques are also present in the current state-of-the-art intrusion detection systems. The presence of these vulnerabilities shows that these deep learning based systems are still to immature to be deployed in actual vehicles. Since if an attacker is able to use these weaknesses to circumvent the intrusion detection system, they can still control many parts of the vehicles such as the windows, the brakes and even the engine. Current research regarding deep learning weaknesses has mainly focused on the image recognition domain. Relatively little research has investigated the influence of these weaknesses for intrusion detection, especially on vehicle networks. To show these weaknesses, firstly two baseline deep learning intrusion detection systems were created. Additionally, two state-of-the-art systems from recent research papers were recreated. Afterwards, adversarial samples were generated using the fast gradient-sign method on one of the baseline systems. These adversarial samples were then used to show the drop in performance of all systems. The thesis shows that the adversarial samples negatively impact the two baseline models and one state-of-the-art model. The state-of-the-art model’s drop in performance goes as high as 60% in the f1-score. Additionally, some of the adversarial samples need as little as 2 bits to be changed in order to evade the intrusion detection systems. / Detta examensarbete kommer att täcka ämnet cybersäkerhet i fordon. Nuvarande fordon innehåller många datorer som kommunicerar över ett så kallat controller area network. Detta nätverk har många sårbarheter som kan utnyttjas av angripare. För att bekämpa dessa angripare har intrångsdetekteringssystem implementerats. Den senaste forskningen har mestadels fokuserat på användningen av djupinlärningstekniker för dessa intrångsdetekteringssystem. Dessa djupinlärningstekniker är dock inte idiotsäkra och har sina egna säkerhetsbrister. En sådan sårbarhet kommer i form av kontradiktoriska prover. Dessa är attacker som manipuleras för att undvika upptäckt av dessa intrångsdetekteringssystem. I det här examensarbetet kommer vi att försöka visa att de kända sårbarheterna hos tekniker för djupinlärning också finns i de nuvarande toppmoderna systemen för intrångsdetektering. Förekomsten av dessa sårbarheter visar att dessa djupinlärningsbaserade system fortfarande är för omogna för att kunna användas i verkliga fordon. Eftersom om en angripare kan använda dessa svagheter för att kringgå intrångsdetekteringssystemet, kan de fortfarande kontrollera många delar av fordonet som rutorna, bromsarna och till och med motorn. Aktuell forskning om svagheter i djupinlärning har främst fokuserat på bildigenkänningsdomänen. Relativt lite forskning har undersökt inverkan av dessa svagheter för intrångsdetektering, särskilt på fordonsnätverk. För att visa dessa svagheter skapades först två baslinjesystem för djupinlärning intrångsdetektering. Dessutom återskapades två toppmoderna system från nya forskningsartiklar. Efteråt genererades motstridiga prover med hjälp av den snabba gradient-teckenmetoden på ett av baslinjesystemen. Dessa kontradiktoriska prover användes sedan för att visa nedgången i prestanda för alla system. Avhandlingen visar att de kontradiktoriska proverna negativt påverkar de två baslinjemodellerna och en toppmodern modell. Den toppmoderna modellens minskning av prestanda går så högt som 60% i f1-poängen. Dessutom behöver några av de kontradiktoriska samplen så lite som 2 bitar att ändras för att undvika intrångsdetekteringssystem.

Articular chondral lesions in the knee joint can be diagnosed at an early stage using MRI. Segmenting and visualizing lesions and the overall joint structure allows improved communication between the radiologist and referring physician. It can also be of help when determining diagnosis or conducting surgical planning. Although there are a variety of studies proving good results of segmentation of larger structures such as bone and cartilage in the knee, there are no studies available researching segmentation of articular cartilage lesions. Automating the segmentation will save time and money since manual segmentation is very time-consuming. In this thesis, a U-Net based convolutional neural network is used to perform automatic segmentation of chondral lesions located on the distal part of the femur, in the knee joint. Using two different techniques, batch normalization and dropout, a network was trained and tested using MRI sequences collected from Episurf Medical's database. The network was then evaluated using a segmentation approach and a detection approach. For the segmentation approach, the highest achieved dice coefficient and sensitivity of 0.4059 ± 0.1833 and 0.4591 ± 0.2387, was obtained using batch normalization and 260 training subjects, consisting of MRI sequence and corresponding masks. Using a detection approach, the predicted output could correctly identify 81.8% of the chondral lesions in the MRI sequences. Although there is a need for improvement of technique and datasets used in this thesis, the achieved results show prerequisites for future improvement and possible implementation. / Skador i knäledens brosk kan diagnostiseras i ett tidigt stadie med hjälp av MR. Segmentering och visualisering av skadorna, samt ledens struktur i helhet, bidrar till en förbättrad kommunikation mellan radiolog och remitterande läkare. Det kan också underlätta för att ställa diagnos eller utföra operationsplanering. I dagsläget finns flertalet studier som påvisar goda resultat för segmentering av större strukturer, t.ex. ben och brosk. Det finns dock få studier som studerar segmentering av skador i ledbrosk. Genom att automatisera segmenteringsprocessen kan både tid och pengar sparas. Detta eftersom att manuell segmentering är mycket tidskrävande. I detta arbete kommer ett U-Net baserat convolutional neural network att användas för att utföra automatisk segmentering av skador på distala femur i knäleden. Nätverket kommer att tränas med två olika tekniker, batch normalization och dropout. Nätverket kommer att tränas med data som är hämtad från Episurf Medicals databas och består av MR sekvenser. Nätverket kommer att tränas och utvärderas med hjälp av två metoder, en segmenteringsmetod och detekteringsmetod. Den högsta uppnådda dice koefficienten och sensitiviteten vid utvärderingen av segmenteringsmetoden uppmätte 0,4059 ± 0,1833 och 0,4591 ± 0,2387. Den upnåddes med hjälp av batch normalization och 260 MR sekvenser för träning och testning. För detektionsmetoden kunde programmet identifiera 81,8% av skadorna synliga på MR sekvenserna. Även om tekniken och datan som används behöver optimeras, så visar det uppnådda resultatet på bra förutsättningar för fortsatta studier och i framtiden möjligen även implementering av tekniken.