Deep learning has been rapidly growing in recent years obtaining excellent results for many computer vision applications, such as image classification and object detection. One aspect for the increased popularity of deep learning is that it mitigates the need for hand-crafted features. This thesis work investigates deep learning as a methodology to solve the problem of autonomous collision avoidance for a small robotic car. To accomplish this, transfer learning is used with the VGG16 deep network pre-trained on ImageNet dataset. A dataset has been collected and then used to fine-tune and validate the network offline. The deep network has been used with the robotic car in a real-time manner. The robotic car sends images to an external computer, which is used for running the network. The predictions from the network is sent back to the robotic car which takes actions based on those predictions. The results show that deep learning has great potential in solving the collision avoidance problem.

The 2019 novel coronavirus has been proven to present several unique features on chest X-rays and CT-scans that distinguish it from imaging of other pulmonary diseases such as bacterial pneumonia and viral pneumonia unrelated to COVID-19. However, the key characteristics of a COVID-19 infection have been proven challenging to detect with the human eye. The aim of this project is to explore if it is possible to distinguish a patient with COVID-19 from a patient who is not suffering from the disease from posteroanterior chest X-ray images using synthetic minority over-sampling and transfer learning. Furthermore, the report will also present the mechanics of COVID-19, the used dataset and models and the validity of the results.

Monte Carlo methods in derivative pricing are computationally expensive, in particular for evaluating models partial derivatives with regard to inputs. This research proposes the use of deep learning to approximate such valuation models for highly exotic derivatives, using automatic differentiation to evaluate input sensitivities. Deep learning models are trained to approximate Phoenix Autocall valuation using a proprietary model used by Svenska Handelsbanken AB. Models are trained on large datasets of low-accuracy (10^4 simulations) Monte Carlo data, successfully learning the true model with an average error of 0.1% on validation data generated by 10^8 simulations. A specific model parametrisation is proposed for 2-day valuation only, to be recalibrated interday using transfer learning. Automatic differentiation approximates sensitivity to (normalised) underlying asset prices with a mean relative error generally below 1.6%. Overall error when predicting sensitivity to implied volatililty is found to lie within 10%-40%. Near identical results are found by finite difference as automatic differentiation in both cases. Automatic differentiation is not successful at capturing sensitivity to interday contract change in value, though errors of 8%-25% are achieved by finite difference. Model recalibration by transfer learning proves to converge over 15 times faster and with up to 14% lower relative error than training using random initialisation. The results show that deep learning models can efficiently learn Monte Carlo valuation, and that these can be quickly recalibrated by transfer learning. The deep learning model gradient computed by automatic differentiation proves a good approximation of the true model sensitivities. Future research proposals include studying optimised recalibration schedules, using training data generated by single Monte Carlo price paths, and studying additional parameters and contracts.

In the scheme of Non Destructive Testing (NDT), defect detection is an important process. Traditional image processing techniques have successfully been used for defect recognition. Usage of machine learning techniques is still in the initial stages of development. Convolution Neural Networks (CNN) is widely used for object classification one such scenario is defect classification in weld tubes. With the advent of deep learning techniques such as transfer learning, we can transfer knowledge gained in one domain successfully into other. Pre-trained models successfully learn features from large scale datasets that can be used for in domains having sparse data and smaller datasets. The aim of this work is to help a manual inspector in recognition of defects on the weld tubes. With a given set of images, we proceed by forming unique pipeline architecture for automatic defect recognition. The research in this thesis focuses on extraction of welds using image segmentation techniques, creating a dataset of defects and using it to on pre-trained Convolution Neural Networks of VGG16, VGG19, Inception V3 and ResNet101. We evaluate the models on different metrics finding the best suited model for the created dataset. Further a prototype sliding window solution is used to find defects over the extracted weld region. We also present the limitations of this approach and suggest modifications that can be implemented in the future.

<div>Supervised classification methods based on deep learning have achieved great success in many domains and tasks that are previously unimaginable. Such approaches build on learning paradigms that require hundreds of examples in order to learn to classify objects or events. Thus, their immediate application to the domains with few or no observations is limited. This is because of the lack of ability to rapidly generalize to new categories from a few examples or from high-level descriptions of categories. This can be attributed to the significant gap between the way machines represent knowledge and the way humans represent categories in their minds and learn to recognize them. In this context, this research represents categories as semantic trees in a high-level attribute space and proposes an approach to utilize these representations to conduct N-Shot, Few-Shot, One-Shot, and Zero-Shot Learning (ZSL). This work refers to this paradigm as the problem of general classification (GCP) and proposes a unified framework for GCP referred to as the Few and Zero-Shot Technique (FAZT). FAZT framework is an end-to-end approach that uses trainable 3D convolutional neural networks and recurrent neural networks to simultaneously optimize for both the semantic and the classification tasks. Lastly, the problem of systematically obtaining semantic attributes by utilizing domain-specific ontologies is presented. The proposed framework is validated in the domains of hand gesture and action/activity recognition, however, this research can be applied to other domains such as video understanding, the study of human behavior, emotion recognition, etc. First, an attribute-based dataset for gestures is developed in a systematic manner by relying on literature in gestures and semantics, and crowdsourced platforms such as Amazon Mechanical Turk. To the best of our knowledge, this is the first ZSL dataset for hand gestures (ZSGL dataset). Next, our framework is evaluated in two experimental conditions: 1. Within-category (to test the attribute recognition power) and 2. Across-category (to test the ability to recognize an unknown category). In addition, we conducted experiments in zero-shot, one-shot, few-shot and continuous learning conditions in both open-set and closed-set scenarios. Results showed that our framework performs favorably on the ZSGL, Kinetics, UIUC Action, UCF101 and HMDB51 action datasets in all the experimental conditions.<br></div><div><br></div>

Biomechanical Big Data is of great significance to precision health applications, among which we take special interest in Physical Activity Detection (PAD). In this study, we have performed extensive research on deep learning-based PAD from biomechanical big data, focusing on the challenges raised by the need of real-time edge inference. First, considering there are many places we can place the motion sensors, we have thoroughly compared and analyzed the location difference in terms of deep learning-based PAD performance. We have further compared the difference among six sensor channels (3-axis accelerometer and 3-axis gyroscope). Second, we have selected the optimal sensor and the optimal sensor channel, which can not only provide sensor usage suggestions but also enable ultra-low-power application on the edge. Third, we have investigated innovative methods to minimize the training effort of the deep learning model, leveraging the transfer learning strategy. More specifically, we propose to pre-train a transferable deep learning model using the data from other subjects and then fine-tune the model using limited data from the target-user. In such a way, we have found that, for single-channel case, the transfer learning can effectively increase the deep model performance even when the fine-tuning effort is very small. This research, demonstrated by comprehensive experimental evaluation, have shown the potential of ultra-low-power PAD with minimized sensor stream and minimized training effort.

This master’s thesis deals with the implementation of functional solution for classifying road users using mobile device with Android operating system. The goal is to create Android application which classifies vehicles in real time using rear-facing camera and saves timestamps of classification. Testing is performed mostly with own, diversely modificated dataset. Five models are trained and their performance is measured in dependence on hardware. The best classification performance is from pretrained MobileNet model where transfer learning with 6 classes of own dataset is used – 62,33 %. The results are summarized and a method for faster and more accurate traffic analysis is proposed.

This thesis explores multi-task learning for combustion flame characterization i.e to learn different characteristics of the combustion flame. We propose a multi-task convolutional neural network for two tasks i.e. PFR (Pilot fuel ratio) and fuel type classification based on the images of stable combustion. We utilize transfer learning and adopt VGG16 to develop a multi-task convolutional neural network to jointly learn the aforementioned tasks. We also compare the performance of the individual CNN model for two tasks with multi-task CNN which learns these two tasks jointly by sharing visual knowledge among the tasks. We share the effectiveness of our proposed approach to a private company’s dataset. To the best of our knowledge, this is the first work being done for jointly learning different characteristics of the combustion flame. / <p>This wrok as done with Siemens, and we have applied for a patent which is still pending.</p>

The Internet of Things is a constantly developing field. With advancements of algorithms for object detection and classification for images and videos, the possibilities of what can be made with small and cost efficient edge-devices are increasing. This work presents how camera traps and deep learning can be utilized for surveillance in remote environments, such as animal sanctuaries in the African Savannah. The camera traps connect to a smart surveillance network where images and sensor-data are analysed. The analysis can then be used to produce valuable information, such as the location of endangered animals or unauthorized humans, to park rangers working to protect the wildlife in these animal sanctuaries. Different motion detection algorithms are tested and evaluated based on related research within the subject. The work made in this thesis builds upon two previous theses made within Project Ngulia. The implemented surveillance system in this project consists of camera sensors, a database, a REST API, a classification service, a FTP-server and a web-dashboard for displaying sensor data and resulting images. A contribution of this work is an end-to-end smart surveillance system that can use different camera sources to produce valuable information to stakeholders. The camera software developed in this work is targeting the ESP32 based M5Stack Timer Camera and runs a motion detection algorithm based on Self-Organizing Maps. This improves the selection of data that is fed to the image classifier on the server. This thesis also contributes with an algorithm for doing iterative image classifications that handles the issues of objects taking up small parts of an image, making them harder to classify correctly.

The purpose of this master thesis was to investigate if using both X-ray images and patient's register data could increase the performance of a neural network in discrimination of two types of fractures in the thigh bone, called atypical femoral fractures (AFF) and normal femoral fractures (NFF). We also examined and evaluated how the fusion of the two data types could be done and how different types of fusion affect the performance. Finally, we evaluated how the number of variables in the register data affect a network's performance. Our image dataset consisted of 1,442 unique images from 580 patients (16.85% of the images were labelled AFF corresponding to 15.86% of the patients). Since the dataset is very imbalanced, sensitivity is a prioritized evaluation metric. The register data network was evaluated using five different versions of register data parameters: two (age and sex), seven (binary and non-binary) and 44 (binary and non-binary). Having only age and sex as input resulted in a classifier predicting all samples to class 0 (NFF), for all tested network architectures. Using a certain network structure (celled register data model 2), in combination with the seven non-binary parameters outperforms using both two and 44 (both binary and non-binary) parameters regarding mean AUC and sensitivity. Highest mean accuracy is obtained by using 44 non-binary parameters. The seven register data parameters have a known connection to AFF and includes age and sex. The network with X-ray images as input uses a transfer learning approach with a pre-trained ResNet50-base. This model performed better than all the register data models, regarding all considered evaluation metrics.        Three fusion architectures were implemented and evaluated: probability fusion (PF), feature fusion (FF) and learned feature fusion (LFF). PF concatenates the prediction provided from the two separate baseline models. The combined vector is fed into a shallow neural network, which are the only trainable part in this architecture. FF fuses a feature vector provided from the image baseline model, with the raw register data parameters. Prior to the concatenation both vectors were normalized and the fused vector is then fed into a shallow trainable network. The final architecture, LFF, does not have completely frozen baseline models but instead learns two separate feature vectors. These feature vectors are then concatenated and fed into a shallow neural network to obtain a final prediction. The three fusion architectures were evaluated twice: using seven non-binary register data parameters, or only age and sex. When evaluated patient-wise, all three fusion architectures using the seven non-binary parameters obtain higher mean AUC and sensitivity than the single modality baseline models. All fusion architectures with only age and sex as register data parameters results in higher mean sensitivity than the baseline models. Overall, probability fusion with the seven non-binary parameters results in the highest mean AUC and sensitivity, and learned feature fusion with the seven non-binary parameters results in the highest mean accuracy.