With the recent advantages of machine learning in cheminformatics, the drug discovery process has been accelerated; providing a high impact in the field of medicine and public health. Molecular property and activity prediction are key elements in the early stages of drug discovery by helping prioritize the experiments and reduce the experimental work. In this thesis, a novel approach for multi-task regression using a text-based Transformer model is introduced and thoroughly explored for training on a number of properties or activities simultaneously. This multi-task regression with Transformer based model is inspired by the field of Natural Language Processing (NLP) which uses prefix tokens to distinguish between each task. In order to investigate our architecture two data categories are used; 133 biological activities from ExCAPE database and three physical chemistry properties from MoleculeNet benchmark datasets. The Transformer model consists of the embedding layer with positional encoding, a number of encoder layers, and a Feedforward Neural Network (FNN) to turn it into a regression problem. The molecules are represented as a string of characters using the Simplified Molecular-Input Line-Entry System (SMILES) which is a ’chemistry language’ with its own syntax. In addition, the effect of Transfer Learning is explored by experimenting with two pretrained Transformer models, pretrained on 1.5 million and on 100 million molecules. The text-base Transformer models are compared with a feature-based Support Vector Regression (SVR) with the Tanimoto kernel where the input molecules are encoded as Extended Connectivity Fingerprint (ECFP), which are calculated features. The results have shown that Transfer Learning is crucial for improving the performance on both property and activity predictions. On bioactivity tasks, the larger pretrained Transformer on 100 million molecules achieved comparable performance to the feature-based SVR model; however, overall SVR performed better on the majority of the bioactivity tasks. On the other hand, on physicochemistry property tasks, the larger pretrained Transformer outperformed SVR on all three tasks. Concluding, the multi-task regression architecture with the prefix token had comparable performance with the traditional feature-based approach on predicting different molecular properties or activities. Lastly, using the larger pretrained models trained on a wide chemical space can play a key role in improving the performance of Transformer models on these tasks.

Transformer-based architectures have in recent years advanced state-of-the-art performance in Natural Language Processing. Researchers have successfully adapted such models to downstream tasks within NLP in a domain-specific setting. This thesis examines the application of these models to the legal domain by doing Named Entity Recognition (NER) in a setting of scarce training data. Three different pre-trained BERT models are fine-tuned on a set of 101 court case documents, whereof one model is pre-trained on legal corpora and the other two on general corpora. Experiments are run to evaluate the models’ predictive performance given smaller or larger quantities of data to fine-tune on. Results show that BERT models work reasonably well for NER with legal data. Unlike many other domain-specific BERT models, the BERT model trained on legal corpora does not outperform the base models. Modest amounts of annotated data seem sufficient for reasonably good performance.

I detta arbete genomförs utvecklingen av ett realtids objektdetekteringssystem för iOS. För detta ändamål används YOLO, en ett-stegs objektdetekterare och ett s.k. ihoplänkat neuralt nätverk vilket åstadkommer betydligt bättre prestanda än övriga realtidsdetek- terare i termer av hastighet och precision. En dörrdetekterare baserad på YOLO tränas och implementeras i en systemutvecklingsprocess. Maskininlärningsprocessen sammanfat- tas och praxis för att undvika överträning eller “overfitting” samt för att öka precision och hastighet diskuteras och appliceras. Vidare genomförs en rad experiment vilka pekar på att dataaugmentation och inkludering av negativ data i ett dataset medför ökad precision. Hyperparameteroptimisering och kunskapsöverföring pekas även ut som medel för att öka en objektdetekringsmodells prestanda. Författaren lyckas öka modellens mAP, ett sätt att mäta precision för objektdetekterare, från 63.76% till 86.73% utifrån de erfarenheter som dras av experimenten. En modells tendens för överträning utforskas även med resultat som pekar på att träning med över 300 epoker rimligen orsakar en övertränad modell. / This paper features the creation of a real time object detection system for mobile iOS using YOLO, a state-of-the-art one stage object detector and convoluted neural network far surpassing other real time object detectors in speed and accuracy. In this process an object detecting model is trained to detect doors. The machine learning process is outlined and practices to combat overfitting and increasing accuracy and speed are discussed. A series of experiments are conducted, the results of which suggests that data augmentation, including negative data in a dataset, hyperparameter optimisation and transfer learning are viable techniques in improving the performance of an object detection model. The author is able to increase mAP, a measurement of accuracy for object detectors, from 63.76% to 86.73% based on the results of experiments. The tendency for overfitting is also explored and results suggest that training beyond 300 epochs is likely to produce an overfitted model.

No description available.

No description available.

With more aircraft in the air than ever before, there is a need for automating the surveillance of the airspace. It is widely known that aircraft with different intentions fly in different flight patterns. Support systems for finding different flight patterns are therefore needed. In this thesis, we investigate the possibility of detecting circular flight patterns using deep learning models. The basis for detection is ADS-B data which is continuously transmitted by aircraft containing information related to the aircraft status. Two deep learning models are constructed to solve the binary classification problem of detecting circular flight patterns. The first model is a Long Short-Term Memory (LSTM) model and utilizes techniques such as sliding window and bidirectional LSTM layers to solve the given task. The second model is a Convolutional Neural Network (CNN) and utilizes transfer learning. For the CNN model, the trajectory data is converted into image representations which are fed into a pre-trained model with a custom final dense layer. While ADS-B is openly available, finding specific flight patterns and producing a labeled data set of that pattern is hard and time-consuming. The data set is therefore expanded using other sources of data. Two additional sources of trajectory data are added to the data set; radar and simulated data. Training a model on data of a different distribution than the model is being evaluated on can be problematic and introduces a new source of error known as training-validation mismatch. One of the main goals of this thesis is to be able to quantify the size of this error to decide if using data from other sources is a viable option. The results show that the CNN model outperforms the LSTM model and achieves an accuracy of 98.2%. The results also show that there is a cost, in terms of accuracy, associated with not only training on ADS-B data. For the CNN model that cost was a 1-4% loss in accuracy depending on the training data used. The corresponding cost for the LSTM model was 2-10%.

Training a deep neural network from scratch can be very expensive in terms of resources.In addition, training a neural network on a new task is usually done by training themodel form scratch. Recently there are new approaches in machine learning which usesthe knowledge from a pre-trained deep neural network on a new task. The technique ofreusing the knowledge from previously trained deep neural networks is called Transferlearning. In this paper we are going to evaluate transfer learning capabilities of deep neuralnetwork architectures for image classification. This research attempts to implementtransfer learning with different datasets and models in order to investigate transfer learningin different situations.

Melanoma is a life threatening type of skin cancer with numerous fatal incidences all over the world. The 5-year survival rate is very high for cases that are diagnosed in early stage. So, early detection of melanoma is of vital importance. Except for several techniques that clinicians apply so as to improve the reliability of detecting melanoma, many automated algorithms and mobile applications have been developed for the same purpose.In this paper, deep learning model designed from scratch as well as the pretrained models Inception v3 and VGG-16 are used with the aim of developing a reliable tool that can be used for melanoma detection by clinicians and individual users. Dermatologists who use dermoscopes can take advantage of the algorithms trained on dermoscopical images and acquire a confirmation about their diagnosis. On the other hand, the models trained on clinical images can be used on mobile applications, since a cell phone camera takes images similar to them.The results using Inception v3 model for dermoscopical images achieved accuracy 91.4%, sensitivity 87.8% and specificity 92.3%. For clinical images, the VGG-16 model achieved accuracy 86.3%, sensitivity 84.5% and specificity 88.8%. The results are compared to those of clinicians, which shows that the algorithms can be used reliably for the detection of melanoma.

Lung cancer is the most commonly diagnosed cancer in the world and its finding is mainly incidental. New technologies and more specifically artificial intelligence has lately acquired big interest in the medical field as it can automate or bring new information to the medical staff. Many research have been done on the detection or classification of lung cancer. These works are done on local region of interest but only a few of them have been done looking at a full CT-scan. The aim of this thesis was to assess lung damages from CT images using new machine learning methods. First, single predictors had been learned by a 3D resnet architecture: cancer, emphysema, and opacities. Emphysema was learned by the network reaching an AUC of 0.79 whereas cancer and opacity predictions were not really better than chance AUC = 0.61 and AUC = 0.61. Secondly, a multi-task network was used to predict the factors altogether. A training with no prior knowledge and a transfer learning approach using self-supervision were compared. The transfer learning approach showed similar results in the multi-task approach for emphysema with AUC=0.78 vs 0.60 without pre-training and opacities with an AUC=0.61. Moreover using the pre-training approach enabled the network to reach the same performance as each of single factor predictor but with only one multi-task network which saves a lot of computational time. Finally a risk score can be derived from the training to use this information in a clinical context.

In recent years, the field of computer vision has seen remarkable advancements, particularly with the rise of deep learning techniques. Object detection, a challenging task in image analysis, has benefited from these developments. This thesis investigates the application of object detection models, specifically You Only Look Once (YOLO), in the context of food recognition and health assessment based on the Swedish plate model. The study aims to assess the effectiveness of YOLO models in predicting the healthiness of food compositions according to the guidelines provided by the Swedish plate model. The research utilizes a custom dataset comprising 3707 images with 42 different food classes. Various preprocessing- and augmentation techniques are applied to enhance dataset quality and model robustness. The performance of the three YOLO models (YOLOv7, YOLOv8, and YOLOv9) are evaluated using precision, recall, mean Average Precision (mAP), and F1 score metrics. Results indicate that YOLOv8 showed higher performance, making it the recommended choice for further implementation in dietary assessment and health promotion initiatives. The study contributes to the understanding of how deep learning models can be leveraged for food recognition and health assessment. Overall, this thesis underscores the potential of deep learning in advancing computational approaches to dietary assessment and promoting healthier eating habits.