Biomechanical Big Data is of great significance to precision health applications, among which we take special interest in Physical Activity Detection (PAD). In this study, we have performed extensive research on deep learning-based PAD from biomechanical big data, focusing on the challenges raised by the need of real-time edge inference. First, considering there are many places we can place the motion sensors, we have thoroughly compared and analyzed the location difference in terms of deep learning-based PAD performance. We have further compared the difference among six sensor channels (3-axis accelerometer and 3-axis gyroscope). Second, we have selected the optimal sensor and the optimal sensor channel, which can not only provide sensor usage suggestions but also enable ultra-low-power application on the edge. Third, we have investigated innovative methods to minimize the training effort of the deep learning model, leveraging the transfer learning strategy. More specifically, we propose to pre-train a transferable deep learning model using the data from other subjects and then fine-tune the model using limited data from the target-user. In such a way, we have found that, for single-channel case, the transfer learning can effectively increase the deep model performance even when the fine-tuning effort is very small. This research, demonstrated by comprehensive experimental evaluation, have shown the potential of ultra-low-power PAD with minimized sensor stream and minimized training effort.

This master’s thesis deals with the implementation of functional solution for classifying road users using mobile device with Android operating system. The goal is to create Android application which classifies vehicles in real time using rear-facing camera and saves timestamps of classification. Testing is performed mostly with own, diversely modificated dataset. Five models are trained and their performance is measured in dependence on hardware. The best classification performance is from pretrained MobileNet model where transfer learning with 6 classes of own dataset is used – 62,33 %. The results are summarized and a method for faster and more accurate traffic analysis is proposed.

This thesis explores multi-task learning for combustion flame characterization i.e to learn different characteristics of the combustion flame. We propose a multi-task convolutional neural network for two tasks i.e. PFR (Pilot fuel ratio) and fuel type classification based on the images of stable combustion. We utilize transfer learning and adopt VGG16 to develop a multi-task convolutional neural network to jointly learn the aforementioned tasks. We also compare the performance of the individual CNN model for two tasks with multi-task CNN which learns these two tasks jointly by sharing visual knowledge among the tasks. We share the effectiveness of our proposed approach to a private company’s dataset. To the best of our knowledge, this is the first work being done for jointly learning different characteristics of the combustion flame. / <p>This wrok as done with Siemens, and we have applied for a patent which is still pending.</p>

The Internet of Things is a constantly developing field. With advancements of algorithms for object detection and classification for images and videos, the possibilities of what can be made with small and cost efficient edge-devices are increasing. This work presents how camera traps and deep learning can be utilized for surveillance in remote environments, such as animal sanctuaries in the African Savannah. The camera traps connect to a smart surveillance network where images and sensor-data are analysed. The analysis can then be used to produce valuable information, such as the location of endangered animals or unauthorized humans, to park rangers working to protect the wildlife in these animal sanctuaries. Different motion detection algorithms are tested and evaluated based on related research within the subject. The work made in this thesis builds upon two previous theses made within Project Ngulia. The implemented surveillance system in this project consists of camera sensors, a database, a REST API, a classification service, a FTP-server and a web-dashboard for displaying sensor data and resulting images. A contribution of this work is an end-to-end smart surveillance system that can use different camera sources to produce valuable information to stakeholders. The camera software developed in this work is targeting the ESP32 based M5Stack Timer Camera and runs a motion detection algorithm based on Self-Organizing Maps. This improves the selection of data that is fed to the image classifier on the server. This thesis also contributes with an algorithm for doing iterative image classifications that handles the issues of objects taking up small parts of an image, making them harder to classify correctly.

The purpose of this master thesis was to investigate if using both X-ray images and patient's register data could increase the performance of a neural network in discrimination of two types of fractures in the thigh bone, called atypical femoral fractures (AFF) and normal femoral fractures (NFF). We also examined and evaluated how the fusion of the two data types could be done and how different types of fusion affect the performance. Finally, we evaluated how the number of variables in the register data affect a network's performance. Our image dataset consisted of 1,442 unique images from 580 patients (16.85% of the images were labelled AFF corresponding to 15.86% of the patients). Since the dataset is very imbalanced, sensitivity is a prioritized evaluation metric. The register data network was evaluated using five different versions of register data parameters: two (age and sex), seven (binary and non-binary) and 44 (binary and non-binary). Having only age and sex as input resulted in a classifier predicting all samples to class 0 (NFF), for all tested network architectures. Using a certain network structure (celled register data model 2), in combination with the seven non-binary parameters outperforms using both two and 44 (both binary and non-binary) parameters regarding mean AUC and sensitivity. Highest mean accuracy is obtained by using 44 non-binary parameters. The seven register data parameters have a known connection to AFF and includes age and sex. The network with X-ray images as input uses a transfer learning approach with a pre-trained ResNet50-base. This model performed better than all the register data models, regarding all considered evaluation metrics.        Three fusion architectures were implemented and evaluated: probability fusion (PF), feature fusion (FF) and learned feature fusion (LFF). PF concatenates the prediction provided from the two separate baseline models. The combined vector is fed into a shallow neural network, which are the only trainable part in this architecture. FF fuses a feature vector provided from the image baseline model, with the raw register data parameters. Prior to the concatenation both vectors were normalized and the fused vector is then fed into a shallow trainable network. The final architecture, LFF, does not have completely frozen baseline models but instead learns two separate feature vectors. These feature vectors are then concatenated and fed into a shallow neural network to obtain a final prediction. The three fusion architectures were evaluated twice: using seven non-binary register data parameters, or only age and sex. When evaluated patient-wise, all three fusion architectures using the seven non-binary parameters obtain higher mean AUC and sensitivity than the single modality baseline models. All fusion architectures with only age and sex as register data parameters results in higher mean sensitivity than the baseline models. Overall, probability fusion with the seven non-binary parameters results in the highest mean AUC and sensitivity, and learned feature fusion with the seven non-binary parameters results in the highest mean accuracy.

In this research the focus is on data-centric AI with a specific concentration on data quality evaluation and improvement for machine learning. We first present a practical framework for data quality evaluation and improvement, using a legal domain as a case study and build a corpus for legal argument mining. We first created an initial corpus with 4,937 instances that were manually labeled. We define five data quality evaluation dimensions: comprehensiveness, correctness, variety, class imbalance, and duplication, and conducted a quantitative evaluation on these dimensions for the legal dataset and two existing datasets in the medical domain for medical concept normalization. The first group of experiments showed that class imbalance and insufficient training data are the two major data quality issues that negatively impacted the quality of the system that was built on the legal corpus. The second group of experiments showed that the overlap between the test datasets and the training datasets, which we defined as "duplication," is the major data quality issue for the two medical corpora. We explore several widely used machine learning methods for data quality improvement. Compared to pseudo-labeling, co-training, and expectation-maximization (EM), generative adversarial network (GAN) is more effective for automated data augmentation, especially when a small portion of labeled data and a large amount of unlabeled data is available. The data validation process, the performance improvement strategy, and the machine learning framework for data evaluation and improvement discussed in this dissertation can be used by machine learning researchers and practitioners to build high-performance machine learning systems. All the materials including the data, code, and results will be released at: https://github.com/haihua0913/dissertation-dqei.

Cardiovascular diseases are a leading cause of death globally. Early diagnosis and treatment is of prime importance to prevent or mitigate health complications. Electrocardiogram (ECG) is a standard test modality used for early diagnosis of arrhythmias. The standard ECG uses 12 leads (i.e., 12 different views of the electrical activity of the heart). However, it is not always possible to perform a standard 12-lead ECG, for instance, in certain emergency situations. Such devices used in emergency situations are able to measure only a subset of leads. Although it is a simpler way of recording ECG, it comes at the cost of losing some information. The project presented in this thesis applies three different models based on canonical correlation analysis (CCA) to perform transfer learning from 12-lead ECGs to improve performance when only a subset of leads is available. The models used were linear canonical correlation analysis, deep canonical correlation analysis (DCCA) and deep canonically correlated bidirectional long short-term memory networks (DCC-BiLSTMs). These models are compared to each other using different configurations to study their performance on ECG data. Linear canonical correlation analysis performed better than its more complex variants, DCCA and DCC-BiLSTMs. With this method, it was possible to improve performance on ECG classification when using two, three, four and six leads in a computationally efficient way. / Hjärt- och kärlsjukdomar är den främsta dödsorsaken i världen. Tidig diagnos och behandling är av största vikt för att förhindra ytterligare och allvarliga hälsoproblem. Elektrokardiogram (EKG) är den standardmetod som används för tidig diagnos av arytmier. Standardförfarandet inom EKG använder sig av 12 avledningar (dvs. 12 olika vyer av hjärtats elektriska aktivitet). Det är dock inte alltid möjligt att utföra ett standard-EKG med 12 ledningar, vilket t.ex. förekommer i vissa nödsituationer. I dessa fall kan utrustning som gör det möjligt att ta fram ett 12-ledars EKG inte vara tillgänglig av flera olika skäl, och därför används andra apparater som kan mäta endast en delmängd av ledningarna för tidig diagnostik. Även om det är ett enklare sätt att utföra ett EKG, innebär det att man förlorar en del information. I det projekt som presenteras i detta dokument används tre olika modeller baserade på kanonisk korrelationsanalys (CCA) för att utföra överföringsinlärning från 12-ledars EKG för att förbättra prestanda när endast en delmängd av avledningar används. De modeller som användes var linjär kanonisk korrelationsanalys, djup kanonisk korrelationsanalys (DCCA) och djupa kanoniskt korrelerade bidirektionella långtidsminnesnätverk (DCCBiLSTMs). Dessa modeller jämförs med varandra med hjälp av olika konfigurationer för att studera deras prestanda på EKG-data. Linjär kanonisk korrelationsanalys presterade bättre än dess mer komplexa varianter, DCCA och DCC-BiLSTMs. Med denna metod var det möjligt att förbättra prestandan för klassificering av EKG när man använder två, tre, fyra och sex ledningar på ett beräkningseffektivt sätt.

Deep learning works best with vast andd well-distributed data collections. However, collecting and annotating large data sets can be very time-consuming and expensive. Moreover, deep learning is specific to domain knowledge, even with data and computation. E.g., models trained to classify animals would probably underperform when they classify vehicles. Although techniques such as domain adaptation and transfer learning have been popularised recently, tasks in cross-domain knowledge transfer have also taken off. However, most of these works are limited to computer vision. In the domain of time series, this is relatively underexplored. This thesis explores methods to use time series data from one domain to classify data generated from another domain via transfer learning. It focuses on using accelerometer data from running recordings to improve the classification performance on jumping data based on the apparent similarity of individual recordings. Thus, transfer learning and domain adaptation techniques were used to use the learning acquired through deep model training on running sequences. This thesis has performed four experiments to test this domain similarity. The first one consists of transforming time series with the continuous wavelet transform to get both time and frequency information. The model is then pre-trained within a contrastive learning framework. However, the continuous wavelet transformation (CWT) did not improve the classification results. The following two experiments consisted of pre-training the models with self-supervised learning. The first one with a contrastive pretext-task improved the classification results, and the resilience to data decrease. The second one with a forward forecasting pretext-task improved the results when all the data was available but was very sensitive to data decrease. Finally, the domain adaptation was tested and showed interesting performances on the classification task. Although some of the employed techniques did not show improvement, pre-training using contrastive learning on the running dataset has shown great improvement to classify the jumping dataset. / Djupinlärning fungerar bäst med stora och väl distribuerade datasamlingar. Det kan dock vara mycket tidskrävande och dyrt att samla in och kommentera stora datamängder. Även med alla data och beräkningar är djupinlärning specifik för domänkunskap. Exempelvis skulle modeller som tränats för att klassificera djur förmodligen underprestera när de klassificerar fordon. Även om tekniker som domänanpassning och överföringsinlärning har populariserats på senare tid, har även uppgifter inom kunskapsöverföring mellan olika domäner tagit fart. De flesta av dessa arbeten är dock begränsade till datorseende. Inom tidsseriernas område är detta relativt outforskat. I den här avhandlingen undersöks metoder för att använda tidsseriedata från en domän för att klassificera data från en annan domän med hjälp av djupinlärning. Fokus ligger på att använda accelerometerdata från löpning för att förbättra klassificeringen av hoppdata, baserat på den uppenbara likheten mellan löpning och hoppning. Således användes tekniker för överföringsinlärning och domänanpassning för att använda den inlärning som förvärvats genom träning av djupa modeller på löpsekvenser. I den här avhandlingen har fyra experiment utförts för att testa denna domänlikhet. Det första består av att omvandla tidsserier med den kontinuerliga wavelettransformen för att få fram både tids- och frekvensinformation. Modellen förtränas sedan inom en ram för kontrastiv inlärning. Användningen av CWT förbättrade dock inte klassificeringsresultaten. De följande två experimenten bestod av att förträna modellerna med självövervakad inlärning. Det första försöket med en kontrasterande förtextuppgift förbättrade klassificeringsresultaten och motståndskraften mot dataförlust. Det andra försöket med en prognostiserande förtextuppgift förbättrade resultaten när alla data var tillgängliga, men var mycket känslig för dataförlust. Slutligen testades domänanpassningen och visade intressanta resultat i klassificeringsuppgiften. Även om några av de använda teknikerna inte visade någon förbättring, har förträning med hjälp av kontrastinlärning på löpande dataset visat sig ge stora förbättringar när det gäller klassificering av hoppdata.

Molecular property prediction has the ability to improve many processes in molecular chemistry industry. One important application is the development of new drugs where molecular property prediction can decrease both the cost and time of finding new drugs. The current trend is to use graph neural networks or transformers which tend to need moderate and large amounts of data respectively to perform well. Because of the scarceness of molecular property data it is of great interest to find an effective method to transfer learning from other more data-abundant problems. In this thesis I present an approach to pre-train transformer encoders on reaction prediction in order to improve performance on downstream molecular property prediction tasks. I have built a model based on the full transformer architecture but modify it for the purpose of pre-training the encoder. Model performance and specifically the effect of pre-training is tested by predicting lipophilicity, HIV inhibition and hERG channel blocking using both pre-trained models and models without any pre-training. The results demonstrate a tendency for improvement of performance on all molecular property prediction tasks using the suggested pre-training but this tendency for improvement is not statistically significant. The major limitation with the conclusive evaluation stems from the limited simulations due to computational constraints

This thesis investigates the performance of Mask R-CNN when utilizing transfer learning on a small dataset. The aim was to instance segment eyeglass lenses as accurately as possible from self-portrait images. Five different models were trained, where the key difference was the types of eyeglasses the models were trained on. The eyeglasses were grouped into three types, fully rimmed, semi-rimless, and rimless glasses. 1550 images were used for training, validation, and testing. The model's performances were evaluated using TensorBoard training data and mean Intersection over Union scores (mIoU). No major differences in performance were found in four of the models, which grouped all three types of glasses into one class. Their mIoU scores range from 0.913 to 0.94 whereas the model with one class for each group of glasses, performed worse, with a mIoU of 0.85. The thesis revealed that one can achieve great instance segmentation results using a limited dataset when taking advantage of transfer learning. / Denna uppsats undersöker prestandan för Mask R-CNN vid användning av överföringsinlärning på en liten datamängd. Syftet med arbetet var att segmentera glasögonlinser så exakt som möjligt från självporträttbilder. Fem olika modeller tränades, där den viktigaste skillnaden var de typer av glasögon som modellerna tränades på. Glasögonen delades in i 3 typer, helbåge, halvbåge och båglösa. Totalt samlades 1550 träningsbilder in, dessa annoterades och användes för att träna modellerna.  Modellens prestanda utvärderades med TensorBoard träningsdata samt genomsnittlig Intersection over Union (IoU). Inga större skillnader i prestanda hittades mellan modellerna som endast tränades på en klass av glasögon. Deras genomsnittliga IoU varierar mellan 0,913 och 0,94. Modellen där varje glasögonkategori representerades som en unik klass, presterade sämre med en genomsnittlig IoU på 0,85. Resultatet av uppsatsen påvisar att goda instanssegmenteringsresultat går att uppnå med hjälp av en begränsad datamängd om överföringsinlärning används.