The forestry and the sawmill industry have a lot of incoming and outgoing piles of wood. It's important to maintain quality and efficiency. This motivates an examination of whether machine learning- or more specifically, anomaly detection techniques can be implemented and used to detect faulty shipments. This thesis presents and evaluates some computer vision techniques and some deep learning techniques. Deep learning can be divided into groups; supervised, semi-supervised and unsupervised. In this thesis, all three groups were examined and it covers supervised methods such as Convolutional Neural Networks, semi-supervised methods such as a modified Convolutional Autoencoder (CAE) and lastly, an unsupervised technique such as Generative Adversarial Network (GAN) was being tested and evaluated.  A version of a GAN model proved to perform best for this thesis in terms of the accuracy of faulty detecting shipments with an accuracy rate of 68.2% and 79.8\% overall, which was satisfactory given the problems that were discovered during the progress of the thesis.

The thesis proposes an unsupervised representation learning method to predict 3D human pose from a 2D skeleton via a VAEGAN (Variational Autoencoder Generative Adversarial Network) hybrid network. The method learns to lift poses from 2D to 3D using selfsupervision and adversarial learning techniques. The method does not use images, heatmaps, 3D pose annotations, paired/unpaired 2Dto3D skeletons, 3D priors, synthetic 2D skeletons, multiview or temporal information in any shape or form. The 2D skeleton input is taken by a VAE that encodes it in a latent space and then decodes that latent representation to a 3D pose. The 3D pose is then reprojected to 2D for a constrained, selfsupervised optimization using the input 2D pose. Parallelly, the 3D pose is also randomly rotated and reprojected to 2D to generate a ’novel’ 2D view for unconstrained adversarial optimization using a discriminator network. The combination of the optimizations of the original and the novel 2D views of the predicted 3D pose results in a ’realistic’ 3D pose generation. The thesis shows that the encoding and decoding process of the VAE addresses the major challenge of erroneous and incomplete skeletons from 2D detection networks as inputs and that the variance of the VAE can be altered to get various plausible 3D poses for a given 2D input. Additionally, the latent representation could be used for crossmodal training and many downstream applications. The results on Human3.6M datasets outperform previous unsupervised approaches with less model complexity while addressing more hurdles in scaling the task to the real world. / Uppsatsen föreslår en oövervakad metod för representationslärande för att förutsäga en 3Dpose från ett 2D skelett med hjälp av ett VAE GAN (Variationellt Autoenkodande Generativt Adversariellt Nätverk) hybrid neuralt nätverk. Metoden lär sig att utvidga poser från 2D till 3D genom att använda självövervakning och adversariella inlärningstekniker. Metoden använder sig vare sig av bilder, värmekartor, 3D poseannotationer, parade/oparade 2D till 3D skelett, a priori information i 3D, syntetiska 2Dskelett, flera vyer, eller tidsinformation. 2Dskelettindata tas från ett VAE som kodar det i en latent rymd och sedan avkodar den latenta representationen till en 3Dpose. 3D posen är sedan återprojicerad till 2D för att genomgå begränsad, självövervakad optimering med hjälp av den tvådimensionella posen. Parallellt roteras dessutom 3Dposen slumpmässigt och återprojiceras till 2D för att generera en ny 2D vy för obegränsad adversariell optimering med hjälp av ett diskriminatornätverk. Kombinationen av optimeringarna av den ursprungliga och den nya 2Dvyn av den förutsagda 3Dposen resulterar i en realistisk 3Dposegenerering. Resultaten i uppsatsen visar att kodningsoch avkodningsprocessen av VAE adresserar utmaningen med felaktiga och ofullständiga skelett från 2D detekteringsnätverk som indata och att variansen av VAE kan modifieras för att få flera troliga 3D poser för givna 2D indata. Dessutom kan den latenta representationen användas för crossmodal träning och flera nedströmsapplikationer. Resultaten på datamängder från Human3.6M är bättre än tidigare oövervakade metoder med mindre modellkomplexitet samtidigt som de adresserar flera hinder för att skala upp uppgiften till verkliga tillämpningar.

Magnetic resonance imaging (MRI) is a commonly used non-invasive imaging technique that provides high resolution images of soft tissue. One problem with MRI is that it is sensitive to signal quality issues. The issues can arise for various reasons, for example by metal located either inside or outside of the body. Another common signal quality issue is caused by the patient being partly placed outside field of view of the MRI scanner.   This thesis aims to investigate the possibility to automatically detect these signal quality issues using deep neural networks. More specifically, two different 3D CNN network types were studied, a classification-based approach and a reconstruction-based approach. The datasets used consist of MRI volumes from UK Biobank which have been processed and manually annotated by operators at AMRA Medical. For the classification method four different network architectures were explored utilising supervised learning with multi-label classification. The classification method was evaluated using accuracy and label-based evaluation metrics, such as macro-precision, macro-recall and macro-F1. The reconstruction method was based on anomaly detection using an autoencoder which was trained to reconstruct volumes without any artefacts. A mean squared prediction error was calculated for the reconstructed volume and compared against a threshold in order to classify a volume with or without artefacts. The idea was that volumes containing artefacts should be more difficult to reconstruct and thus, result in a higher prediction error. The reconstruction method was evaluated using accuracy, precision, recall and F1-score.  The results show that the classification method has overall higher performance than the reconstruction method. The achieved accuracy for the classification method was 98.0% for metal artefacts and 97.5% for outside field of view artefacts. The best architecture for the classification method proved to be DenseNet201. The reconstruction method worked for metal artefacts with an achieved accuracy of 75.7%. Furthermore, it was concluded that reconstruction method did not work for detection of outside field of view artefacts.    The results from the classification method indicate that there is a possibility to automatically detect artefacts with deep neural networks. However, it is needed to further improve the method in order to completely replace a manual quality control step before using the volumes for calculation of biomarkers.

Anomaly detection in high-dimensional financial transaction data is challenging and resource-intensive, particularly when the dataset is unlabeled. Sometimes, one can alleviate the computational cost and improve the results by utilizing a pre-trained model, provided that the features learned from the pre-training are useful for learning the second task. Investigating this issue was the main purpose of this thesis. More specifically, it was to explore the potential gain of pre-training a detection model on one trader's transaction history and then retraining the model to detect anomalous trades in another trader's transaction history. In the context of transfer learning, the pre-trained and the retrained model are usually referred to as the source model and target model, respectively.  A deep LSTM autoencoder was proposed as the source model due to its advantages when dealing with sequential data, such as financial transaction data. Moreover, to test its anomaly detection ability despite the lack of labeled true anomalies, synthetic anomalies were generated and included in the test set. Various experiments confirmed that the source model learned to detect synthetic anomalies with highly distinctive features. Nevertheless, it is hard to draw any conclusions regarding its anomaly detection performance due to the lack of labeled true anomalies. While the same is true for the target model, it is still possible to achieve the thesis's primary goal by comparing a pre-trained model with an identical untrained model. All in all, the results suggest that transfer learning offers a significant advantage over traditional machine learning in this context.

Style transfer using Generative adversarial networks (GANs) has been successful in recent publications. One field in style transfer is music style transfer, in which a piece of music is transformed in some way, be it through genre-, harmonic-, rhythmic transfer, etc. In this thesis, we have performed genre style transfer using a CycleGAN architecture and symbolic representation of data. Previous work using the same architecture and representation has focused solely on transferring the arrangement of the notes (composition). We have improved this work by including the transfer of multiple instruments (timbre) to create more convincing results. Additional discriminators were added to the CycleGAN architecture to achieve this, and they are individually tasked with enforcing the timbre and composition of a song. Previous works have also used variable autoencoders (VAEs) with sequential data representation for style transfer. The use of VAEs for genre style transfer using symbolic data representation instead of sequential was explored, and recommendations for future work include omitting faults found during exploration. Two different classifiers were created to evaluate the results of the CycleGAN model. One uses symbolic representation, in which all instruments are merged into one, thus evaluating the composition of the generated songs. The other classifier uses a spectrogram representation which evaluates the transfer as a whole, both timbre and composition. The evaluation of the improved CycleGAN model using the classifiers showed that it could perform genre style transfer successfully even when adding timbre to the style transfer.

Increasingly flexible and efficient industrial processes and automation systems are developed by integrating computational systems and physical processes, thereby forming large heterogeneous systems of cyber-physical systems. Such systems depend on particular data models and payload formats for communication, and making different entities interoperable is a challenging problem that drives the engineering costs and time to deployment. Interoperability is typically established and maintained manually using domain knowledge and tools for processing and visualization of symbolic metadata, which limits the scalability of the present approach. The vision of next generation automation frameworks, like the Arrowhead Framework, is to provide autonomous interoperability solutions. In this thesis the problem to automatically establish interoperability between cyber-physical systems is reviewed and formulated as a mathematical optimisation problem, where symbolic metadata and message payloads are combined with machine learning methods to enable message translation and improve system of systems utility. An autoencoder based implementation of the model is investigated and simulation results for a heating and ventilation system are presented, where messages are partially translated correctly by semantic interpolation and generalisation of the latent representations. A maximum translation accuracy of 49% is obtained using this unsupervised learning approach. Further work is required to improve the translation accuracy, in particular by further exploiting metadata in the model architecture and autoencoder training protocol, and by considering more advanced regularization methods and utility optimization. / Productive 4.0

Chronic pain (CP) is a personal and economic burden that affects more than 30% of the world's population. While being the leading cause of disability, it is complicated to diagnose and manage. The optimal way to treat CP is to identify the pain mechanism or the underlying cause. The substantial overlap of the pain mechanisms (i.e., Nociceptive, Neuropathic, and Nociplastic) usually makes identification unreachable in a clinical setting where finding the dominant mechanism is complicated. Additionally, many specialists regard CP classification as a spectrum or continuum. Despite the importance, a data-driven way to identify co-existing CP mechanisms and quantification is still absent. This work successfully identified the co-existing CP mechanisms within a patient using Unsupervised Learning while quantifying them without the help of diagnosis established by the clinicians. Two different datasets from different cohorts comprised of patient-reported history and questionnaires were used in this work. Unsupervised Learning (k-prototypes) revealed notable overlaps in the data. It was further emphasized by the outcomes of the Semi-supervised Learning algorithms when the same trend was observed with some diagnosis or class information. It became evident that the CP mechanisms overlap and cannot be classified as distinct conditions. Additionally, mixed pain mechanisms do not make an individual cluster or class, and CP should be considered as a continuum. To reduce data dimension and extract hidden features, Autoencoder was used. Using an overlapping clustering technique, the pain mechanisms were identified. The pain mechanisms were also quantified while elucidating overlaps, and the dominant CP mechanism was successfully pointed out with explainable element. The hamming loss of 0.43 and average precision of 0.5 were achieved when considered as a multi-label classification problem. This work is a data-driven validation that there are significant overlaps in CP conditions, and CP should be considered a continuum where all CP mechanisms may co-exist. / Thesis / Master of Applied Science (MASc) / Chronic pain (CP) is a global burden and the primary cause for patients to seek medical attention. Despite continuous efforts in this area, CP remains clinically challenging to manage. The most effective method of treating CP is identifying the underlying cause or mechanism, which is often unattainable. This thesis attempted to identify the CP mechanisms existing in a patient while quantifying them from patient-reported history and questionnaire data. Unsupervised Learning was used to identify clinically meaningful clusters that revealed the three main CP mechanisms, i.e., Nociceptive, Neuropathic, and Nociplastic, achieving acceptable hamming loss (0.43) and average precision (0.5). The results exhibited that the CP mechanisms co-exist and CP should be regarded as a continuum rather than distinct entities. The algorithm successfully indicated the dominant CP mechanism, a goal for optimal CP management and treatment. The results were also validated by a comparative analysis with data from another cohort that demonstrated a similar trend.

Here, we show that training with multiple noncommunicable diseases (NCDs) is both feasible and beneficial to modeling this class of diseases. We first use data from the Cancer Genome Atlas (TCGA) to train a pan cancer model, and then characterize the information the model has learned about the cancers. In doing this we show that the model has learned concepts that are relevant to the task of cancer classification. We also test the model on datasets derived independently of the TCGA cohort and show that the model is robust to data outside of its training distribution such as precancerous legions and metastatic samples. We then utilize the cancer model as the basis of a transfer learning study where we retrain it on other, non-cancer NCDs. In doing so we show that NCDs with very differing underlying biology contain extractible information relevant to each other allowing for a broader model of NCDs to be developed with existing datasets. We then test the importance of the samples source tissue in the model and find that the NCD class and tissue source may not be independent in our model. To address this, we use the tissue encodings to create augmented samples. We test how successfully we can use these augmented samples to remove or diminish tissue source importance to NCD class through retraining the model. In doing this we make key observations about the nature of concept importance and its usefulness in future neural network explainability efforts.

Phase transitions are fundamental phenomena in (quantum) many-body systems. They are associated with changes in the macroscopic physical properties of the system in response to the alteration in the conditions controlled by one or more parameters, like temperature or coupling constants. Quantum phase transitions are particularly intriguing as they reveal new insights into the fundamental nature of matter and the laws of physics. The study of phase transitions in such systems is crucial in aiding our understanding of how materials behave in extreme conditions, which are difficult to replicate in laboratory, and also the behavior of exotic states of matter with unique and potentially useful properties like superconductors and superfluids. Moreover, this understanding has other practical applications and can lead to the development of new materials with specific properties or more efficient technologies, such as quantum computers. Hence, detecting the transition point from one phase of matter to another and constructing the corresponding phase diagram is of great importance for examining many-body systems and predicting their response to external perturbations. Traditionally, phase transitions have been identified either through analytical methods like mean field theory or numerical simulations. The pinpointing of the critical value normally involves the measure of specific quantities such as local observables, correlation functions, energy gaps, etc. reflecting the changes in the physics through the transition. However, the latter approach requires prior knowledge of the system to calculate the order parameter of the transition, which is uniquely associated to its universality class. Recently, another method has gained more and more attention in the physics community. By using raw and very general representative data of the system, one can resort to machine learning techniques to distinguish among patterns within the data belonging to different phases. The relevance of these techniques is rooted in the ability of a properly trained machine to efficiently process complex data for the sake of pursuing classification tasks, pattern recognition, generating brand new data and even developing decision processes. The aim of this thesis is to explore phase transitions from this new and promising data-centric perspective. On the one hand, our work is focused on the developement of new machine learning architectures using state-of-the-art and interpretable models. On the other hand, we are interested in the study of the various possible data which can be fed to the artificial intelligence model for the mapping of a quantum many-body system phase diagram. Our analysis is supported by numerical examples obtained via matrix-product-states (MPS) simulations for several one-dimensional zero-temperature systems on a lattice such as the XXZ model, the Extended Bose-Hubbard model (EBH) and the two-species Bose Hubbard model (BH2S). In Part I, we provide a general introduction to the background concepts for the understanding of the physics and the numerical methods used for the simulations and the analysis with deep learning. In Part II, we first present the models of the quantum many-body systems that we study. Then, we discuss the machine learning protocol to identify phase transitions, namely anomaly detection technique, that involves the training of a model on a dataset of normal behavior and use it to recognize deviations from this behavior on test data. The latter can be applied for our purpose by training in a known phase so that, at test-time, all the other phases of the system are marked as anomalies. Our method is based on Generative Adversarial Networks (GANs) and improves the networks adopted by the previous works in the literature for the anomaly detection scheme taking advantage of the adversarial training procedure. Specifically, we train the GAN on a dataset composed of bipartite entanglement spectra (ES) obtained from Tensor Network simulations for the three aforementioned quantum systems. We focus our study on the detection of the elusive Berezinskii-Kosterlitz-Thouless (BKT) transition that have been object of intense theoretical and experimental studies since its first prediction for the classical two-dimensional XY model. The absence of an explicit symmetry breaking and its gappless-to-gapped nature which characterize such a transition make the latter very subtle to be detected, hence providing a challenging testing ground for the machine-driven method. We train the GAN architecture on the ES data in the gapless side of BKT transition and we show that the GAN is able to automatically distinguish between data from the same phase and beyond the BKT. The protocol that we develop is not supposed to become a substitute to the traditional methods for the phase transitions detection but allows to obtain a qualitative map of a phase diagram with almost no prior knowledge about the nature and the arrangement of the phases -- in this sense we refer to it as agnostic -- in an automatic fashion. Furthermore, it is very general and it can be applied in principle to all kind of representative data of the system coming both from experiments and numerics, as long as they have different patterns (even hidden to the eye) in different phases. Since the kind of data is crucially linked with the success of the detection, together with the ES we investigate another candidate: the probability density function (PDF) of a globally U(1) conserved charge in an extensive sub-portion of the system. The full PDF is one of the possible reductions of the ES which is known to exhibit relations and degeneracies reflecting very peculiar aspects of the physics and the symmetries of the system. Its patterns are often used to tell different kinds of phases apart and embed information about non-local quantum correlations. However, the PDF is measurable, e.g. in quantum gas microscopes experiments, and it is quite general so that it can be considered not only in the cases of the study but also in other systems with different symmetries and dimensionalities. Both the ES and the PDF can be extracted from the simulation of the ground state by dividing the one-dimensional chain into two complementary subportions. For the EBH we calculate the PDF of the bosonic occupation number in a wide range of values of the couplings and we are able to reproduce the very rich phase diagram containing several phases (superfluid, Mott insulator, charge density wave, phase separation of supersolid and superfluid and the topological Haldane insulator) just with an educated gaussian fit of the PDF. Even without resorting to machine learning, this analysis is instrumental to show the importance of the experimentally accessible PDF for the task. Moreover, we highlight some of its properties according to the gapless and gapped nature of the ground state which require a further investigation and extension beyond zero-temperature regimes and one-dimensional systems. The last chapter of the results contains the description of another architecture, namely the Concrete Autoencoder (CAE) which can be used for detecting phase transitions with the anomaly detection scheme while being able to automatically learn what the most relevant components of the input data are. We show that the CAE can recognize the important eigenvalues out of the entire ES for the EBH model in order to characterize the gapless phase. Therefore the latter architecture can be used to provide not only a more compact version of the input data (dimensionality reduction) -- which can improve the training -- but also some meaningful insights in the spirit of machine learning interpretability. In conclusion, in this thesis we describe two advances in the solution to the problem of phase recognition in quantum many-body systems. On one side, we improve the literature standard anomaly detection protocol for an automatic and agnostic identification of the phases by employing the GAN network. Moreover, we implement and test an explainable model which can make the interpretation of the results easier. On the other side we put the focus on the PDF as a new candidate quantity for the scope of discerning phases of matter. We show that it contains a lot of information about the many-body state being very general and experimentally accessible.

Le processus du don d’organes, crucial pour la survie de nombreux patients, ne répond pas à la demande croissante. Il dépend d’une identification, par les cliniciens, des potentiels donneurs d’organes. Cette étape est imparfaite et manque entre 30% et 60% des potentiels donneurs d’organes et ce indépendamment des pays étudiés. Améliorer ce processus est un impératif à la fois moral et économique. L’objectif de ce mémoire était de développer et valider un modèle afin de détecter automatiquement les potentiels donneurs d’organes. Pour ce faire, les données cliniques de l’ensemble des patients adultes hospitalisés aux soins intensifs du CHUM entre 2012 et 2019 ont été utilisées. 103 valeurs de laboratoires temporelles différentes et 2 valeurs statiques ont été utilisées pour développer un modèle de réseaux de neurones convolutifs entrainé à prédire les potentiels donneurs d’organes. Ce modèle a été comparé à un modèle fréquentiste linéaire non temporel. Le modèle a par la suite été validé dans une population externe cliniquement distincte. Différentes stratégies ont été comparées pour peaufiner le modèle dans cette population externe et améliorer les performances. Un total de 19 463 patients, dont 397 donneurs potentiels, ont été utilisés pour développer le modèle et 4 669, dont 36 donneurs potentiels, ont été utilisés pour la validation externe. Le modèle démontrait une aire sous la courbe ROC (AUROC) de 0.966 (IC95% 0.9490.981), supérieure au modèle fréquentiste linéaire (AUROC de 0.940 IC95% 0.908-0.969, p=0.014). Le modèle était aussi supérieur dans certaines sous populations d’intérêt clinique. Dans le groupe de validation externe, l’AUROC du modèle de réseaux de neurones était de 0.820 (0.682-0.948) augmentant à 0.874 (0.731-0.974) à l’aide d’un ré-entrainement. Ce modèle prometteur a le potentiel de modifier et d’améliorer la détection des potentiels donneurs d’organes. D’autres étapes de validation prospectives et d’amélioration du modèle, notamment l’ajout de données spécifiques, sont nécessaires avant une utilisation clinique de routine. / The organ donation process, however crucial for many patients’ survival, is not enough to address the increasing demand. Its efficiency depends on potential organ donors’ identification by clinicians. This imperfect step misses between 30%–60% of potential organ donor. Improving that process is a moral and economic imperative. The main goal of this work was to address that liming step by developing and validating a predictive model that could automatically detect potential organ donors. The clinical data from all patients hospitalized, between 2012 and 2019 to the CHUM critical care units were extracted. The temporal evolution of 103 types of laboratory analysis and 2 static clinical data was used to develop and test a convolutive neural network (CNN), trained to predict potential organ donors. This model was compared to a non-temporal logistical model as a baseline. The CNN model was validated in a clinically distinct external population. To improve the performance in this external cohort, strategies to fine-tune the network were compared. 19 463 patients, including 397 potential organ donors, were used to create the model and 4 669 patients, including 36 potential organ donors, served as the external validation cohort. The CNN model performed better with an AUROC of 0.966 (IC95% 0.949-0.981), compared to the logistical model (AUROC de 0.940 IC95% 0.908-0.969, p=0.014). The CNN model was also superior in specific subpopulation of increased clinical interest. In the external validation cohort, the CNN model’s AUROC was 0.820 (0.682-0.948) and could be improved to 0.874 (0.731-0.974) after fine tuning. This promising model could change potential organ donors' detection for the better. More studies are however required to improve the model, by adding more types of data, and to validate prospectively the mode before routine clinical usage.