Background: Periodontitis is a chronic immuno-inflammatory disease characterized by inflammatory destruction of tooth-supporting tissues. Its pathogenesis involves a dysregulated local host immune response that is ineffective in combating microbial challenges. An integrated investigation of genes involved in mediating immune response suppression in periodontitis, based on multiple studies, can reveal genes pivotal to periodontitis pathogenesis. Here, we aimed to apply a deep learning (DL)-based autoencoder (AE) for predicting immunosuppression genes involved in periodontitis by integrating multiples omics datasets. Methods: Two periodontitis-related GEO transcriptomic datasets (GSE16134 and GSE10334) and immunosuppression genes identified from DisGeNET and HisgAtlas were included. Immunosuppression genes related to periodontitis in GSE16134 were used as input to build an AE, to identify the top disease-representative immunosuppression gene features. Using K-means clustering and ANOVA, immune subtype labels were assigned to disease samples and a support vector machine (SVM) classifier was constructed. This classifier was applied to a validation set (Immunosuppression genes related to periodontitis in GSE10334) for predicting sample labels, evaluating the accuracy of the AE. In addition, differentially expressed genes (DEGs), signaling pathways, and transcription factors (TFs) involved in immunosuppression and periodontitis were determined with an array of bioinformatics analysis. Shared DEGs common to DEGs differentiating periodontitis from controls and those differentiating the immune subtypes were considered as the key immunosuppression genes in periodontitis. Results: We produced representative molecular features and identified two immune subtypes in periodontitis using an AE. Two subtypes were also predicted in the validation set with the SVM classifier. Three “master” immunosuppression genes, PECAM1, FCGR3A, and FOS were identified as candidates pivotal to immunosuppressive mechanisms in periodontitis. Six transcription factors, NFKB1, FOS, JUN, HIF1A, STAT5B, and STAT4, were identified as central to the TFs-DEGs interaction network. The two immune subtypes were distinct in terms of their regulating pathways. Conclusion: This study applied a DL-based AE for the first time to identify immune subtypes of periodontitis and pivotal immunosuppression genes that discriminated periodontitis from the healthy. Key signaling pathways and TF-target DEGs that putatively mediate immune suppression in periodontitis were identified. PECAM1, FCGR3A, and FOS emerged as high-value biomarkers and candidate therapeutic targets for periodontitis.

No description available.

Generative models are broadly used in many subfields of DL. DNNs have recently developed a core approach to solving data-centric problems in image classification, translation, etc. The latest developments in parameterizing these models using DNNs and stochastic optimization algorithms have allowed scalable modeling of complex, high-dimensional data, including speech, text, and image. This dissertation proposal presents our state-the-art probabilistic bases and DL algorithms for generative models, including VAEs, GANs, and RNN-based encoder-decoder. The proposal also discusses application areas that may benefit from deep generative models in both NLP and computer vision. In NLP, we proposed an Arabic poetry generation model with extended phonetic and semantic embeddings (Phonetic CNN_subword embeddings). Extensive quantitative experiments using BLEU scores and Hamming distance show notable enhancements over strong baselines. Additionally, a comprehensive human evaluation confirms that the poems generated by our model outperform the base models in criteria including meaning, coherence, fluency, and poeticness. We proposed a generative video model using a hybrid VAE-GAN model in computer vision. Besides, we integrate two attentional mechanisms with GAN to get the essential regions of interest in a video, focused on enhancing the visual implementation of the human motion in the generated output. We have considered quantitative and qualitative experiments, including comparisons with other state-of-the-arts for evaluation. Our results indicate that our model enhances performance compared with other models and performs favorably under different quantitive metrics PSNR, SSIM, LPIPS, and FVD.Recently, mimicking biologically inspired learning in generative models based on SNNs has been shown their effectiveness in different applications. SNNs are the third generation of neural networks, in which neurons communicate through binary signals known as spikes. Since SNNs are more energy-efficient than DNNs. Moreover, DNN models have been vulnerable to small adversarial perturbations that cause misclassification of legitimate images. This dissertation shows the proposed ``VAE-Sleep'' that combines ideas from VAE and the sleep mechanism leveraging the advantages of deep and spiking neural networks (DNN--SNN).On top of that, we present ``Defense–VAE–Sleep'' that extended work of ``VAE-Sleep'' model used to purge adversarial perturbations from contaminated images. We demonstrate the benefit of sleep in improving the generalization performance of the traditional VAE when the testing data differ in specific ways even by a small amount from the training data. We conduct extensive experiments, including comparisons with the state–of–the–art on different datasets.

This case study examined emitted sound and actuated piezoelectric current in a solderpaste jet printing machine to conclude whether quality degradation could be detected with an autoencoder machine learning model. An autoencoder was used to detect anomalies in non-realtime that were defined asa diameter drift with an averaging window from a target diameter. A sensor and datacollection system existed for the piezoelectric current, and a microphone was proposedas a new sensor to monitor the system. The sound was preprocessed with a Fast Fourier Transform to extract information of the existing frequencies. The results of the model, visualized through reconstruction error plots and an Area Under the Curve score, show that the autoencoder successfully detected conspicuous anomalies. The study indicated that anomalies can be detected prior to solder paste supply failure using the sound. When the temperature was varied or when the jetting head nozzle was clogged by residual solder paste, the sound model identified most anomalies although the current network showed better performance. / Denna fallstudie undersökte emitterat ljud och drivande piezoelektrisk ström i en jetprinter med lödpasta för att dra slutsatsen om kvalitetsbrister kunde detekteras med en autoencoder maskininlärningsmodell. En autoencoder användes för att detektera avvikelser definierade som diametertrend med ett glidande medelvärde från en bördiameter. Tidigare studier har visat att den piezoelektriska strömmen i liknande maskiner kan användas för att förutspå kvalitetsbrister. En mikrofon föreslogs som en ny sensor för att övervaka systemet. Ljudet förbehandlades genom en snabb fouriertransform och frekvensinnehållet användes som indata i modellen. Resultaten visualiserades genom rekonstruktionsfel och metoden Area Under the Curve. Modellen upptäckte framgångsrikt tydliga avvikelser. För vissa felfall visade ljudet som indata bättre prestanda än strömmen, och för andra visade strömmen bättre prestanda. Till exempel indikerade studien att avvikelser kan detekteras före lodpasta-försörjningsfel med ljudet. Under varierande temperatur och då munstycket var igentäppt av kvarvarande lödpasta identifierade nätverket med ljud som indata de flesta avvikelser även om nätverket med strömmen visade bättre prestanda.

With increased growth of the Internet of Things (IoT) and physical exposure of devices to adversaries, a class of physical attacks called side-channel analysis (SCA) has emerged which compromises the security of systems. While security claims of cryptographic algorithms are based on the complexity of classical cryptanalysis attacks, they exclude information leakage by implementations on hardware platforms. Recent standardization processes require assessment of hardware security against SCA. In this dissertation, we study SCA based on deep learning techniques (DL-SCA) as a universal analysis toolbox for assessing the leakage of secret information by hardware implementations. We demonstrate that DL-SCA techniques provide a trade-off between the amount of prior knowledge of a hardware implementation and the amount of measurements required to identify the secret key. A DL-SCA based on supervised learning requires a training set, including information about the details of the hardware implementation, for a successful attack. Supervised learning has been widely used in power analysis (PA) to recover the secret key with a limited size of measurements. We demonstrate a similar trend in fault injection analysis (FIA) by introducing fault intensity map analysis with a neural network key distinguisher (FIMA-NN). We use dynamic timing simulations on an ASIC implementation of AES to develop a statistical model for biased fault injection. We employ the model to train a convolutional neural network (CNN) key distinguisher that achieves a superior efficiency, nearly $10times$, compared to classical FIA techniques. When a priori knowledge of the details of hardware implementations is limited, we propose DL-SCA techniques based on unsupervised learning, called SCAUL, to extract the secret information from measurements without requiring a training set. We further demonstrate the application of reinforcement learning by introducing the SCARL attack, to estimate a proper model for the leakage of secret data in a self-supervised approach. We demonstrate the success of SCAUL and SCARL attacks using power measurements from FPGA implementations of the AES and Ascon authenticated ciphers, respectively, to recover entire 128-bit secret keys without using any prior knowledge or training data. / Doctor of Philosophy / With the growth of the Internet of Things (IoT) and mobile devices, cryptographic algorithms have become essential components of end-to-end cybersecurity. A cryptographic algorithm is a highly nonlinear mathematical function which often requires a secret key. Only the user who knows the secret key is able to interpret the output of the algorithm to find the encoded information. Standardized algorithms are usually secure against attacks in which in attacker attempts to find the secret key given a set of input data and the corresponding outputs of the algorithm. The security of algorithms is defined based on the complexity of known cryptanalysis attacks to recover the secret key. However, a device executing a cryptographic algorithm leaks information about the secret key. Several studies have shown that the behavior of a device, such as power consumption, electromagnetic radiation and the response to external stimulation provide additional information to an attacker that can be exploited to find the secret key with much less effort than cryptanalysis attacks. Hence, exposure of devices to adversaries has enabled the class of physical attacks called side-channel analysis (SCA). In SCA, an attacker attempts to find the secret key by observing the behavior of the device executing the algorithm. Recent government and industry standardization processes, which choose future cryptographic algorithms, require assessing the security of hardware implementations against SCA in addition to the algorithmic level security of the cryptographic systems. The difficulty of an SCA attack depends on the details of a hardware implementation and the form of information leakage on a particular device. The diversity of possible hardware implementations and platforms, including application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) and microprocessors, has hindered the development of a unified measure of complexity in SCA attacks. In this research, we study SCA with deep learning techniques (DL-SCA) as a universal methodology to evaluate the leakage of secret information by hardware platforms. We demonstrate that DL-SCA based on supervised learning can be considered as a generalization of classical SCA techniques, and is able to find the secret information with a limited size of measurements. However, supervised learning techniques require a training set of data that includes information about the details of hardware implementation. We propose unsupervised learning techniques that are able to find the secret key even without knowledge of the details of the hardware. We further demonstrate the ability of reinforcement learning in estimating a proper model for data leakage in a self-supervised approach. We demonstrate that DL-SCA techniques are able to find the secret information even if the timing of data leakage in measurements are random. Hence, traditional countermeasures are unable to protect a hardware implementation against DL-SCA attacks. We propose a unified countermeasure to protect the hardware implementations against a wide range of SCA attacks.

Identification of travelers' transportation modes is a fundamental step for various problems that arise in the domain of transportation such as travel demand analysis, transport planning, and traffic management. This thesis aims to identify travelers' transportation modes purely based on their GPS trajectories. First, a segmentation process is developed to partition a user's trip into GPS segments with only one transportation mode. A majority of studies have proposed mode inference models based on hand-crafted features, which might be vulnerable to traffic and environmental conditions. Furthermore, the classification task in almost all models have been performed in a supervised fashion while a large amount of unlabeled GPS trajectories has remained unused. Accordingly, a deep SEmi-Supervised Convolutional Autoencoder (SECA) architecture is proposed to not only automatically extract relevant features from GPS segments but also exploit useful information in unlabeled data. The SECA integrates a convolutional-deconvolutional autoencoder and a convolutional neural network into a unified framework to concurrently perform supervised and unsupervised learning. The two components are simultaneously trained using both labeled and unlabeled GPS segments, which have already been converted into an efficient representation for the convolutional operation. An optimum schedule for varying the balancing parameters between reconstruction and classification errors are also implemented. The performance of the proposed SECA model, trip segmentation, the method for converting a raw trajectory into a new representation, the hyperparameter schedule, and the model configuration are evaluated by comparing to several baselines and alternatives for various amounts of labeled and unlabeled data. The experimental results demonstrate the superiority of the proposed model over the state-of-the-art semi-supervised and supervised methods with respect to metrics such as accuracy and F-measure. / Master of Science / Identifying users' transportation modes (e.g., bike, bus, train, and car) is a key step towards many transportation related problems including (but not limited to) transport planning, transit demand analysis, auto ownership, and transportation emissions analysis. Traditionally, the information for analyzing travelers' behavior for choosing transport mode(s) was obtained through travel surveys. High cost, low-response rate, time-consuming manual data collection, and misreporting are the main demerits of the survey-based approaches. With the rapid growth of ubiquitous GPS-enabled devices (e.g., smartphones), a constant stream of users' trajectory data can be recorded. A user's GPS trajectory is a sequence of GPS points, recorded by means of a GPS-enabled device, in which a GPS point contains the information of the device geographic location at a particular moment. In this research, users' GPS trajectories, rather than traditional resources, are harnessed to predict their transportation mode by means of statistical models. With respect to the statistical models, a wide range of studies have developed travel mode detection models using on hand-designed attributes and classical learning techniques. Nonetheless, hand-crafted features cause some main shortcomings including vulnerability to traffic uncertainties and biased engineering justification in generating effective features. A potential solution to address these issues is by leveraging deep learning frameworks that are capable of capturing abstract features from the raw input in an automated fashion. Thus, in this thesis, deep learning architectures are exploited in order to identify transport modes based on only raw GPS tracks. It is worth noting that a significant portion of trajectories in GPS data might not be annotated by a transport mode and the acquisition of labeled data is a more expensive and labor-intensive task in comparison with collecting unlabeled data. Thus, utilizing the unlabeled GPS trajectory (i.e., the GPS trajectories that have not been annotated by a transport mode) is a cost-effective approach for improving the prediction quality of the travel mode detection model. Therefore, the unlabeled GPS data are also leveraged by developing a novel deep-learning architecture that is capable of extracting information from both labeled and unlabeled data. The experimental results demonstrate the superiority of the proposed models over the state-of-the-art methods in literature with respect to several performance metrics.

The aim of this thesis is to derive an unsupervised method for detecting anomalies in time series. Autoencoder-based approaches are widely used for the task of detecting anomalies where a model learns to reconstruct the pattern of the given data. The main idea is that the model will be good at reconstructing data that does not contain anomalous behavior. If the model fails to reconstruct an observation it will be marked as anomalous. In this thesis, the derived method is applied to data from active medical devices manufactured by B. Braun. The given data consist of 6,000 length-varying time series, where the average length is greater than 14,000. Hence, the given sample size is small compared to their lengths. Subsequences of the same pattern where anomalies are expected to appear can be extracted from the time series taking expert knowledge about the data into account. Considering the subsequences for the model training, the problem can betranslated into a problem with a large dataset of short time series. It is shown that a common autoencoder is able to reconstruct anomalies well and is therefore not useful to solve the task. It is demonstrated that a variational autoencoder works better as there are large differences between the given anomalous observations and their reconstructions. Furthermore, several thresholds for these differences are compared. The relative number of detected anomalies in the two given datasets are 3.12% and 5.03%.

The following thesis proposes solutions to problems in two main areas of focus, computer vision and machine learning. Chapter 2 utilizes traditional computer vision methods implemented in a novel manner to successfully identify overlays contained in broadcast footage. The remaining chapters explore machine learning algorithms and apply them in various manners to big data, multi-channel image data, and ECG data. L1 and L2 principal component analysis (PCA) algorithms are implemented and tested against each other in Python, providing a metric for future implementations. Selected algorithms from this set are then applied in conjunction with other methods to solve three distinct problems. The first problem is that of big data error detection, where PCA is effectively paired with statistical signal processing methods to create a weighted controlled algorithm. Problem 2 is an implementation of image fusion built to detect and remove noise from multispectral satellite imagery, that performs at a high level. The final problem examines ECG medical data classification. PCA is integrated into a neural network solution that achieves a small performance degradation while requiring less then 20% of the full data size.

The main goal of this project, running at Neurofy AB, was that developing an AI recognition algorithm also known as, gas sensing algorithm or simply recognition algorithm, based on Artificial Intelligence (AI) technology, which would have the ability to detect or predict diary cattle diseases using odor signal data gathered, measured and provided by Gas Sensor Array (GSA) also known as, Electronic Nose or simply E-nose developed by the company. Two major challenges in this project were to first overcome the noises and errors in the odor signal data, as the E-nose is supposed to be used in an environment with difference conditions than laboratory, for instance, in a bail (A stall for milking cows) with varying humidity and temperatures, and second to find a proper feature extraction method appropriate for GSA. Normalization and Principal component analysis (PCA) are two classic methods which not only intended for re-scaling and reducing of features in a data-set at pre-processing phase of developing of odor identification algorithm, but also it thought that these methods reduce the affect of noises in odor signal data. Applying classic approaches, like PCA, for feature extraction and dimesionality reduction gave rise to loss of valuable data which made it difficult for classification of odors. A new method was developed to handle noises in the odors signal data and also deal with dimentionality reduction without loosing of valuable data, instead of the PCA method in feature extraction stage. This method, which is consisting of signal segmentation and Autoencoder with encoder-decoder, made it possible to overcome the noise issues in data-sets and it also is more appropriate feature extraction method due to better prediction accuracy performed by the AI gas recognition algorithm in comparison to PCA. For evaluating of Autoencoder monitoring of its learning rate of was performed. For classification and predicting of odors, several classifier, among alias, Logistic Regression (LR), Support vector machine (SVM), Linear Discriminant Analysis (LDA), Random forest Classifier (RFC) and MultiLayer perceptron (MLP), was investigated. The best prediction was obtained by classifiers MLP . To validate the prediction, obtained by the new AI recognition algorithm, several validation methods like Cross validation, Accuracy score, balanced accuracy score , precision score, Recall score, and Learning Curve, were performed. This new AI recognition algorithm has the ability to diagnose 3 different diary cattle diseases with an accuracy of 96% despite lack of samples. / Syftet med detta projekt var att utveckla en igenkänning algoritm baserad på maskinintelligens (Artificiell intelligens (AI) ), även känd som gasavkänning algoritm eller igenkänningsalgoritm, baserad på artificiell intelligens (AI) teknologi såsom maskininlärning ach djupinlärning, som skulle kunna upptäcka eller diagnosera vissa mjölkkor sjukdomar med hjälp av luktsignaldata som samlats in, mätts och tillhandahållits av Gas Sensor Array (GSA), även känd som elektronisk näsa eller helt enkelt E-näsa, utvecklad av företaget Neorofy AB. Två stora utmaningar i detta projekt bearbetades. Första utmaning var att övervinna eller minska effekten av brus i signaler samt fel (error) i dess data då E-näsan är tänkt att användas i en miljö där till skillnad från laboratorium förekommer brus, till example i ett stall avsett för mjölkkor, i form av varierande fukthalt och temperatur. Andra utmaning var att hitta rätt dimensionalitetsreduktion som är anpassad till GSA. Normalisering och Principal component analysis (PCA) är två klassiska metoder som används till att både konvertera olika stora datavärden i datamängd (data-set) till samma skala och dimensionalitetsminskning av datamängd (data-set), under förbehandling process av utvecling av luktidentifieringsalgoritms. Dessa metoder används även för minskning eller eliminering av brus i luktsignaldata (odor signal data). Tillämpning av klassiska dimensionalitetsminskning algoritmer, såsom PCA, orsakade förlust av värdefulla informationer som var viktiga för kllasifisering. Den nya metoden som har utvecklats för hantering av brus i luktsignaldata samt dimensionalitetsminskning, utan att förlora värdefull data, är signalsegmentering och Autoencoder. Detta tillvägagångssätt har gjort det möjligt att övervinna brusproblemen i datamängder samt det visade sig att denna metod är lämpligare metod för dimensionalitetsminskning jämfört med PCA. För utvärdering of Autoencoder övervakning of inlärningshastighet av Autoencoder tillämpades. För klassificering, flera klassificerare, bland annat, LogisticRegression (LR), Support vector machine (SVM) , Linear Discriminant Analysis (LDA), Random forest Classifier (RFC) och MultiLayer perceptron (MLP) undersöktes. Bästa resultate erhölls av klassificeraren MLP. Flera valideringsmetoder såsom, Cross-validering, Precision score, balanced accuracy score samt inlärningskurva tillämpades. Denna nya AI gas igenkänningsalgoritm har förmågan att diagnosera tre olika mjölkkor sjukdomar med en noggrannhet på högre än 96%.

Thèse numérisée par la Division de la gestion de documents et des archives de l'Université de Montréal.