This thesis presents training of an end-to-end autoencoder model using the transformer, with an encoder that can encode sentences into fixed-length latent vectors and a decoder that can reconstruct the sentences using image representations. Encoding and decoding sentences to and from these image representations are central to the model design. This method allows new sentences to be generated by traversing the Euclidean space, which makes vector arithmetic possible using sentences. Machines excel in dealing with concrete numbers and calculations, but do not possess an innate infrastructure designed to help them understand abstract concepts like natural language. In order for a machine to process language, scaffolding must be provided wherein the abstract concept becomes concrete. The main objective of this research is to provide such scaffolding so that machines can process human language in an intuitive manner.

Gene-expression profiling enables researchers to quantify transcription levels in cells, thus providing insight into functional mechanisms of diseases and other biological processes. However, because of the high dimensionality of these data and the sensitivity of measuring equipment, expression data often contains unwanted confounding effects that can skew analysis. For example, collecting data in multiple runs causes nontrivial differences in the data (known as batch effects), known covariates that are not of interest to the study may have strong effects, and there may be large systemic effects when integrating multiple expression datasets. Additionally, many of these confounding effects represent higher-order interactions that may not be removable using existing techniques that identify linear patterns. We created Confounded to remove these effects from expression data. Confounded is an adversarial variational autoencoder that removes confounding effects while minimizing the amount of change to the input data. We tested the model on artificially constructed data and commonly used gene expression datasets and compared against other common batch adjustment algorithms. We also applied the model to remove cancer-type-specific signal from a pan-cancer expression dataset. Our software is publicly available at https://github.com/jdayton3/Confounded.

Magnetic resonance imaging (MRI) is a critical diagnostic tool in medical practice, enabling non-invasive visualization of anatomy and physiological processes. Nonetheless, MRI has inherent spatial resolution limitations, which may limit its diagnostic capabilities. Recently, a new technology employing Radio-frequency Amplification by Stimulated emission of Radiation (RASER) has emerged to improve MRI resolution. Similar to a laser, RASER-MRI signals spontaneously emerge without the need for a radio frequency pulse(RF), which additionally enhances the safety of the process. However, RASER-MRI images frequently exhibit a significant presence of image artifacts due to the nonlinear behavior between image slices. This master’s thesis aims to determine whether image artifacts can be eliminated using deep artificial neural networks. The neural networks were trained on purely synthetic data, due to the complexity of real RASER experiments. The implementation was split into three phases. The first phase focused on the reconstruction of 1D RASER profiles. The test done during this phase showed that the reconstruction was preferably made with a Convolutional Neural Network (CNN). The CNN does not require knowledge of the total population inversion, and the ideal input was the most volatile RASER spectrum. The second phase was dedicated to reconstructing simulated RASER-MRI images. This phase started with the creation of a random RASER-MRI image generator which was used to generate the training and testing data. The reconstruction was successful and was further enhanced with an image-to-image Unet. The entire deep learning pipeline did not suffice for real data, which sparked the third phase. The third phase focused on simulating more realistic RASER data. The new data improved the result, however, the reconstruction did not suffice. Further research needs to be done into ways to make the simulation more realistic to improve the reconstruction of the real RASER-MRI image. However, this project concludes that simulated RASER-spectra can be reconstructed using deep learning.

The following thesis proposes solutions to problems in two main areas of focus, computer vision and machine learning. Chapter 2 utilizes traditional computer vision methods implemented in a novel manner to successfully identify overlays contained in broadcast footage. The remaining chapters explore machine learning algorithms and apply them in various manners to big data, multi-channel image data, and ECG data. L1 and L2 principal component analysis (PCA) algorithms are implemented and tested against each other in Python, providing a metric for future implementations. Selected algorithms from this set are then applied in conjunction with other methods to solve three distinct problems. The first problem is that of big data error detection, where PCA is effectively paired with statistical signal processing methods to create a weighted controlled algorithm. Problem 2 is an implementation of image fusion built to detect and remove noise from multispectral satellite imagery, that performs at a high level. The final problem examines ECG medical data classification. PCA is integrated into a neural network solution that achieves a small performance degradation while requiring less then 20% of the full data size.

Treating cancer with radiotherapy requires precise planning. Several planning pipelines rely on reference dose mimicking, where one tries to find machine parameters best mimicking a given reference dose. Dose mimicking relies on having a function that quantifies dose similarity well, necessitating methods for feature extraction of dose images. In this thesis we investigate ways of extracting features from clinical doseimages, and propose a few proof-of-concept dose mimicking functions using the extracted features. We extend current techniques and lay the foundation for new techniques for feature extraction, using mathematical frameworks developed in entirely different areas. In particular we give an introduction to wavelet theory, which provides signal decomposition techniques suitable for analysing local structure, and propose two different dose mimicking functions using wavelets. Furthermore, we extend ROI-based mimicking functions to use artificial ROIs, and we investigate variational autoencoders and their application to the clinical dose feature extraction problem. We conclude that the proposed functions have the potential to address certain shortcomings of current dose mimicking functions. The four methods all seem to approximately capture some notion of dose similarity. Used in combination with the current framework they have the potential of improving dose mimickingresults. However, the numerical tests supporting this are brief, and more thorough numerical investigations are necessary to properly evaluate the usefulness of the new dose mimicking functions. / Behandling av cancer med strålterapi kräver precis planering. Flera olika planeringsramverk bygger på doshärmning, som innebär att hitta de maskinparametrar som bäst härmar en given referensdos. För doshärmning behövs en funktion som kvantifierar likheten mellan två doser, vilket kräver ett sätt att extrahera utmärkande egenskaper – så kallade features – från dosbilder. I det här examensarbetet undersöker vi olika matematiska metoder för att extrahera features från kliniska dosbilder, och presenterar några olika förslag på prototyper till doshärmningsfunktioner, konstruerade utifrån extraherade features. Vi utvidgar nuvarande tekniker och lägger grunden för nya tekniker genom att använda matematiska ramverk utvecklade för helt andra syften. Speciellt så ger vi en introduktion till wavelet-teori, som ger matematiska verktyg för att analysera lokala beteenden hos signaler, exempelvis bilder. Vi föreslår två olika doshärmningsfunktioner som utnyttjar wavelets, och utvidgar ROI-baseraddoshärmning genom att introducera artificiella ROIar. Vidare så undersökervi så kallade variational autoencoders  och möjligheten att använda dessa för extrahering av features från dosbilder. Vi kommer fram till att de föreslagna funktionerna har potential att åtgärda vissa begränsningar som finns hos de doshärmningsfunktioner som används idag. De fyra metoderna verkar alla approximativt kvantifiera begreppet doslikhet. Användning av dessa nya metoder i kombination med nuvarande ramverk för doshärmning har potential att förbättra resultaten från doshärmning. De numeriska undersökningar som underbygger dessa slutsatser är dock inte särskilt ingående, så mer noggranna numeriska tester krävs för att kunna ge några definitiva svar angående de presenterade doshärmningsfunktionernas användbarhet ipraktiken.

Background: Periodontitis is a chronic immuno-inflammatory disease characterized by inflammatory destruction of tooth-supporting tissues. Its pathogenesis involves a dysregulated local host immune response that is ineffective in combating microbial challenges. An integrated investigation of genes involved in mediating immune response suppression in periodontitis, based on multiple studies, can reveal genes pivotal to periodontitis pathogenesis. Here, we aimed to apply a deep learning (DL)-based autoencoder (AE) for predicting immunosuppression genes involved in periodontitis by integrating multiples omics datasets. Methods: Two periodontitis-related GEO transcriptomic datasets (GSE16134 and GSE10334) and immunosuppression genes identified from DisGeNET and HisgAtlas were included. Immunosuppression genes related to periodontitis in GSE16134 were used as input to build an AE, to identify the top disease-representative immunosuppression gene features. Using K-means clustering and ANOVA, immune subtype labels were assigned to disease samples and a support vector machine (SVM) classifier was constructed. This classifier was applied to a validation set (Immunosuppression genes related to periodontitis in GSE10334) for predicting sample labels, evaluating the accuracy of the AE. In addition, differentially expressed genes (DEGs), signaling pathways, and transcription factors (TFs) involved in immunosuppression and periodontitis were determined with an array of bioinformatics analysis. Shared DEGs common to DEGs differentiating periodontitis from controls and those differentiating the immune subtypes were considered as the key immunosuppression genes in periodontitis. Results: We produced representative molecular features and identified two immune subtypes in periodontitis using an AE. Two subtypes were also predicted in the validation set with the SVM classifier. Three “master” immunosuppression genes, PECAM1, FCGR3A, and FOS were identified as candidates pivotal to immunosuppressive mechanisms in periodontitis. Six transcription factors, NFKB1, FOS, JUN, HIF1A, STAT5B, and STAT4, were identified as central to the TFs-DEGs interaction network. The two immune subtypes were distinct in terms of their regulating pathways. Conclusion: This study applied a DL-based AE for the first time to identify immune subtypes of periodontitis and pivotal immunosuppression genes that discriminated periodontitis from the healthy. Key signaling pathways and TF-target DEGs that putatively mediate immune suppression in periodontitis were identified. PECAM1, FCGR3A, and FOS emerged as high-value biomarkers and candidate therapeutic targets for periodontitis.

No description available.

Generative models are broadly used in many subfields of DL. DNNs have recently developed a core approach to solving data-centric problems in image classification, translation, etc. The latest developments in parameterizing these models using DNNs and stochastic optimization algorithms have allowed scalable modeling of complex, high-dimensional data, including speech, text, and image. This dissertation proposal presents our state-the-art probabilistic bases and DL algorithms for generative models, including VAEs, GANs, and RNN-based encoder-decoder. The proposal also discusses application areas that may benefit from deep generative models in both NLP and computer vision. In NLP, we proposed an Arabic poetry generation model with extended phonetic and semantic embeddings (Phonetic CNN_subword embeddings). Extensive quantitative experiments using BLEU scores and Hamming distance show notable enhancements over strong baselines. Additionally, a comprehensive human evaluation confirms that the poems generated by our model outperform the base models in criteria including meaning, coherence, fluency, and poeticness. We proposed a generative video model using a hybrid VAE-GAN model in computer vision. Besides, we integrate two attentional mechanisms with GAN to get the essential regions of interest in a video, focused on enhancing the visual implementation of the human motion in the generated output. We have considered quantitative and qualitative experiments, including comparisons with other state-of-the-arts for evaluation. Our results indicate that our model enhances performance compared with other models and performs favorably under different quantitive metrics PSNR, SSIM, LPIPS, and FVD.Recently, mimicking biologically inspired learning in generative models based on SNNs has been shown their effectiveness in different applications. SNNs are the third generation of neural networks, in which neurons communicate through binary signals known as spikes. Since SNNs are more energy-efficient than DNNs. Moreover, DNN models have been vulnerable to small adversarial perturbations that cause misclassification of legitimate images. This dissertation shows the proposed ``VAE-Sleep'' that combines ideas from VAE and the sleep mechanism leveraging the advantages of deep and spiking neural networks (DNN--SNN).On top of that, we present ``Defense–VAE–Sleep'' that extended work of ``VAE-Sleep'' model used to purge adversarial perturbations from contaminated images. We demonstrate the benefit of sleep in improving the generalization performance of the traditional VAE when the testing data differ in specific ways even by a small amount from the training data. We conduct extensive experiments, including comparisons with the state–of–the–art on different datasets.

This case study examined emitted sound and actuated piezoelectric current in a solderpaste jet printing machine to conclude whether quality degradation could be detected with an autoencoder machine learning model. An autoencoder was used to detect anomalies in non-realtime that were defined asa diameter drift with an averaging window from a target diameter. A sensor and datacollection system existed for the piezoelectric current, and a microphone was proposedas a new sensor to monitor the system. The sound was preprocessed with a Fast Fourier Transform to extract information of the existing frequencies. The results of the model, visualized through reconstruction error plots and an Area Under the Curve score, show that the autoencoder successfully detected conspicuous anomalies. The study indicated that anomalies can be detected prior to solder paste supply failure using the sound. When the temperature was varied or when the jetting head nozzle was clogged by residual solder paste, the sound model identified most anomalies although the current network showed better performance. / Denna fallstudie undersökte emitterat ljud och drivande piezoelektrisk ström i en jetprinter med lödpasta för att dra slutsatsen om kvalitetsbrister kunde detekteras med en autoencoder maskininlärningsmodell. En autoencoder användes för att detektera avvikelser definierade som diametertrend med ett glidande medelvärde från en bördiameter. Tidigare studier har visat att den piezoelektriska strömmen i liknande maskiner kan användas för att förutspå kvalitetsbrister. En mikrofon föreslogs som en ny sensor för att övervaka systemet. Ljudet förbehandlades genom en snabb fouriertransform och frekvensinnehållet användes som indata i modellen. Resultaten visualiserades genom rekonstruktionsfel och metoden Area Under the Curve. Modellen upptäckte framgångsrikt tydliga avvikelser. För vissa felfall visade ljudet som indata bättre prestanda än strömmen, och för andra visade strömmen bättre prestanda. Till exempel indikerade studien att avvikelser kan detekteras före lodpasta-försörjningsfel med ljudet. Under varierande temperatur och då munstycket var igentäppt av kvarvarande lödpasta identifierade nätverket med ljud som indata de flesta avvikelser även om nätverket med strömmen visade bättre prestanda.

With increased growth of the Internet of Things (IoT) and physical exposure of devices to adversaries, a class of physical attacks called side-channel analysis (SCA) has emerged which compromises the security of systems. While security claims of cryptographic algorithms are based on the complexity of classical cryptanalysis attacks, they exclude information leakage by implementations on hardware platforms. Recent standardization processes require assessment of hardware security against SCA. In this dissertation, we study SCA based on deep learning techniques (DL-SCA) as a universal analysis toolbox for assessing the leakage of secret information by hardware implementations. We demonstrate that DL-SCA techniques provide a trade-off between the amount of prior knowledge of a hardware implementation and the amount of measurements required to identify the secret key. A DL-SCA based on supervised learning requires a training set, including information about the details of the hardware implementation, for a successful attack. Supervised learning has been widely used in power analysis (PA) to recover the secret key with a limited size of measurements. We demonstrate a similar trend in fault injection analysis (FIA) by introducing fault intensity map analysis with a neural network key distinguisher (FIMA-NN). We use dynamic timing simulations on an ASIC implementation of AES to develop a statistical model for biased fault injection. We employ the model to train a convolutional neural network (CNN) key distinguisher that achieves a superior efficiency, nearly $10times$, compared to classical FIA techniques. When a priori knowledge of the details of hardware implementations is limited, we propose DL-SCA techniques based on unsupervised learning, called SCAUL, to extract the secret information from measurements without requiring a training set. We further demonstrate the application of reinforcement learning by introducing the SCARL attack, to estimate a proper model for the leakage of secret data in a self-supervised approach. We demonstrate the success of SCAUL and SCARL attacks using power measurements from FPGA implementations of the AES and Ascon authenticated ciphers, respectively, to recover entire 128-bit secret keys without using any prior knowledge or training data. / Doctor of Philosophy / With the growth of the Internet of Things (IoT) and mobile devices, cryptographic algorithms have become essential components of end-to-end cybersecurity. A cryptographic algorithm is a highly nonlinear mathematical function which often requires a secret key. Only the user who knows the secret key is able to interpret the output of the algorithm to find the encoded information. Standardized algorithms are usually secure against attacks in which in attacker attempts to find the secret key given a set of input data and the corresponding outputs of the algorithm. The security of algorithms is defined based on the complexity of known cryptanalysis attacks to recover the secret key. However, a device executing a cryptographic algorithm leaks information about the secret key. Several studies have shown that the behavior of a device, such as power consumption, electromagnetic radiation and the response to external stimulation provide additional information to an attacker that can be exploited to find the secret key with much less effort than cryptanalysis attacks. Hence, exposure of devices to adversaries has enabled the class of physical attacks called side-channel analysis (SCA). In SCA, an attacker attempts to find the secret key by observing the behavior of the device executing the algorithm. Recent government and industry standardization processes, which choose future cryptographic algorithms, require assessing the security of hardware implementations against SCA in addition to the algorithmic level security of the cryptographic systems. The difficulty of an SCA attack depends on the details of a hardware implementation and the form of information leakage on a particular device. The diversity of possible hardware implementations and platforms, including application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) and microprocessors, has hindered the development of a unified measure of complexity in SCA attacks. In this research, we study SCA with deep learning techniques (DL-SCA) as a universal methodology to evaluate the leakage of secret information by hardware platforms. We demonstrate that DL-SCA based on supervised learning can be considered as a generalization of classical SCA techniques, and is able to find the secret information with a limited size of measurements. However, supervised learning techniques require a training set of data that includes information about the details of hardware implementation. We propose unsupervised learning techniques that are able to find the secret key even without knowledge of the details of the hardware. We further demonstrate the ability of reinforcement learning in estimating a proper model for data leakage in a self-supervised approach. We demonstrate that DL-SCA techniques are able to find the secret information even if the timing of data leakage in measurements are random. Hence, traditional countermeasures are unable to protect a hardware implementation against DL-SCA attacks. We propose a unified countermeasure to protect the hardware implementations against a wide range of SCA attacks.