Unsupervised image-to-image translation (UNIT) is a challenging task in the image manipulation field, where input images in a visual domain are mapped into another domain with desired visual patterns (also called styles). An ideal direction in this field is to build a model that can map an input image in a domain to multiple target domains and generate diverse outputs in each target domain, which is termed as multi-domain and multi-modal unsupervised image-to-image translation (MMUIT). Recent studies have shown remarkable results in UNIT but they suffer from four main limitations: (1) State-of-the-art UNIT methods are either built from several two-domain mappings that are required to be learned independently or they generate low-diversity results, a phenomenon also known as model collapse. (2) Most of the manipulation is with the assistance of visual maps or digital labels without exploring natural languages, which could be more scalable and flexible in practice. (3) In an MMUIT system, the style latent space is usually disentangled between every two image domains. While interpolations within domains are smooth, interpolations between two different domains often result in unrealistic images with artifacts when interpolating between two randomly sampled style representations from two different domains. Improving the smoothness of the style latent space can lead to gradual interpolations between any two style latent representations even between any two domains. (4) It is expensive to train MMUIT models from scratch at high resolution. Interpreting the latent space of pre-trained unconditional GANs can achieve pretty good image translations, especially high-quality synthesized images (e.g., 1024x1024 resolution). However, few works explore building an MMUIT system with such pre-trained GANs. In this thesis, we focus on these vital issues and propose several techniques for building better MMUIT systems. First, we base on the content-style disentangled framework and propose to fit the style latent space with Gaussian Mixture Models (GMMs). It allows a well-trained network using a shared disentangled style latent space to model multi-domain translations. Meanwhile, we can randomly sample different style representations from a Gaussian component or use a reference image for style transfer. Second, we show how the GMM-modeled latent style space can be combined with a language model (e.g., a simple LSTM network) to manipulate multiple styles by using textual commands. Then, we not only propose easy-to-use constraints to improve the smoothness of the style latent space in MMUIT models, but also design a novel metric to quantitatively evaluate the smoothness of the style latent space. Finally, we build a new model to use pretrained unconditional GANs to do MMUIT tasks.

Readability metrics assess the ease with which human beings read and understand written texts. With the advent of machine learning techniques that allow computers to also analyse text, this provides an interesting opportunity to investigate whether readability metrics can be used to inform on the ease with which machines understand texts. To that end, the specific machine analysed in this paper uses word embeddings to conduct unsupervised sentiment analysis. This specification minimises the need for labelling and human intervention, thus relying heavily on the machine instead of the human. Across two different datasets, sentiment predictions are made using Google’s Word2Vec word embedding algorithm, and are evaluated to produce a dichotomous output variable per sentiment. This variable, representing whether a prediction is correct or not, is then used as the dependent variable in a logistic regression with 17 readability metrics as independent variables. The resulting model has high explanatory power and the effects of readability metrics on the results from the sentiment analysis are mostly statistically significant. However, metrics affect sentiment classification in the two datasets differently, indicating that the metrics are expressions of linguistic behaviour unique to the datasets. The implication of the findings is that readability metrics could be used directly in sentiment classification models to improve modelling accuracy. Moreover, the results also indicate that machines are able to pick up on information that human beings do not pick up on, for instance that certain words are associated with more positive or negative sentiments. / Läsbarhetsmått bedömer hur lätt eller svårt det är för människor att läsa och förstå skrivna texter. Eftersom nya maskininlärningstekniker har utvecklats kan datorer numera också analysera texter. Därför är en intressant infallsvinkel huruvida läsbarhetsmåtten också kan användas för att bedöma hur lätt eller svårt det är för maskiner att förstå texter. Mot denna bakgrund använder den specifika maskinen i denna uppsats ordinbäddningar i syfte att utföra oövervakad sentimentanalys. Således minimeras behovet av etikettering och mänsklig handpåläggning, vilket resulterar i en mer djupgående analys av maskinen istället för människan. I två olika dataset jämförs rätt svar mot sentimentförutsägelser från Googles ordinbäddnings-algoritm Word2Vec för att producera en binär utdatavariabel per sentiment. Denna variabel, som representerar om en förutsägelse är korrekt eller inte, används sedan som beroende variabel i en logistisk regression med 17 olika läsbarhetsmått som oberoende variabler. Den resulterande modellen har högt förklaringsvärde och effekterna av läsbarhetsmåtten på resultaten från sentimentanalysen är mestadels statistiskt signifikanta. Emellertid är effekten på klassificeringen beroende på dataset, vilket indikerar att läsbarhetsmåtten ger uttryck för olika lingvistiska beteenden som är unika till datamängderna. Implikationen av resultaten är att läsbarhetsmåtten kan användas direkt i modeller som utför sentimentanalys för att förbättra deras prediktionsförmåga. Dessutom indikerar resultaten också att maskiner kan plocka upp på information som människor inte kan, exempelvis att vissa ord är associerade med positiva eller negativa sentiment.

Product matching in e-commerce is an area that faces more and more challenges with growth in the e-commerce marketplace as well as variation in the quality of data available online for each product. Product matching for e-commerce provides competitive possibilities for vendors and flexibility for customers by identifying identical products from different sources. Traditional methods in product matching are often conducted through rule-based methods and methods tackling the issue through machine learning usually do so through unimodal systems. Moreover, existing methods would tackle the issue through product identifiers which are not always unified for each product. This thesis provides multimodal approaches through product name, description, and image to the problem area of product matching that outperforms unimodal approaches. Three multimodal approaches were taken, one unsupervised and two supervised. The unsupervised approach uses straight-forward embedding space to nearest neighbor search that provides better results than unimodal approaches. One of the supervised multimodal approaches uses Siamese network on the embedding space which outperforms the unsupervised multi- modal approach. Finally, the last supervised approach instead tackles the issue by exploiting distance differences in each modality through logistic regression and a decision system that provided the best results. / Produktmatchning inom e-handel är ett område som möter fler och fler utmaningar med hänsyn till den tillväxt som e-handelsmarknaden undergått och fortfarande undergår samt variation i kvaliteten på den data som finns tillgänglig online för varje produkt. Produktmatchning inom e-handel är ett område som ger konkurrenskraftiga möjligheter för leverantörer och flexibilitet för kunder genom att identifiera identiska produkter från olika källor. Traditionella metoder för produktmatchning genomfördes oftast genom regelbaserade metoder och metoder som utnyttjar maskininlärning gör det vanligtvis genom unimodala system. Dessutom utnyttjar mestadels av befintliga metoder produktidentifierare som inte alltid är enhetliga för varje produkt mellan olika källor. Denna studie ger istället förslag till multimodala tillvägagångssätt som istället använder sig av produktnamn, produktbeskrivning och produktbild för produktmatchnings-problem vilket ger bättre resultat än unimodala metoder. Tre multimodala tillvägagångssätt togs, en unsupervised och två supervised. Den unsupervised metoden använder embeddings vektorerna rakt av för att göra en nearest neighborsökning vilket gav bättre resultat än unimodala tillvägagångssätt. Ena supervised multimodal tillvägagångssätten använder siamesiska nätverk på embedding utrymmet vilket gav resultat som överträffade den unsupervised multimodala tillvägagångssättet. Slutligen tar den sista supervised metoden istället avståndsskillnader i varje modalitet genom logistisk regression och ett beslutssystem som gav bästa resultaten.

No description available.

No description available.

Representation Learning is ubiquitous in state-of-the-art machine learning workflow, including data exploration/visualization, data preprocessing, data model learning, and model interpretations. However, the majority of the newly proposed Representation Learning methods are more suitable for problems with a large amount of data. Applying these methods to problems with a limited amount of data may lead to unsatisfactory performance. Therefore, there is a need for developing Representation Learning methods which are tailored for problems with ``small data", such as, clinical and biomedical data analytics. In this dissertation, we describe our studies of tackling the challenging clinical and biomedical data analytics problem from four perspectives: data preprocessing, temporal data representation learning, output representation learning, and joint input-output representation learning. Data scaling is an important component in data preprocessing. The objective in data scaling is to scale/transform the raw features into reasonable ranges such that each feature of an instance will be equally exploited by the machine learning model. For example, in a credit flaw detection task, a machine learning model may utilize a person's credit score and annual income as features, but because the ranges of these two features are different, a machine learning model may consider one more heavily than another. In this dissertation, I thoroughly introduce the problem in data scaling and describe an approach for data scaling which can intrinsically handle the outlier problem and lead to better model prediction performance. Learning new representations for data in the unstandardized form is a common task in data analytics and data science applications. Usually, data come in a tubular form, namely, the data is represented by a table in which each row is a feature (row) vector of an instance. However, it is also common that the data are not in this form; for example, texts, images, and video/audio records. In this dissertation, I describe the challenge of analyzing imperfect multivariate time series data in healthcare and biomedical research and show that the proposed method can learn a powerful representation to encounter various imperfections and lead to an improvement of prediction performance. Learning output representations is a new aspect of Representation Learning, and its applications have shown promising results in complex tasks, including computer vision and recommendation systems. The main objective of an output representation algorithm is to explore the relationship among the target variables, such that a prediction model can efficiently exploit the similarities and potentially improve prediction performance. In this dissertation, I describe a learning framework which incorporates output representation learning to time-to-event estimation. Particularly, the approach learns the model parameters and time vectors simultaneously. Experimental results do not only show the effectiveness of this approach but also show the interpretability of this approach from the visualizations of the time vectors in 2-D space. Learning the input (feature) representation, output representation, and predictive modeling are closely related to each other. Therefore, it is a very natural extension of the state-of-the-art by considering them together in a joint framework. In this dissertation, I describe a large-margin ranking-based learning framework for time-to-event estimation with joint input embedding learning, output embedding learning, and model parameter learning. In the framework, I cast the functional learning problem to a kernel learning problem, and by adopting the theories in Multiple Kernel Learning, I propose an efficient optimization algorithm. Empirical results also show its effectiveness on several benchmark datasets. / Computer and Information Science

With increased growth of the Internet of Things (IoT) and physical exposure of devices to adversaries, a class of physical attacks called side-channel analysis (SCA) has emerged which compromises the security of systems. While security claims of cryptographic algorithms are based on the complexity of classical cryptanalysis attacks, they exclude information leakage by implementations on hardware platforms. Recent standardization processes require assessment of hardware security against SCA. In this dissertation, we study SCA based on deep learning techniques (DL-SCA) as a universal analysis toolbox for assessing the leakage of secret information by hardware implementations. We demonstrate that DL-SCA techniques provide a trade-off between the amount of prior knowledge of a hardware implementation and the amount of measurements required to identify the secret key. A DL-SCA based on supervised learning requires a training set, including information about the details of the hardware implementation, for a successful attack. Supervised learning has been widely used in power analysis (PA) to recover the secret key with a limited size of measurements. We demonstrate a similar trend in fault injection analysis (FIA) by introducing fault intensity map analysis with a neural network key distinguisher (FIMA-NN). We use dynamic timing simulations on an ASIC implementation of AES to develop a statistical model for biased fault injection. We employ the model to train a convolutional neural network (CNN) key distinguisher that achieves a superior efficiency, nearly $10times$, compared to classical FIA techniques. When a priori knowledge of the details of hardware implementations is limited, we propose DL-SCA techniques based on unsupervised learning, called SCAUL, to extract the secret information from measurements without requiring a training set. We further demonstrate the application of reinforcement learning by introducing the SCARL attack, to estimate a proper model for the leakage of secret data in a self-supervised approach. We demonstrate the success of SCAUL and SCARL attacks using power measurements from FPGA implementations of the AES and Ascon authenticated ciphers, respectively, to recover entire 128-bit secret keys without using any prior knowledge or training data. / Doctor of Philosophy / With the growth of the Internet of Things (IoT) and mobile devices, cryptographic algorithms have become essential components of end-to-end cybersecurity. A cryptographic algorithm is a highly nonlinear mathematical function which often requires a secret key. Only the user who knows the secret key is able to interpret the output of the algorithm to find the encoded information. Standardized algorithms are usually secure against attacks in which in attacker attempts to find the secret key given a set of input data and the corresponding outputs of the algorithm. The security of algorithms is defined based on the complexity of known cryptanalysis attacks to recover the secret key. However, a device executing a cryptographic algorithm leaks information about the secret key. Several studies have shown that the behavior of a device, such as power consumption, electromagnetic radiation and the response to external stimulation provide additional information to an attacker that can be exploited to find the secret key with much less effort than cryptanalysis attacks. Hence, exposure of devices to adversaries has enabled the class of physical attacks called side-channel analysis (SCA). In SCA, an attacker attempts to find the secret key by observing the behavior of the device executing the algorithm. Recent government and industry standardization processes, which choose future cryptographic algorithms, require assessing the security of hardware implementations against SCA in addition to the algorithmic level security of the cryptographic systems. The difficulty of an SCA attack depends on the details of a hardware implementation and the form of information leakage on a particular device. The diversity of possible hardware implementations and platforms, including application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) and microprocessors, has hindered the development of a unified measure of complexity in SCA attacks. In this research, we study SCA with deep learning techniques (DL-SCA) as a universal methodology to evaluate the leakage of secret information by hardware platforms. We demonstrate that DL-SCA based on supervised learning can be considered as a generalization of classical SCA techniques, and is able to find the secret information with a limited size of measurements. However, supervised learning techniques require a training set of data that includes information about the details of hardware implementation. We propose unsupervised learning techniques that are able to find the secret key even without knowledge of the details of the hardware. We further demonstrate the ability of reinforcement learning in estimating a proper model for data leakage in a self-supervised approach. We demonstrate that DL-SCA techniques are able to find the secret information even if the timing of data leakage in measurements are random. Hence, traditional countermeasures are unable to protect a hardware implementation against DL-SCA attacks. We propose a unified countermeasure to protect the hardware implementations against a wide range of SCA attacks.

<p dir="ltr">The past two decades have seen photography shift from the hands of professionals to that of the average smartphone user. However, fitting a camera module in the palm of your hand has come with its own cost. The reduced sensor size, and hence the smaller pixels, has made the image inherently noisier due to fewer photons being captured. To compensate for fewer photons, we can increase the exposure of the camera but this may exaggerate the effect of hand shake, making the image blurrier. The presence of both noise and blur has made the post-processing algorithms necessary to produce a clean and sharp image. </p><p dir="ltr">In this thesis, we discuss various methods of deblurring images in the presence of noise. Specifically, we address the problem of photon-limited deconvolution, both with and without the underlying blur kernel being known i.e. non-blind and blind deconvolution respectively. For the problem of blind deconvolution, we discuss the flaws of the conventional approach of joint estimation of the image and blur kernel. This approach, despite its drawbacks, has been the go-to method for solving blind deconvolution for decades. We then discuss the relatively unexplored kernel-first approach to solving the problem which is numerically stable than the alternating minimization counterpart. We show how to implement this framework using deep neural networks in practice for both photon-limited and noiseless deconvolution problems. </p>

<p dir="ltr">Bias and fairness issues in artificial intelligence algorithms are major concerns as people do not want to use AI software they cannot trust. This work uses college admissions data as a case study to develop methodology to define and detect bias, and then introduces a new method for interactive bias mitigation.</p><p dir="ltr">Admissions data spanning six years was used to create machine learning-based predictive models to determine whether a given student would be directly admitted into the School of Science under various scenarios at a large urban research university. During this time, submission of standardized test scores as part of a student’s application became optional which led to interesting questions about the impact of standardized test scores on admission decisions. We developed and analyzed predictive models to understand which variables are important in admissions decisions, and how the decision to exclude test scores affects the demographics of the students who are admitted.</p><p dir="ltr">Then, using a variety of bias and fairness metrics, we analyzed these predictive models to detect biases the models may carry with respect to three variables chosen to represent sensitive populations: gender, race, and whether a student was the first in his/her family to attend college. We found that high accuracy rates can mask underlying algorithmic bias towards these sensitive groups.</p><p dir="ltr">Finally, we describe our method for bias mitigation which uses a combination of machine learning and user interaction. Because bias is intrinsically a subjective and context-dependent matter, it requires human input and feedback. Our approach allows the user to iteratively and incrementally adjust bias and fairness metrics to change the training dataset for an AI model to make the model more fair. This interactive bias mitigation approach was then used to successfully decrease the biases in three AI models in the context of undergraduate student admissions.</p>

The rapid evolution of Connected and Autonomous Vehicles (CAVs) has led to a surge in research on efficient anomaly detection methods to ensure their safe and reliable operation. While state-of-the-art deep learning models offer promising results in this domain, their high computational requirements present challenges for deployment in resource-constrained environments, such as Electronic Control Units (ECU) in vehicles. In this context, we consider using the ensemble learning technique specifically the cascaded modeling approach for real-time and resource-efficient multivariate time series anomaly detection in CAVs. The study was done in collaboration with SCANIA, a transport solutions provider. The company is now undergoing a transformation of providing autonomous and sustainable solutions and this work will contribute towards that transformation. Our methodology employs unsupervised learning techniques to construct a cascade of models, comprising a coarse-grained model with lower computational complexity at level one, and a more intricate fine-grained model at level two. Furthermore, we incorporate cascaded model training to refine the complex model's ability to make decisions on uncertain and anomalous events, leveraging insights from the simpler model. Through extensive experimentation, we investigate the trade-off between model performance and computational complexity, demonstrating that our proposed cascaded model achieves greater efficiency with no performance degradation. Further, we do a comparative analysis of the impact of probabilistic versus deterministic approaches and assess the feasibility of model training at edge environments using the Federated Learning concept.