abstract: Machine learning models convert raw data in the form of video, images, audio, text, etc. into feature representations that are convenient for computational process- ing. Deep neural networks have proven to be very efficient feature extractors for a variety of machine learning tasks. Generative models based on deep neural networks introduce constraints on the feature space to learn transferable and disentangled rep- resentations. Transferable feature representations help in training machine learning models that are robust across different distributions of data. For example, with the application of transferable features in domain adaptation, models trained on a source distribution can be applied to a data from a target distribution even though the dis- tributions may be different. In style transfer and image-to-image translation, disen- tangled representations allow for the separation of style and content when translating images. This thesis examines learning transferable data representations in novel deep gen- erative models. The Semi-Supervised Adversarial Translator (SAT) utilizes adversar- ial methods and cross-domain weight sharing in a neural network to extract trans- ferable representations. These transferable interpretations can then be decoded into the original image or a similar image in another domain. The Explicit Disentangling Network (EDN) utilizes generative methods to disentangle images into their core at- tributes and then segments sets of related attributes. The EDN can separate these attributes by controlling the ow of information using a novel combination of losses and network architecture. This separation of attributes allows precise modi_cations to speci_c components of the data representation, boosting the performance of ma- chine learning tasks. The effectiveness of these models is evaluated across domain adaptation, style transfer, and image-to-image translation tasks. / Dissertation/Thesis / Masters Thesis Computer Science 2018

abstract: In recent years, machine learning and data mining technologies have received growing attention in several areas such as recommendation systems, natural language processing, speech and handwriting recognition, image processing and biomedical domain. Many of these applications which deal with physiological and biomedical data require person specific or person adaptive systems. The greatest challenge in developing such systems is the subject-dependent data variations or subject-based variability in physiological and biomedical data, which leads to difference in data distributions making the task of modeling these data, using traditional machine learning algorithms, complex and challenging. As a result, despite the wide application of machine learning, efficient deployment of its principles to model real-world data is still a challenge. This dissertation addresses the problem of subject based variability in physiological and biomedical data and proposes person adaptive prediction models based on novel transfer and active learning algorithms, an emerging field in machine learning. One of the significant contributions of this dissertation is a person adaptive method, for early detection of muscle fatigue using Surface Electromyogram signals, based on a new multi-source transfer learning algorithm. This dissertation also proposes a subject-independent algorithm for grading the progression of muscle fatigue from 0 to 1 level in a test subject, during isometric or dynamic contractions, at real-time. Besides subject based variability, biomedical image data also varies due to variations in their imaging techniques, leading to distribution differences between the image databases. Hence a classifier learned on one database may perform poorly on the other database. Another significant contribution of this dissertation has been the design and development of an efficient biomedical image data annotation framework, based on a novel combination of transfer learning and a new batch-mode active learning method, capable of addressing the distribution differences across databases. The methodologies developed in this dissertation are relevant and applicable to a large set of computing problems where there is a high variation of data between subjects or sources, such as face detection, pose detection and speech recognition. From a broader perspective, these frameworks can be viewed as a first step towards design of automated adaptive systems for real world data. / Dissertation/Thesis / Ph.D. Computer Science 2013

L’apprentissage par transfert consiste `a utiliser un jeu de taches pour influencerl’apprentissage et améliorer les performances sur une autre tache.Cependant, ce paradigme d’apprentissage peut en réalité gêner les performancessi les taches (sources et cibles) sont trop dissemblables. Un défipour l’apprentissage par transfert est donc de développer des approchesqui détectent et évitent le transfert négatif des connaissances utilisant tr`espeu d’informations sur la tache cible. Un cas particulier de ce type d’apprentissageest l’adaptation de domaine. C’est une situation o`u les tachessources et cibles sont identiques mais dans des domaines différents. Danscette thèse, nous proposons des approches adaptatives basées sur la factorisationmatricielle non-figurative permettant ainsi de trouver une représentationadéquate des données pour ce type d’apprentissage. En effet, unereprésentation utile rend généralement la structure latente dans les donnéesexplicite, et réduit souvent la dimensionnalité´e des données afin que d’autresméthodes de calcul puissent être appliquées. Nos contributions dans cettethèse s’articulent autour de deux dimensions complémentaires : théoriqueet pratique.Tout d’abord, nous avons propose deux méthodes différentes pour résoudrele problème de l’apprentissage par transfert non supervise´e bas´e sur destechniques de factorisation matricielle non-négative. La première méthodeutilise une procédure d’optimisation itérative qui vise `a aligner les matricesde noyaux calculées sur les bases des données provenant de deux taches.La seconde représente une approche linéaire qui tente de découvrir unplongement pour les deux taches minimisant la distance entre les distributionsde probabilité correspondantes, tout en préservant la propriété depositivité.Nous avons également propos´e un cadre théorique bas´e sur les plongementsHilbert-Schmidt. Cela nous permet d’améliorer les résultats théoriquesde l’adaptation au domaine, en introduisant une mesure de distancenaturelle et intuitive avec de fortes garanties de calcul pour son estimation.Les résultats propos´es combinent l’etancheite des bornes de la théoried’apprentissage de Rademacher tout en assurant l’estimation efficace deses facteurs cl´es.Les contributions théoriques et algorithmiques proposées ont et évaluéessur un ensemble de données de référence dans le domaine avec des résultatsprometteurs. / The ability of a human being to extrapolate previously gained knowledge to other domains inspired a new family of methods in machine learning called transfer learning. Transfer learning is often based on the assumption that objects in both target and source domains share some common feature and/or data space. If this assumption is false, most of transfer learning algorithms are likely to fail. In this thesis we propose to investigate the problem of transfer learning from both theoretical and applicational points of view.First, we present two different methods to solve the problem of unsuper-vised transfer learning based on Non-negative matrix factorization tech-niques. First one proceeds using an iterative optimization procedure that aims at aligning the kernel matrices calculated based on the data from two tasks. Second one represents a linear approach that aims at discovering an embedding for two tasks that decreases the distance between the cor-responding probability distributions while preserving the non-negativity property.We also introduce a theoretical framework based on the Hilbert-Schmidt embeddings that allows us to improve the current state-of-the-art theo-retical results on transfer learning by introducing a natural and intuitive distance measure with strong computational guarantees for its estimation. The proposed results combine the tightness of data-dependent bounds de-rived from Rademacher learning theory while ensuring the efficient esti-mation of its key factors.Both theoretical contributions and the proposed methods were evaluated on a benchmark computer vision data set with promising results. Finally, we believe that the research direction chosen in this thesis may have fruit-ful implications in the nearest future.

Convolutional Neural Networks (CNNs) are successful tools in image classification. CNNs are inspired by the animal visual cortex using a similar connectivity pattern as between neurons. The purpose of this thesis is to create a classifier, using transfer learning, that manages to classify images of traffic signs from the German Traffic Sign Recognition Benchmark (GTSRB) with good accuracy and to improve the performance further by tuning the hyperparameters. The pre-trained CNN used is the VGG16 network from the paper "Very deep convolutional networks for large-scale image recognition". The result showed that the VGG16 network got an accuracy of 74.5\% for the hyperparameter set where the learning rate was 1e-6, the batch size was 15 and the dropout rate 0.3. The conclusion was that transfer learning using the bottleneck features is a good tool for building a classifier with only a small amount of training data available and that the results probably could be further improved using more real data or data augmentation both for training and testing and by tuning more of the hyperparameters in the network.

abstract: Visual object recognition has achieved great success with advancements in deep learning technologies. Notably, the existing recognition models have gained human-level performance on many of the recognition tasks. However, these models are data hungry, and their performance is constrained by the amount of training data. Inspired by the human ability to recognize object categories based on textual descriptions of objects and previous visual knowledge, the research community has extensively pursued the area of zero-shot learning. In this area of research, machine vision models are trained to recognize object categories that are not observed during the training process. Zero-shot learning models leverage textual information to transfer visual knowledge from seen object categories in order to recognize unseen object categories. Generative models have recently gained popularity as they synthesize unseen visual features and convert zero-shot learning into a classical supervised learning problem. These generative models are trained using seen classes and are expected to implicitly transfer the knowledge from seen to unseen classes. However, their performance is stymied by overfitting towards seen classes, which leads to substandard performance in generalized zero-shot learning. To address this concern, this dissertation proposes a novel generative model that leverages the semantic relationship between seen and unseen categories and explicitly performs knowledge transfer from seen categories to unseen categories. Experiments were conducted on several benchmark datasets to demonstrate the efficacy of the proposed model for both zero-shot learning and generalized zero-shot learning. The dissertation also provides a unique Student-Teacher based generative model for zero-shot learning and concludes with future research directions in this area. / Dissertation/Thesis / Masters Thesis Computer Science 2020

Fault detection and diagnostics are crucial tasks in condition-based maintenance. Industries nowadays are in need of fault identification in their machines as early as possible to save money and take precautionary measures in case of fault occurrence. Also, it is beneficial for the smooth interference in the manufacturing process in which it avoids sudden malfunctioning. Having sufficient training data for industrial machines is also a major challenge which is a prerequisite for deep neural networks to train an accurate prediction model. Transfer learning in such cases is beneficial as it can be helpful in adapting different operating conditions and characteristics which is the casein real-life applications. Our work is focused on a pneumatic system which utilizes compressed air to perform operations and is used in different types of machines in the industrial field. Our novel contribution is to build upon a Domain Adversarial Neural Network (DANN) with a unique approach by incorporating ensembling techniques for diagnostics of air leakage problem in the pneumatic system under transfer learning settings. Our approach of using ensemble methods for feature extraction shows up to 5 % improvement in the performance. We have also performed a comparative analysis of our work with conventional machine and deep learning methods which depicts the importance of transfer learning and we have also demonstrated the generalization ability of our model. Lastly, we also mentioned a problem specific contribution by suggesting a feature engineering approach, such that it could be implemented on almost every pneumatic system and could potentially impact the prediction result positively. We demonstrate that our designed model with domain adaptation ability will be quite useful and beneficial for the industry by saving their time and money and providing promising results for this air leakage problem in the pneumatic system.

One way to improve dependency parsing scores for low-resource languages is to make use of existing resources from other closely related or otherwise similar languages. In this paper, we look at eleven Uralic target languages (Estonian, Finnish, Hungarian, Karelian, Livvi, Komi Zyrian, Komi Permyak, Moksha, Erzya, North Sámi, and Skolt Sámi) with treebanks of varying sizes and select transfer languages based on geographical, genealogical, and syntactic distances. We focus primarily on the performance of parser models trained on various combinations of geographically proximate and genealogically related transfer languages, in target-trained, zero-shot, and cross-lingual configurations. We find that models trained on combinations of geographically proximate and genealogically related transfer languages reach the highest LAS in most zero-shot models, while our highest-performing cross-lingual models were trained on genealogically related languages. We also find that cross-lingual models outperform zero-shot transfer models. We then select syntactically similar transfer languages for three target languages, and find a slight improvement in the case of Hungarian. We discuss the results and conclude with suggestions for possible future work.

The study and the analysis of marine ecosystems is a significant part of the marine science research. These systems are valuable resources for fisheries, improving water quality and can even be used in drugs production. The investigation of ichthyoplankton inhabiting these ecosystems is also an important research field. Ichthyoplankton are fish in their early stages of life. In this stage, the fish have relatively similar shape and are small in size. The currently used way of identifying them is not optimal. Marine scientists typically study such organisms by sending a team that collects samples from the sea which is then taken to the lab for further investigation. These samples need to be studied by an expert and usually end needing a DNA sequencing. This method is time-consuming and requires a high level of experience. The recent advances in AI have helped to solve and automate several difficult tasks which motivated us to develop a classification tool for ichthyoplankton. We show that using machine learning techniques, such as generative adversarial networks combined with transfer learning solves such a problem with high accuracy. We show that using traditional machine learning algorithms fails to solve it. We also give a general framework for creating a classification tool when the dataset used for training is a limited dataset. We aim to build a user-friendly tool that can be used by any user for the classification task and we aim to give a guide to the researchers so that they can follow in creating a classification tool.

京都大学 / 0048 / 新制・課程博士 / 博士(情報学) / 甲第22580号 / 情博第717号 / 新制||情||123(附属図書館) / 京都大学大学院情報学研究科社会情報学専攻 / (主査)教授 吉川 正俊, 教授 黒橋 禎夫, 教授 田島 敬史, 教授 田中 克己(京都大学 名誉教授) / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DFAM

Positive natural selection leaves detectable, distinctive patterns in the genome in the form of a selective sweep. Identifying areas of the genome that have undergone selective sweeps is an area of high interest as it enables understanding of species and population evolution. Previous work has accomplished this by evaluating patterns within summary statistics computed across the genome and through application of machine learning techniques to raw population genomic data. When using raw population genomic data, convolutional neural networks have most recently been employed as they can handle large input arrays and maintain correlations among elements. Yet, such models often require massive amounts of training data and can be computationally expensive to train for a given problem. Instead, transfer learning has recently been used in the image analysis literature to improve machine learning models by learning the important features of images from large unrelated datasets beforehand, and then refining these models through subsequent application on smaller and more relevant datasets. We combine transfer learning with convolutional neural networks to improve classification of selective sweeps from raw population genomic data. We show that the combination of transfer learning with convolutional neural networks allows for accurate classification of selective sweeps. / Includes bibliography. / Thesis (M.S.)--Florida Atlantic University, 2021. / FAU Electronic Theses and Dissertations Collection