Historically author profiling has been used in forensic linguistics. However, it is not until the last decades that the analysis method has worked into computer science and machine learning. In comparison, determining author profiling characteristics in machine learning is nothing new. This paper investigates the possibility to improve upon previous results with modern frameworks using data sets that have seen limited usage. The purpose of this master thesis was to use pre-trained transformers or embeddings together with transfer learning. In addition, to examine if general author profiling characteristics of anonymous users on internet forums or conversations on social media could be determined. The data sets used to investigate the questions above were PAN15 and PANDORA, which contains various properties in text data based on authors paired with ground truth labels such as gender, age, and Big Five/OCEAN. In addition, transfer learning of BERT and GloVe was used as a starting point to decrease the learning time of a new task. PAN15, a Twitter data set, did not contain enough data when training a model and was augmented using PANDORA, a Reddit-based data set. Ultimately, BERT obtained the best performance using a stacked approach, achieving 86 − 91% accuracy for each label on unseen data.

As more IoT devices enter the market it becomes increasingly important to develop reliable and adaptive ways of dealing with the data they generate. These must address data quality and reliability. Such solutions could benefit both the device producers and their customers who, as a result, could receive faster and better customer support services. Thus, this project's goal is twofold. First, it is to identify faulty data points generated by such devices. Second, it is to evaluate whether the knowledge gained from available/known sensors and appliances is transferable to other sensors on similar devices. This would make it possible to evaluate the behaviour of new appliances as soon as they are first switched on, rather than after sufficient data from them has been collected. This project uses time series data from three appliances: washing machine, washer&dryer and refrigerator. For these, two solutions are developed and tested: one for categorical and another for numerical variables. Categorical variables are analysed using the Average Value Frequency and the pure frequency of state-transition methods. Due to the limited number of possible states, the pure frequency proves to be the better solution, and the knowledge gained is transferred from the source device to the target one, with moderate success. Numerical variables are analysed using a One-class Support Vector Machine pipeline, with very promising results. Further, learning and forgetting mechanisms are developed to allow for the pipelines to adapt to changes in appliance patterns of behaviour. This includes a decay function for the numerical variables solution. Interestingly, the different weights for the source and target have little to no impact on the quality of the classification. / Nya IoT-enheter träder in på marknaden så det blir allt viktigare att utveckla tillförlitliga och anpassningsbara sätt att hantera de data de genererar. Dessa bör hantera datakvalitet och tillförlitlig- het. Sådana lösningar kan gynna båda tillverkarna av apparater och deras kunder som som ett resultat kan dra nytta av snabbare och bättre kundsupport / tjänster. Således har detta projekt två mål. Det första är att identifiera felaktiga datapunkter som genereras av sådana enheter. För det andra är det att utvärdera om kunskapen från tillgängliga / kända sensorer och apparater kan överföras till andra sensorer på liknande enheter. Detta skulle göra det möjligt att utvärdera beteendet hos nya apparater så snart de slås på första gången, snarare än efter att tillräcklig information från dem har samlats in. Detta projekt använder tidsseriedata från tre apparater: tvättmaskin, tvättmaskin och torktumlare och kylskåp. För dessa utvecklas och testas två lösningar: en för kategoriska variabler och en annan för numeriska variabler. De kategoriska variablerna analyseras med två metoder: Average Value Frequency och den rena frekvensen för tillståndsövergång. På grund av det begränsade antalet möjliga tillstånd visar sig den rena frekvensen vara den bättre lösningen, och kunskapen som erhålls överförs från källanordningen till målet, med måttlig framgång. De numeriska variablerna analyseras med hjälp av en One-class Support Vector Machine-pipeline, med mycket lovande resultat. Vidare utvecklas inlärnings- och glömningsmekanismer för att möjliggöra för rörledningarna att anpassa sig till förändringar i apparatens beteendemönster. Detta inkluderar en sönderfallningsfunktion för den numeriska variabellösningen. Intressant är att de olika vikterna för källan och målet har liten eller ingen inverkan på kvaliteten på klassificeringen.

Au cours des deux dernières décennies, l'intérêt porté aux interfaces cérébrales ou Brain Computer Interfaces (BCI) s’est considérablement accru, avec un nombre croissant de laboratoires de recherche travaillant sur le sujet. Depuis le projet Brain Computer Interface, où la BCI a été présentée à des fins de réadaptation et d'assistance, l'utilisation de la BCI a été étendue à d'autres applications telles que le neurofeedback et l’industrie du jeux vidéo. Ce progrès a été réalisé grâce à une meilleure compréhension de l'électroencéphalographie (EEG), une amélioration des systèmes d’enregistrement du EEG, et une augmentation de puissance de calcul.Malgré son potentiel, la technologie de la BCI n’est pas encore mature et ne peut être utilisé en dehors des laboratoires. Il y a un tas de défis qui doivent être surmontés avant que les systèmes BCI puissent être utilisés à leur plein potentiel. Ce travail porte sur des aspects importants de ces défis, à savoir la spécificité des systèmes BCI aux capacités physiques des utilisateurs, la robustesse de la représentation et de l'apprentissage du EEG, ainsi que la suffisance des données d’entrainement. L'objectif est de fournir un système BCI qui peut s’adapter aux utilisateurs en fonction de leurs capacités physiques et des variabilités dans les signaux du cerveau enregistrés.À ces fins, deux voies principales sont explorées : la première, qui peut être considérée comme un ajustement de haut niveau, est un changement de paradigmes BCI. Elle porte sur la création de nouveaux paradigmes qui peuvent augmenter les performances de la BCI, alléger l'inconfort de l'utilisation de ces systèmes, et s’adapter aux besoins des utilisateurs. La deuxième voie, considérée comme une solution de bas niveau, porte sur l’amélioration des techniques de traitement du signal et d’apprentissage statistique pour améliorer la qualité du signal EEG, la reconnaissance des formes, ainsi que la tache de classification.D'une part, une nouvelle méthodologie dans le contexte de la robotique d'assistance est définie : il s’agit d’une approche hybride où une interface physique est complémentée par une interface cérébrale pour une interaction homme-machine plus fluide. Ce système hybride utilise les capacités motrices résiduelles des utilisateurs et offre la BCI comme un choix optionnel : l'utilisateur choisit quand utiliser la BCI et peut alterner entre les interfaces cérébrales et musculaire selon le besoin.D'autre part, pour l’amélioration des techniques de traitement du signal et d'apprentissage statistique, ce travail utilise un cadre Riemannien. Un frein majeur dans le domaine de la BCI est la faible résolution spatiale du EEG. Ce problème est dû à l'effet de conductance des os du crâne qui agissent comme un filtre passe-bas non linéaire, en mélangeant les signaux de différentes sources du cerveau et réduisant ainsi le rapport signal-à-bruit. Par conséquent, les méthodes de filtrage spatial ont été développées ou adaptées. La plupart d'entre elles – à savoir la Common Spatial Pattern (CSP), la xDAWN et la Canonical Correlation Analysis (CCA) – sont basées sur des estimations de matrice de covariance. Les matrices de covariance sont essentielles dans la représentation d’information contenue dans le signal EEG et constituent un élément important dans leur classification. Dans la plupart des algorithmes d'apprentissage statistique existants, les matrices de covariance sont traitées comme des éléments de l'espace euclidien. Cependant, étant symétrique et défini positive (SDP), les matrices de covariance sont situées dans un espace courbe qui est identifié comme une variété riemannienne. Utiliser les matrices de covariance comme caractéristique pour la classification des signaux EEG, et les manipuler avec les outils fournis par la géométrie de Riemann, fournit un cadre solide pour la représentation et l'apprentissage du EEG. / In the last two decades, interest in Brain-Computer Interfaces (BCI) has tremendously grown, with a number of research laboratories working on the topic. Since the Brain-Computer Interface Project of Vidal in 1973, where BCI was introduced for rehabilitative and assistive purposes, the use of BCI has been extended to more applications such as neurofeedback and entertainment. The credit of this progress should be granted to an improved understanding of electroencephalography (EEG), an improvement in its measurement techniques, and increased computational power.Despite the opportunities and potential of Brain-Computer Interface, the technology has yet to reach maturity and be used out of laboratories. There are several challenges that need to be addresses before BCI systems can be used to their full potential. This work examines in depth some of these challenges, namely the specificity of BCI systems to users physical abilities, the robustness of EEG representation and machine learning, and the adequacy of training data. The aim is to provide a BCI system that can adapt to individual users in terms of their physical abilities/disabilities, and variability in recorded brain signals.To this end, two main avenues are explored: the first, which can be regarded as a high-level adjustment, is a change in BCI paradigms. It is about creating new paradigms that increase their performance, ease the discomfort of using BCI systems, and adapt to the user’s needs. The second avenue, regarded as a low-level solution, is the refinement of signal processing and machine learning techniques to enhance the EEG signal quality, pattern recognition and classification.On the one hand, a new methodology in the context of assistive robotics is defined: it is a hybrid approach where a physical interface is complemented by a Brain-Computer Interface (BCI) for human machine interaction. This hybrid system makes use of users residual motor abilities and offers BCI as an optional choice: the user can choose when to rely on BCI and could alternate between the muscular- and brain-mediated interface at the appropriate time.On the other hand, for the refinement of signal processing and machine learning techniques, this work uses a Riemannian framework. A major limitation in this filed is the EEG poor spatial resolution. This limitation is due to the volume conductance effect, as the skull bones act as a non-linear low pass filter, mixing the brain source signals and thus reducing the signal-to-noise ratio. Consequently, spatial filtering methods have been developed or adapted. Most of them (i.e. Common Spatial Pattern, xDAWN, and Canonical Correlation Analysis) are based on covariance matrix estimations. The covariance matrices are key in the representation of information contained in the EEG signal and constitute an important feature in their classification. In most of the existing machine learning algorithms, covariance matrices are treated as elements of the Euclidean space. However, being Symmetric and Positive-Definite (SPD), covariance matrices lie on a curved space that is identified as a Riemannian manifold. Using covariance matrices as features for classification of EEG signals and handling them with the tools provided by Riemannian geometry provide a robust framework for EEG representation and learning.

Indiana University-Purdue University Indianapolis (IUPUI) / In recent years, deep learning models have become popular in the real-time embedded application, but there are many complexities for hardware deployment because of limited resources such as memory, computational power, and energy. Recent research in the field of deep learning focuses on reducing the model size of the Convolution Neural Network (CNN) by various compression techniques like Architectural compression, Pruning, Quantization, and Encoding (e.g., Huffman encoding). Network pruning is one of the promising technique to solve these problems. This thesis proposes methods to prune the convolution neural network (SqueezeNet) without introducing network sparsity in the pruned model. This thesis proposes three methods to prune the CNN to decrease the model size of CNN without a significant drop in the accuracy of the model. 1: Pruning based on Taylor expansion of change in cost function Delta C. 2: Pruning based on L2 normalization of activation maps. 3: Pruning based on a combination of method 1 and method 2. The proposed methods use various ranking methods to rank the convolution kernels and prune the lower ranked filters afterwards SqueezeNet model is fine-tuned by backpropagation. Transfer learning technique is used to train the SqueezeNet on the CIFAR-10 dataset. Results show that the proposed approach reduces the SqueezeNet model by 72% without a significant drop in the accuracy of the model (optimal pruning efficiency result). Results also show that Pruning based on a combination of Taylor expansion of the cost function and L2 normalization of activation maps achieves better pruning efficiency compared to other individual pruning criteria and most of the pruned kernels are from mid and high-level layers. The Pruned model is deployed on BlueBox 2.0 using RTMaps software and model performance was evaluated.

京都大学 / 0048 / 新制・課程博士 / 博士(情報学) / 甲第21210号 / 情博第663号 / 新制||情||114(附属図書館) / 京都大学大学院情報学研究科知能情報学専攻 / (主査)教授 黒橋 禎夫, 教授 河原 達也, 教授 森 信介 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DFAM

No description available.

<p>Domain adaptation is one of the hottest directions in solving annotation insufficiency problem of deep learning. General domain adaptation is not consistent with the practical scenarios in the industry. In this thesis, we focus on two concerns as below.</p> <p>  </p> <p>  First is that labeled data are generally collected from multiple domains. In other words, multi-source adaptation is a more common situation. Simply extending these single-source approaches to the multi-source cases could cause sub-optimal inference, so specialized multi-source adaptation methods are essential. The main challenge in the multi-source scenario is a more complex divergence situation. Not only the divergence between target and each source plays a role, but the divergences among distinct sources matter as well. However, the significance of maintaining consistency among multiple sources didn't gain enough attention in previous work. In this thesis, we propose an Enhanced Consistency Multi-Source Adaptation (EC-MSA) framework to address it from three perspectives. First, we mitigate feature-level discrepancy by cross-domain conditional alignment, narrowing the divergence between each source and target domain class-wisely. Second, we enhance multi-source consistency via dual mix-up, diminishing the disagreements among different sources. Third, we deploy a target distilling mechanism to handle the uncertainty of target prediction, aiming to provide high-quality pseudo-labeled target samples to benefit the previous two aspects. Extensive experiments are conducted on several common benchmark datasets and demonstrate that our model outperforms the state-of-the-art methods.</p> <p>  </p> <p>  Second is that data privacy and security is necessary in practice. That is, we hope to keep the raw data stored locally while can still obtain a satisfied model. In such a case, the risk of data leakage greatly decreases. Therefore, it is natural for us to combine the federated learning paradigm with domain adaptation. Under the source-private setting, the main challenge for us is to expose information from the source domain to the target domain while make sure that the communication process is safe enough. In this thesis, we propose a method named Fourier Transform-Assisted Federated Domain Adaptation (FTA-FDA) to alleviate the difficulties in two ways. We apply Fast Fourier Transform to the raw data and transfer only the amplitude spectra during the communication. Then frequency space interpolations between these two domains are conducted, minimizing the discrepancies while ensuring the contact of them and keeping raw data safe. What's more, we make prototype alignments by using the model weights together with target features, trying to reduce the discrepancy in the class level. Experiments on Office-31 demonstrate the effectiveness and competitiveness of our approach, and further analyses prove that our algorithm can help protect privacy and security.</p>

The automotive industry is becoming more dependent on sustainable and efficient systems within vehicles. With the diverse combination of conditions affecting vehicle performance, such as environmental conditions and drivers' behaviour, the interest in monitoring machine health increases. This master thesis examines the machine learning approach to sensor reconstruction of hauler engine sensors for deviation detection applications across multiple domains. A novel proposal for sequence learning and modelling was by introducing a weighted difference of sequence derivatives. Impacts of including differences of derivatives assisted the learning capabilities of sequential data for the majority of the target sensors across multiple operating domains. Robust sensor reconstruction was also examined by using inductive transfer learning with a Long Short-Term Memory-Domain Adversarial Neural Network. Obtained results implied an improvement in using the Long Short-Term Memory-Domain Adversarial Neural Network, then using a regular Long Short-Term Memory network trained on both source and target domains. Suggested methods were evaluated towards model-based performance and computational limitations. The combined aspects of model performance and system performance are used to discuss the trade-offs using each proposed method.

Data augmentation is a technique that acquires more training data by augmenting available samples, where the training data is used to fit model parameters. Data augmentation is utilized due to a shortage of training data in certain domains and to reduce overfitting. Augmenting a training dataset for image classification with a Generative Adversarial Network (GAN) has been shown to increase classification accuracy. This report investigates if transfer learning within a GAN can further increase classification accuracy when utilizing the augmented training dataset. The method section describes a specific GAN architecture for the experiments that includes a label condition. When using transfer learning within the specific GAN architecture, a statistical analysis shows a statistically significant increase in classification accuracy for a classification problem with the EMNIST dataset, which consists of images of handwritten alphanumeric characters. In the discussion section, the authors analyze the results and motivates other use cases for the proposed GAN architecture. / Datautökning är en metod som skapar mer träningsdata genom att utöka befintlig träningsdata, där träningsdatan används för att anpassa modellers parametrar. Datautökning används på grund av en brist på träningsdata inom vissa områden samt för att minska overfitting. Att utöka ett träningsdataset för att genomföra bildklassificering med ett generativt adversarialt nätverk (GAN) har visats kunna öka precisionen av klassificering av bilder. Denna rapport undersöker om transferlärande inom en GAN kan vidare öka klassificeringsprecisionen när ett utökat träningsdataset används. Metoden beskriver en specific GANarkitektur som innehåller ett etikettvillkor. När transferlärande används inom den utvalda GAN-arkitekturen visar en statistisk analys en statistiskt säkerställd ökning av klassificeringsprecisionen för ett klassificeringsproblem med EMNIST datasetet, som innehåller bilder på handskrivna bokstäver och siffror. I diskussionen diskuteras orsakerna bakom resultaten och fler användningsområden nämns.

Breast cancer is one of the most common cancers among women in the world, and the average error rate among radiologists during diagnosis is 30%. Computer-aided medical diagnosis aims to assist doctors by giving them a second opinion, thus decreasing the error rate. Convolutional neural networks (CNNs) have shown to be good for visual detection and recognition tasks, and have been explored in combination with transfer learning. However, the performance of a deep learning model does not only rely on the model itself, but on the nature of the dataset as well In breast cancer diagnosis, the area surrounding a mass provides useful context for diagnosis. In this study, we explore providing different amounts of context to the CNN model ResNet50, to see how it affects the model’s performance. We test masses with no additional context, twice the amount of original context and four times the amount of original context, using 10-fold cross-validation with ROC AUC and average precision (AP ) as our metrics. The results suggest that providing additional context does improve the model’s performance. However, giving two and four times the amount of context seems to give similar performance. / Bröstcancer är en av de vanligaste cancersjukdomar bland kvinnor i världen, och den genomsnittliga felfrekvensen under diagnoser är 30%. Datorstödd medicinsk diagnos syftar till att hjälpa läkare genom att ge dem en andra åsikt, vilket minskar felfrekvensen. Konvolutionella neurala nätverk (CNNs) har visat sig vara bra för visuell detektering och igenkännande, och har utforskats i samband med det s.k. “transfer learning”. Prestationen av en djup inlärningsmodell är däremot inte enbart beroende på modellen utan också på datasetets natur. I bröstcancerdiagnos ger området runt en bröstmassa användbar kontext för diagnos. I den här studien testar vi att ge olika mängder kontext till CNNmodellen ResNet50, för att se hur det påverkar modellens prestanda. Vi testar bröstmassor utan ytterligare kontext, dubbelt så mycket som den originala mängden kontext och fyra gånger så mycket som den orginala mängden kontext, med hjälp av “10-fold cross-validation” med ROC AUC och “average precision” (AP ) som våra mätvärden. Resultaten visar att mer kontext förbättrar modellens prestanda. Däremot verkar att ge två och fyra gånger så mycket kontext resultera i liknande prestanda.