[pt] Os desenvolvedores de sistemas cliente-servidor constantemente se deparam com questões de desempenho e escalabilidade, muitas vezes o número de clientes simultâneos de um servidor pode crescer de forma significativa. Além disso, se esses sistemas podem executar códigos de terceiros, passa a existir a preocupação com a questão de segurança da execução destes trechos de códigos. O objetivo desse trabalho é testar o desempenho de diversas combinações de modelos de concorrência e sandboxes. Como exemplo de sistema sujeito a solicitações concorrentes que precisa permitir a execução de programas de terceiros, temos os servidores web. O Xavante é um servidor web desenvolvido na linguagem de programação Lua que originalmente utiliza um modelo de programação concorrente baseado em co-rotinas. Esse servidor já possuía a flexibilidade de mapear tipos de requisição a tratadores diferentes. Alguns desses tratadores de requisi ção permitem a utilização de código de terceiros em tempo de execução e, por isso, já fazem uso de sandboxes a fim de oferecer um ambiente de execução protegido. Nesse trabalho testamos o uso dessas sandboxes nativas ao Xavante além de outros tipos de sandboxes. Também aumentamos as possibilidades de extensão do servidor, flexibilizando a arquitetura de modo a permitir diferentes estratégias de concorrência. Utilizando a nova arquitetura testamos o desempenho das diversas combinações de sandboxes e modelos de programação concorrente, permitindo a comparação dos modelos não só do ponto de vista qualitativo mas também medindo o impacto de desempenho do seu uso. / [en] Client-server developers constantly have to deal with questions concerning performance and scalability, since the number of simultaneous clients can grow significantly. Moreover, if these systems can execute third-party code, there is also the concern of secure execution of these pieces of code. This work goal is to benchmark several combinations of concurrency and sandboxes models. As an example of a system exposed to several concurrent requests that must be able to execute third-party code, we have the web servers. Xavante is a web server developed in the Lua programming language that originally uses an concurrency programming model based on coroutines. This server offers the flexibility to map request types to different handlers. Some of these request handlers dynamically allow the use of third-party code, therefore they make use of sandboxes in order to offer a protected running environment. In this work we test the use of Xavante´s native sandboxes and other types of sandboxes.We also improve the extensibility of the server, using a different architecture in order to allow different concurrency strategies. Using the new architecture we tested the performance of several combinations of sandboxes and concurrency programming models, allowing the model comparison not only in a qualitative point of view but also measuring the related performance impacts.

Cette thèse porte sur la conception de méthodes de surveillance de système à partir de données collectées sur des composants de conceptions identiques exploités par plusieurs processus. Nous nous sommes intéressés aux approches de diagnostic sans modèle a priori et plus particulièrement à l'élaboration des modèles de bon fonctionnement des composants à partir des données collectées sur le parc. Nous avons ainsi abordé ce problème comme un problème d'apprentissage multi-tâches qui consiste à élaborer conjointement les modèles de chaque composant, l'hypothèse sous-jacente étant que ces modèles partagent des parties communes. Dans le deuxième chapitre, on considère, dans un premier temps, des modèles linéaires de type multi-entrées/mono-sortie, ayant des structures a priori connues. Dans une première approche, après une phase d'analyse des modèles obtenus par régression linéaire pour les machines prises indépendamment les unes des autres, on identifie leurs parties communes, puis on procède à une nouvelle estimation des coefficients des modèles pour tenir compte des parties communes. Dans une seconde approche, on identifie simultanément les coefficients des modèles ainsi que leurs parties communes. Dans un deuxième temps, on cherche à obtenir directement les relations de redondance existant entre les variables mesurées par l'ACP. On s'affranchit alors des hypothèses sur la connaissance des structures des modèles et on prend en compte la présence d'erreurs sur l'ensemble des variables. Dans un troisième chapitre, une étude de la discernabilité des modèles est réalisée. Il s'agit de déterminer les domaines de variation des variables d'entrée garantissant la discernabilité des sorties des modèles. Ce problème d'inversion ensembliste est résolu soit en utilisant des pavés circonscrits aux différents domaines soit une approximation par pavage de ces domaines. Finalement, une application des approches proposées est réalisée sur des simulateurs d'échangeurs thermiques / This thesis deals with the conception of diagnosis systems using the data collected on identical machines working under different conditions. We are interested in the fault diagnosis method without a priori model and in modelling a fleet of machines using the data collected on all the machines. Hence, the problem can be formulated as a multi-task learning problem where models of the different machines are constructed simultaneously. These models are supposed to share some common parts. In the second chapter, we first consider linear models of type multiple-input/single-output. A first approach consists in analyzing the linear regression models generated using the data of each machine independently from the others in order to identify their common parts. Using this knowledge, new models for the machines are generated. The second approach consists in identifying simultaneously the coefficients of the models and their common parts. Secondly, the redundancy models are searched for using PCA. This way, no hypothesis on the knowledge of the structures of models describing the normal behavior of each machine is needed. In addition, this method allows to take into consideration the errors existing on all the variables since it does not differentiate between input or output variables. In the third chapter, a study on the discernibility of the outputs of the models is realized. The problem consists in identifying the range of variation of the input variables leading to discernible outputs of the models. This problem is solved using either the confined pavements to the different domains or a pavement method. Finally, the multi-task modelling approaches are applied on simulators of heat exchangers

Cette thèse traite du problème de l'apprentissage automatique supervisé dans le cas ou l'on considère plusieurs sorties, potentiellement de différent types. Nous proposons d'explorer trois différents axes de recherche en rapport avec ce sujet. Dans un premier temps, nous nous concentrons sur le cas homogène et proposons un cadre théorique pour étudier la consistance des problèmes multi-labels dans le cas de l'utilisation de chaîne de classifieurs. Ensuite, en nous plaçant dans ce cadre, nous proposons une borne de Rademacher sur l'erreur de généralisation pour tous les classifieurs de la chaîne et exposons deux facteurs de dépendance reliant les sorties les unes aux autres. Dans un deuxième temps, nous développons et analysons la performance de modèles en lien avec la théorie proposée. Toujours dans le cadre de l'apprentissage avec plusieurs sorties homogènes, nous proposons un modèle basé sur des réseaux de neurones pour l'analyse de sentiments à grain fin. Enfin, nous proposons un cadre et une étude empirique qui montrent la pertinence de l'apprentissage multi-objectif dans le cas de multiples sorties hétérogènes. / In this thesis, we study the problem of learning with multiple outputs related to different tasks, such as classification and ranking. In this line of research, we explored three different axes. First we proposed a theoretical framework that can be used to show the consistency of multi-label learning in the case of classifier chains, where outputs are homogeneous. Based on this framework, we proposed Rademacher generalization error bound made by any classifier in the chain and exhibit dependency factors relating each output to the others. As a result, we introduced multiple strategies to learn classifier chains and select an order for the chain. Still focusing on the homogeneous multi-output framework, we proposed a neural network based solution for fine-grained sentiment analysis and show the efficiency of the approach. Finally, we proposed a framework and an empirical study showing the interest of learning with multiple tasks, even when the outputs are of different types.

Given the "right" representation, learning is easy. This thesis studies representation learning and meta-learning, with a special focus on sparse representations. Meta-learning is fundamental to machine learning, and it translates to learning to learn itself. The presentation unfolds in two parts. In the first part, we establish learning theoretic results for learning sparse representations. The second part introduces new multi-task and meta-learning paradigms for representation learning. On the sparse representations front, our main pursuits are generalization error bounds to support a supervised dictionary learning model for Lasso-style sparse coding. Such predictive sparse coding algorithms have been applied with much success in the literature; even more common have been applications of unsupervised sparse coding followed by supervised linear hypothesis learning. We present two generalization error bounds for predictive sparse coding, handling the overcomplete setting (more original dimensions than learned features) and the infinite-dimensional setting. Our analysis led to a fundamental stability result for the Lasso that shows the stability of the solution vector to design matrix perturbations. We also introduce and analyze new multi-task models for (unsupervised) sparse coding and predictive sparse coding, allowing for one dictionary per task but with sharing between the tasks' dictionaries. The second part introduces new meta-learning paradigms to realize unprecedented types of learning guarantees for meta-learning. Specifically sought are guarantees on a meta-learner's performance on new tasks encountered in an environment of tasks. Nearly all previous work produced bounds on the expected risk, whereas we produce tail bounds on the risk, thereby providing performance guarantees on the risk for a single new task drawn from the environment. The new paradigms include minimax multi-task learning (minimax MTL) and sample variance penalized meta-learning (SVP-ML). Regarding minimax MTL, we provide a high probability learning guarantee on its performance on individual tasks encountered in the future, the first of its kind. We also present two continua of meta-learning formulations, each interpolating between classical multi-task learning and minimax multi-task learning. The idea of SVP-ML is to minimize the task average of the training tasks' empirical risks plus a penalty on their sample variance. Controlling this sample variance can potentially yield a faster rate of decrease for upper bounds on the expected risk of new tasks, while also yielding high probability guarantees on the meta-learner's average performance over a draw of new test tasks. An algorithm is presented for SVP-ML with feature selection representations, as well as a quite natural convex relaxation of the SVP-ML objective.

In many real-world applications of supervised learning, only a limited number of labeled examples are available because the cost of obtaining high-quality examples is high. Even with a relatively large number of labeled examples, the learning problem may still suffer from limited supervision as the complexity of the prediction function increases. Therefore, learning with limited supervision presents a major challenge to machine learning. With the goal of supervision reduction, this thesis studies the representation, discovery and incorporation of extra input and output information in learning. Information about the input space can be encoded by regularization. We first design a semi-supervised learning method for text classification that encodes the correlation of words inferred from seemingly irrelevant unlabeled text. We then propose a multi-task learning framework with a matrix-normal penalty, which compactly encodes the covariance structure of the joint input space of multiple tasks. To capture structure information that is more general than covariance and correlation, we study a class of regularization penalties on model compressibility. Then we design the projection penalty, which encodes the structure information from a dimension reduction while controlling the risk of information loss. Information about the output space can be exploited by error correcting output codes. Using the composite likelihood view, we propose an improved pairwise coding for multi-label classification, which encodes pairwise label density (as opposed to label comparisons) and decodes using variational methods. We then investigate problemdependent codes, where the encoding is learned from data instead of being predefined. We first propose a multi-label output code using canonical correlation analysis, where predictability of the code is optimized. We then argue that both discriminability and predictability are critical for output coding, and propose a max-margin formulation that promotes both discriminative and predictable codes. We empirically study our methods in a wide spectrum of applications, including document categorization, landmine detection, face recognition, brain signal classification, handwritten digit recognition, house price forecasting, music emotion prediction, medical decision, email analysis, gene function classification, outdoor scene recognition, and so forth. In all these applications, our proposed methods for encoding input and output information lead to significantly improved prediction performance.

Recueillir et labelliser un ensemble important et pertinent de données pour apprendre des systèmes de prédiction d'informations à partir de visages est à la fois difficile et long. Par conséquent, les données disponibles sont souvent de taille limitée comparée à la difficultés des tâches. Cela rend le problème du sur-apprentissage particulièrement important dans de nombreuses applications d'apprentissage statistique liées au visage. Dans cette thèse, nous proposons une nouvelle méthode de régression de labels multi-dimensionnels, nommée Hard Multi-Task Metric Learning for Kernel Regression (H-MT-MLKR). Notre méthode a été développée en focalisant sur la réduction du phénomène de sur-apprentissage. La méthode Metric Learning for Kernel Regression qui a été proposée par Kilian Q. Weinberger en 2007 vise à apprendre un sous-espace pour minimiser l'erreur quadratique d'un estimateur de Nadaraya-Watson sur la base d'apprentissage. Dans notre méthode, on étend la méthode MLKR pour une régression de labels multi-dimensionnels en ajoutant une nouvelle régularisation multi-tâches qui réduit les degrés de liberté du modèle appris ainsi que le sur-apprentissage. Nous évaluons notre méthode pour deux applications différentes, à savoir la localisation de points caractéristiques et la prédiction de l'intensité des Action Units. Nous présentons aussi un travail sur la prédiction des émotions en espace continu basé aussi sur l'estimateur de Nadaraya-Watson. Deux des systèmes proposés nous ont permis de remporter deux premières places à des concours internationaux, à savoir le Audio-Visual Emotion Challenge (AVEC'12) et le Facial Expression Recognition and Analysis challenge (FERA'15). / Collecting and labeling various and relevant data for training automatic facial information prediction systems is both hard and time-consuming. As a consequence, available data is often of limited size compared to the difficulty of the prediction tasks. This makes overfitting a particularly important issue in several face-related machine learning applications. In this PhD, we introduce a novel method for multi-dimensional label regression, namely Hard Multi-Task Metric Learning for Kernel Regression (H-MT-MLKR). Our proposed method has been designed taking a particular focus on overfitting reduction. The Metric Learning for Kernel Regression method (MLKR) that has been proposed by Kilian Q. Weinberger in 2007 aims at learning a subspace for minimizing the quadratic training error of a Nadaraya-Watson estimator. In our method, we extend MLKR for multi-dimensional label regression by adding a novel multi-task regularization that reduces the degrees of freedom of the learned model along with potential overfitting. We evaluate our regression method on two different applications, namely landmark localization and Action Unit intensity prediction. We also present our work on automatic emotion prediction in a continuous space which is based on the Nadaraya-Watson estimator as well. Two of our frameworks let us win international data science challenges, namely the Audio-Visual Emotion Challenge (AVEC’12) and the fully continuous Facial Expression Recognition and Analysis challenge (FERA’15).

Multi-Task Learning is today an interesting and promising field which many mention as a must for achieving the next level advancement within machine learning. However, in reality, Multi-Task Learning is much more rarely used in real-world implementations than its more popular cousin Transfer Learning. The questionis why that is and if Multi-Task Learning outperforms its Single-Task counterparts. In this thesis different Multi-Task Learning architectures were utilized in order to build a model that can handle labeling real technical issues within two categories. The model faces a challenging imbalanced data set with many labels to choose from and short texts to base its predictions on. Can task-sharing be the answer to these problems? This thesis investigated three Multi-Task Learning architectures and compared their performance to a Single-Task model. An authentic data set and two labeling tasks was used in training the models with the method of supervised learning. The four model architectures; Single-Task, Multi-Task, Cross-Stitched and the Shared-Private, first went through a hyper parameter tuning process using one of the two layer options LSTM and GRU. They were then boosted by auxiliary tasks and finally evaluated against each other.

With the recent advantages of machine learning in cheminformatics, the drug discovery process has been accelerated; providing a high impact in the field of medicine and public health. Molecular property and activity prediction are key elements in the early stages of drug discovery by helping prioritize the experiments and reduce the experimental work. In this thesis, a novel approach for multi-task regression using a text-based Transformer model is introduced and thoroughly explored for training on a number of properties or activities simultaneously. This multi-task regression with Transformer based model is inspired by the field of Natural Language Processing (NLP) which uses prefix tokens to distinguish between each task. In order to investigate our architecture two data categories are used; 133 biological activities from ExCAPE database and three physical chemistry properties from MoleculeNet benchmark datasets. The Transformer model consists of the embedding layer with positional encoding, a number of encoder layers, and a Feedforward Neural Network (FNN) to turn it into a regression problem. The molecules are represented as a string of characters using the Simplified Molecular-Input Line-Entry System (SMILES) which is a ’chemistry language’ with its own syntax. In addition, the effect of Transfer Learning is explored by experimenting with two pretrained Transformer models, pretrained on 1.5 million and on 100 million molecules. The text-base Transformer models are compared with a feature-based Support Vector Regression (SVR) with the Tanimoto kernel where the input molecules are encoded as Extended Connectivity Fingerprint (ECFP), which are calculated features. The results have shown that Transfer Learning is crucial for improving the performance on both property and activity predictions. On bioactivity tasks, the larger pretrained Transformer on 100 million molecules achieved comparable performance to the feature-based SVR model; however, overall SVR performed better on the majority of the bioactivity tasks. On the other hand, on physicochemistry property tasks, the larger pretrained Transformer outperformed SVR on all three tasks. Concluding, the multi-task regression architecture with the prefix token had comparable performance with the traditional feature-based approach on predicting different molecular properties or activities. Lastly, using the larger pretrained models trained on a wide chemical space can play a key role in improving the performance of Transformer models on these tasks.

No description available.

Lung cancer is the most commonly diagnosed cancer in the world and its finding is mainly incidental. New technologies and more specifically artificial intelligence has lately acquired big interest in the medical field as it can automate or bring new information to the medical staff. Many research have been done on the detection or classification of lung cancer. These works are done on local region of interest but only a few of them have been done looking at a full CT-scan. The aim of this thesis was to assess lung damages from CT images using new machine learning methods. First, single predictors had been learned by a 3D resnet architecture: cancer, emphysema, and opacities. Emphysema was learned by the network reaching an AUC of 0.79 whereas cancer and opacity predictions were not really better than chance AUC = 0.61 and AUC = 0.61. Secondly, a multi-task network was used to predict the factors altogether. A training with no prior knowledge and a transfer learning approach using self-supervision were compared. The transfer learning approach showed similar results in the multi-task approach for emphysema with AUC=0.78 vs 0.60 without pre-training and opacities with an AUC=0.61. Moreover using the pre-training approach enabled the network to reach the same performance as each of single factor predictor but with only one multi-task network which saves a lot of computational time. Finally a risk score can be derived from the training to use this information in a clinical context.