AbstractMachine learning is a subfield within artificial intelligence that revolves around constructingalgorithms that can learn from, and make predictions on data. Instead of following strict andstatic instruction, the system operates by adapting and learning from input data in order tomake predictions and decisions. This work will focus on a subcategory of machine learningcalled “MultilabelClassification”, which is the concept of where items introduced to thesystem is categorized by an analytical model, learned through supervised learning, whereeach instance of the dataset can belong to multiple labels, or classes.This paper presents the task of implementing a multilabelclassifier based on the ID3algorithm, which we call MLID (MultilabelIterative Dichotomiser). The solution is presentedboth in a sequentially executed version as well as an parallelized one.We also presents acomparison based on accuracy and execution time, that is performed against algorithms of asimilar nature in order to evaluate the viability of using ID3 as a base to further expand andbuild upon in regards of multi label classification.In order to evaluate the performance of the MLID algorithm, we have measured theexecution time, accuracy, and made a summarization of precision and recall into what iscalled Fmeasure,which is the harmonic mean of both precision and sensitivity of thealgorithm. These results are then compared to already defined and established algorithms,on a range of datasets of varying sizes, in order to assess the viability of the MLID algorithm.The results produced when comparing MLID against other multilabelalgorithms such asBinary relevance, Classifier Chains and Random Trees shows that MLID can compete withother classifiers in term of accuracy and Fmeasure,but in terms of training the algorithm,the time required is proven inferior. Through these results, we can conclude that MLID is aviable option to use as a multilabelclassifier. Although, some constraints inherited from theoriginal ID3 algorithm does impede the full utility of the algorithm, we are certain thatfollowing the same path of development and improvement as ID3 experienced would allowMLID to develop towards a suitable choice of algorithm for a diverse range of multilabelclassification problems.

abstract: One of the most remarkable outcomes resulting from the evolution of the web into Web 2.0, has been the propelling of blogging into a widely adopted and globally accepted phenomenon. While the unprecedented growth of the Blogosphere has added diversity and enriched the media, it has also added complexity. To cope with the relentless expansion, many enthusiastic bloggers have embarked on voluntarily writing, tagging, labeling, and cataloguing their posts in hopes of reaching the widest possible audience. Unbeknown to them, this reaching-for-others process triggers the generation of a new kind of collective wisdom, a result of shared collaboration, and the exchange of ideas, purpose, and objectives, through the formation of associations, links, and relations. Mastering an understanding of the Blogosphere can greatly help facilitate the needs of the ever growing number of these users, as well as producers, service providers, and advertisers into facilitation of the categorization and navigation of this vast environment. This work explores a novel method to leverage the collective wisdom from the infused label space for blog search and discovery. The work demonstrates that the wisdom space can provide a most unique and desirable framework to which to discover the highly sought after background information that could aid in the building of classifiers. This work incorporates this insight into the construction of a better clustering of blogs which boosts the performance of classifiers for identifying more relevant labels for blogs, and offers a mechanism that can be incorporated into replacing spurious labels and mislabels in a multi-labeled space. / Dissertation/Thesis / Doctoral Dissertation Computer Science 2015

This thesis aims to investigate how different feature extraction methods applied to textual data affect the results of multi-label classification. Two different Bag of Words extraction methods are used, specifically the Count Vector and the TF-IDF approaches. A word embedding method is also investigated, called the GloVe extraction method. Multi-label classification can be useful for categorizing items, such as pieces of music or news articles, that may belong to multiple classes or topics. The effect of using different pre-processing methods is also investigated, such as the use of N-grams, stop-word elimination, and stemming. Two different classifiers, an SVM and an ANN, are used for multi-label classification using a Binary Relevance approach. The results indicate that the choice of extraction method has a meaningful impact on the resulting classifications, but that no one method consistently outperforms the others. Instead the results show that the GloVe extraction method performs the best for the recall metrics, while the Bag of Words methods perform the best for the precision metrics. / Detta arbete ämnar att undersöka vilken effekt olika metoder för att extrahera särdrag ur textdata har när dessa används för att multi-tagga textdatan. Två metoder baserat på Bag of Words undersöks, närmare bestämt Count Vector-metoden samt TF-IDF-metoden. Även en metod som använder sig av word embessings undersöks, som kallas för GloVe-metoden. Multi-taggning av data kan vara användbart när datan, exempelvis musikaliska stycken eller nyhetsartiklar, kan tillhöra flera klasser eller områden. Även användandet av flera olika metoder för att förbehandla datan undersöks, såsom användandet utav N-gram, eliminering av icke-intressanta ord, samt transformering av ord med olika böjningsformer till gemensam stamform. Två olika klassificerare, en SVM samt en ANN, används för multi-taggningen genom använding utav en metod kallad Binary Relevance. Resultaten visar att valet av metod för extraktion av särdrag har en betydelsefull roll för den resulterande multi-taggningen, men att det inte finns en metod som ger bäst resultat genom alla tester. Istället indikerar resultaten att extraktionsmetoden baserad på GloVe presterar bäst när det gäller 'recall'-mätvärden, medan Bag of Words-metoderna presterar bäst gällade 'precision'-mätvärden.

L'exploitation des documents multimédias pose des problèmes de représentation des informations textuelles et visuelles contenues dans ces documents. Notre but est de proposer un modèle permettant de représenter chacune de ces informations et de les combiner en vue de deux tâches : la catégorisation et la recherche d'information. Ce modèle représente les documents sous forme de sacs de mots nécessitant la création de vocabulaires spécifiques. Le vocabulaire textuel, généralement de très grande taille, est constitué des mots apparaissant dans les documents. Le vocabulaire visuel est quant à lui construit en extrayant des caractéristiques de bas niveau des images. Nous étudions les différentes étapes de sa création et la pondération tfidf des mots visuels dans les images, inspirée des approches classiquement utilisées pour les mots textuels. Dans le contexte de la catégorisation de documents textuels, nous introduisons un critère qui sélectionne les mots les plus discriminants pour les catégories afin de réduire la taille du vocabulaire sans dégrader les résultats du classement. Nous présentons aussi dans le cadre multilabel, une méthode permettant de sélectionner les différentes catégories à associer à un document. En recherche d’information, nous proposons une approche analytique par apprentissage pour combiner linéairement les résultats issus des informations textuelles et visuelles, permettant d'améliorer significativement la recherche. Notre modèle est validé pour ces différentes tâches en participant à des compétitions internationales telles que XML Mining et ImageCLEF et sur des collections de taille conséquente / Exploiting multimedia documents leads to representation problems of the textual and visual information within documents. Our goal is to propose a model to represent these both information and to combine them for two tasks: categorization and information retrieval. This model represents documents as bags of words, which requires to define adapted vocabularies. The textual vocabulary, usually very large, corresponds to the words of documents while the visual one is created by extracting low-level features from images. We study the different steps of its creation and the tf.idf weighting of visual words in images usually used for textual words. In the context of the text categorization, we introduce a criterion to select the most discriminative words for categories in order to reduce the vocabulary size without degrading the results of classification. We also present in the multilabel context, a method that lets us to select the number of categories which must be associated with a document. In multimedia information retrieval, we propose an analytical approach based on machine learning techniques to linearly combine the results from textual and visual information which significantly improves research results. Our model has shown its efficiency on different collections of important size and was evaluated in several international competitions such as XML Mining and ImageCLEF

Muitos dos problemas de classificação descritos na literatura de Aprendizado de Máquina e Mineração de Dados dizem respeito à classificação de dados em que cada exemplo a ser classificado pertence a um conjunto finito, e geralmente pequeno, de classes que estão em um mesmo nível. Vários problemas de classificação, entretanto, são de natureza hierárquica, em que classes podem ser subclasses ou superclasses de outras classes. Em muitos problemas hierárquicos, principalmente no campo da Bioinformática, um ou mais exemplos podem ser associados a mais de uma classe simultaneamente. Esses problemas são conhecidos como problemas de classificação hierárquica tirrótulo. Nesta pesquisa, foram investigadas diferentes técnicas para lidar com esses tipos de problemas. Essas técnicas são baseadas em duas abordagens: local ou Top-Down e global ou One-Shot. Três técnicas descritas na literatura foram utilizadas. A primeira delas, chamada HMC-BR, é baseada na abordagem Top-Down, e utiliza uma estratégia de classificação binária chamada Um-Contra-Todos. As outras duas técnicas, baseadas na abordagem One-Shot, são chamadas C4.5H (uma extensão do algoritmo de indução de àrvores de decis~ao C4.5), e de Clus-HMC (baseada na noção de Predictive Clustering Trees, em que àrvores de decisão são estruturadas como uma hierarquia de grupos (clusters)). Além das técnicas descritas na literatura, duas novas técnicas foram propostas e implementadas nesta pesquisa, chamadas de HMC-LP e HMC-CT. Essas técnicas são variações hierárquicas de técnicas de classificação multirrótulo não hierárquicas. A técnica HMC-LP utiliza uma estratégia de combinação de classes e a técnica HMC-CT utiliza uma estratégia de decomposição de classes. Para a avaliação das técnicas, foram utilizadas medidas específicas para esse tipo de classificação. Os resultados experimentais mostraram que as técnicas propostas obtiveram desempenhos superiores ou semelhantes aos das técnicas descritas na literatura, dependendo da medida de avaliação utilizada e das características dos conjuntos de dados / Many of the classification problems described in the literature of Machine Learning and Data Mining are related to data classification where each example to be classified belongs to a finite, and usually small, set of classes located at the same level. There are many classification problems, however, that are of hierarchical nature, where classes can be subclasses or superclasses of other classes. In many hierarchical problems, mainly in the Bioinformatics field, one or more examples can be associated to more than one class simultaneously. These problems are known as hierarchical multilabel classification problems. In this research, different techniques to deal with these kinds of problems were investigated, based on two approaches, named local or Top-Down and global or One-Shot. Three techniques described in the literature were used. The first one, named HMC-BR, is based on the Top-Down approach, and uses a binary classification strategy named One-Against-All. The other two techniques, based on the One-Shot approach, are named C4.5H (an extension of the decision tree induction algorithm C4.5), and Clus-HMC (based on the notion of Predictive Clustering Trees, where decision trees are structured as a hierarchy of clusters). In addition to the techniques described in the literature, two new techniques were proposed, named HMC-LP and HMC-CT. These techniques are hierarchical variations of non-hierarchical multilabel classification techniques. The HMCLP technique uses a label combination strategy and the HMC-CT technique uses a label decomposition strategy. The evaluation of the techniques was performed using specific metrics for this kind of classification. The experimental results showed that the proposed techniques achieved better or similar performances than the techniques described in the literature, depending on the evaluation metric used and on the characteristics of the datasets

abstract: Deep learning architectures have been widely explored in computer vision and have depicted commendable performance in a variety of applications. A fundamental challenge in training deep networks is the requirement of large amounts of labeled training data. While gathering large quantities of unlabeled data is cheap and easy, annotating the data is an expensive process in terms of time, labor and human expertise. Thus, developing algorithms that minimize the human effort in training deep models is of immense practical importance. Active learning algorithms automatically identify salient and exemplar samples from large amounts of unlabeled data and can augment maximal information to supervised learning models, thereby reducing the human annotation effort in training machine learning models. The goal of this dissertation is to fuse ideas from deep learning and active learning and design novel deep active learning algorithms. The proposed learning methodologies explore diverse label spaces to solve different computer vision applications. Three major contributions have emerged from this work; (i) a deep active framework for multi-class image classication, (ii) a deep active model with and without label correlation for multi-label image classi- cation and (iii) a deep active paradigm for regression. Extensive empirical studies on a variety of multi-class, multi-label and regression vision datasets corroborate the potential of the proposed methods for real-world applications. Additional contributions include: (i) a multimodal emotion database consisting of recordings of facial expressions, body gestures, vocal expressions and physiological signals of actors enacting various emotions, (ii) four multimodal deep belief network models and (iii) an in-depth analysis of the effect of transfer of multimodal emotion features between source and target networks on classification accuracy and training time. These related contributions help comprehend the challenges involved in training deep learning models and motivate the main goal of this dissertation. / Dissertation/Thesis / Doctoral Dissertation Electrical Engineering 2018

Muitos dos problemas de classificação descritos na literatura de Aprendizado de Máquina e Mineração de Dados dizem respeito à classificação de dados em que cada exemplo a ser classificado pertence a um conjunto finito, e geralmente pequeno, de classes que estão em um mesmo nível. Vários problemas de classificação, entretanto, são de natureza hierárquica, em que classes podem ser subclasses ou superclasses de outras classes. Em muitos problemas hierárquicos, principalmente no campo da Bioinformática, um ou mais exemplos podem ser associados a mais de uma classe simultaneamente. Esses problemas são conhecidos como problemas de classificação hierárquica tirrótulo. Nesta pesquisa, foram investigadas diferentes técnicas para lidar com esses tipos de problemas. Essas técnicas são baseadas em duas abordagens: local ou Top-Down e global ou One-Shot. Três técnicas descritas na literatura foram utilizadas. A primeira delas, chamada HMC-BR, é baseada na abordagem Top-Down, e utiliza uma estratégia de classificação binária chamada Um-Contra-Todos. As outras duas técnicas, baseadas na abordagem One-Shot, são chamadas C4.5H (uma extensão do algoritmo de indução de àrvores de decis~ao C4.5), e de Clus-HMC (baseada na noção de Predictive Clustering Trees, em que àrvores de decisão são estruturadas como uma hierarquia de grupos (clusters)). Além das técnicas descritas na literatura, duas novas técnicas foram propostas e implementadas nesta pesquisa, chamadas de HMC-LP e HMC-CT. Essas técnicas são variações hierárquicas de técnicas de classificação multirrótulo não hierárquicas. A técnica HMC-LP utiliza uma estratégia de combinação de classes e a técnica HMC-CT utiliza uma estratégia de decomposição de classes. Para a avaliação das técnicas, foram utilizadas medidas específicas para esse tipo de classificação. Os resultados experimentais mostraram que as técnicas propostas obtiveram desempenhos superiores ou semelhantes aos das técnicas descritas na literatura, dependendo da medida de avaliação utilizada e das características dos conjuntos de dados / Many of the classification problems described in the literature of Machine Learning and Data Mining are related to data classification where each example to be classified belongs to a finite, and usually small, set of classes located at the same level. There are many classification problems, however, that are of hierarchical nature, where classes can be subclasses or superclasses of other classes. In many hierarchical problems, mainly in the Bioinformatics field, one or more examples can be associated to more than one class simultaneously. These problems are known as hierarchical multilabel classification problems. In this research, different techniques to deal with these kinds of problems were investigated, based on two approaches, named local or Top-Down and global or One-Shot. Three techniques described in the literature were used. The first one, named HMC-BR, is based on the Top-Down approach, and uses a binary classification strategy named One-Against-All. The other two techniques, based on the One-Shot approach, are named C4.5H (an extension of the decision tree induction algorithm C4.5), and Clus-HMC (based on the notion of Predictive Clustering Trees, where decision trees are structured as a hierarchy of clusters). In addition to the techniques described in the literature, two new techniques were proposed, named HMC-LP and HMC-CT. These techniques are hierarchical variations of non-hierarchical multilabel classification techniques. The HMCLP technique uses a label combination strategy and the HMC-CT technique uses a label decomposition strategy. The evaluation of the techniques was performed using specific metrics for this kind of classification. The experimental results showed that the proposed techniques achieved better or similar performances than the techniques described in the literature, depending on the evaluation metric used and on the characteristics of the datasets

Drug discovery is a process that takes many years and hundreds of millions of dollars to reveal a confident conclusion about a specific treatment. Part of this sophisticated process is based on preliminary investigations to suggest a set of chemical compounds as candidate drugs for the treatment. Computational resources have been playing a significant role in this part through a step known as virtual screening. From a data mining perspective, availability of rich data resources is key in training prediction models. Yet, the difficulties imposed by big expansion in data and its dimensionality are inevitable. In this thesis, I address the main challenges that come when data mining techniques are used for virtual screening. In order to achieve an efficient virtual screening using data mining, I start by addressing the problem of feature selection and provide analysis of best ways to describe a chemical compound for an enhanced screening performance. High-throughput screening (HTS) assays data used for virtual screening are characterized by a great class imbalance. To handle this problem of class imbalance, I suggest using a novel algorithm called DRAMOTE to narrow down promising candidate chemicals aimed at interaction with specific molecular targets before they are experimentally evaluated. Existing works are mostly proposed for small-scale virtual screening based on making use of few thousands of interactions. Thus, I propose enabling large-scale (or big) virtual screening through learning millions of interaction while exploiting any relevant dependency for a better accuracy. A novel solution called DRABAL that incorporates structure learning of a Bayesian Network as a step to model dependency between the HTS assays, is showed to achieve significant improvements over existing state-of-the-art approaches.

The Contracts Register is a public database containing contracts concluded by public institutions. Due to the number of documents in the database, data analysis is proble- matic. The objective of this thesis is to find a machine learning approach for sorting the contracts into categories by their area of interest (real estate services, construction, etc.) and implement the approach for usage on the web portal Hlídač státu. A large number of categories and a lack of a tagged dataset of contracts complicate the solution. 1

Dans cette thèse, nous proposons des méthodes a faible complexité pour la classification en présence d'un très grand nombre de catégories. Ces methodes permettent d'accelerer la prediction des classifieurs afin des les rendre utilisables dans les applications courantes. Nous proposons deux methodes destinées respectivement a la classification monolabel et a la classification multilabel. La première méthode utilise l'information hierarchique existante entre les catégories afin de créer un représentation binaire compact de celles-ci. La seconde approche , destinée aux problemes multilabel adpate le framework des Filtres de Bloom a la representation de sous ensembles de labels sous forme de de vecteurs binaires sparses. Dans chacun des cas, des classifieurs binaires sont appris afin de prédire les representations des catégories/labels et un algorithme permettant de retrouver l'ensemble de catégories pertinentes a partir de la représentation prédite est proposée. Les méthodes proposées sont validées par des expérience sur des données de grandes échelles et donnent des performances supérieures aux méthodes classiquement utilisées pour la classification extreme. / We propose in this thesis new methods to tackle classification problems with a large number of labes also called extreme classification. The proposed approaches aim at reducing the inference conplexity in comparison with the classical methods such as one-versus-rest in order to make learning machines usable in a real life scenario. We propose two types of methods respectively for single label and multilable classification. The first proposed approach uses existing hierarchical information among the categories in order to learn low dimensional binary representation of the categories. The second type of approaches, dedicated to multilabel problems, adapts the framework of Bloom Filters to represent subsets of labels with sparse low dimensional binary vectors. In both approaches, binary classifiers are learned to predict the new low dimensional representation of the categories and several algorithms are also proposed to recover the set of relevant labels. Large scale experiments validate the methods.