DNA microarrays are capable of measuring expression levels of thousands of genes, even the whole genome, in a single experiment. Based on this, they have been widely used to extend the studies of cancerous tissues to a genomic level. One of the main goals in DNA microarray experiments is to identify a set of relevant genes such that the desired outputs of the experiment mostly depend on this set, to the exclusion of the rest of the genes. This is motivated by the fact that the biological process in cell typically involves only a subset of genes, and not the whole genome. The task of selecting a subset of relevant genes is called feature (gene) selection. Herein, we propose a feature selection algorithm for gene expression data. It is based on the Hilbert-Schmidt independence criterion, and partly motivated by Rank-One Downdate (R1D) and the Singular Value Decomposition (SVD). The algorithm is computationally very fast and scalable to large data sets, and can be applied to response variables of arbitrary type (categorical and continuous). Experimental results of the proposed technique are presented on some synthetic and well-known microarray data sets. Later, we discuss the capability of HSIC in providing a general framework which encapsulates many widely used techniques for dimensionality reduction, clustering and metric learning. We will use this framework to explain two metric learning algorithms, namely the Fisher discriminant analysis (FDA) and closed form metric learning (CFML). As a result of this framework, we are able to propose a new metric learning method. The proposed technique uses the concepts from normalized cut spectral clustering and is associated with an underlying convex optimization problem.

DNA microarrays are capable of measuring expression levels of thousands of genes, even the whole genome, in a single experiment. Based on this, they have been widely used to extend the studies of cancerous tissues to a genomic level. One of the main goals in DNA microarray experiments is to identify a set of relevant genes such that the desired outputs of the experiment mostly depend on this set, to the exclusion of the rest of the genes. This is motivated by the fact that the biological process in cell typically involves only a subset of genes, and not the whole genome. The task of selecting a subset of relevant genes is called feature (gene) selection. Herein, we propose a feature selection algorithm for gene expression data. It is based on the Hilbert-Schmidt independence criterion, and partly motivated by Rank-One Downdate (R1D) and the Singular Value Decomposition (SVD). The algorithm is computationally very fast and scalable to large data sets, and can be applied to response variables of arbitrary type (categorical and continuous). Experimental results of the proposed technique are presented on some synthetic and well-known microarray data sets. Later, we discuss the capability of HSIC in providing a general framework which encapsulates many widely used techniques for dimensionality reduction, clustering and metric learning. We will use this framework to explain two metric learning algorithms, namely the Fisher discriminant analysis (FDA) and closed form metric learning (CFML). As a result of this framework, we are able to propose a new metric learning method. The proposed technique uses the concepts from normalized cut spectral clustering and is associated with an underlying convex optimization problem.

Cette thèse est rédigée par articles. Les articles sont rédigés en anglais et le reste de la thèse est rédigée en français. / Le travail établit une équivalence en termes de puissance entre les tests basés sur la alpha-distance de covariance et sur le critère d'indépendance de Hilbert-Schmidt (HSIC) avec fonction caractéristique de distribution de probabilité stable d'indice alpha avec paramètre d'échelle suffisamment petit. Des simulations en grandes dimensions montrent la supériorité des tests de distance de covariance et des tests HSIC par rapport à certains tests utilisant les copules. Des simulations montrent également que la distribution de Pearson de type III, très utile et moins connue, approche la distribution exacte de permutation des tests et donne des erreurs de type I précises. Une nouvelle méthode de sélection adaptative des paramètres d'échelle pour les tests HSIC est proposée. Trois simulations, dont deux sont empruntées de l'apprentissage automatique, montrent que la nouvelle méthode de sélection améliore la puissance des tests HSIC. Le problème de tests d'indépendance entre deux vecteurs est généralisé au problème de tests d'indépendance mutuelle entre plusieurs vecteurs. Le travail traite aussi d'un problème très proche à savoir, le test d'indépendance sérielle d'une suite multidimensionnelle stationnaire. La décomposition de Möbius des fonctions caractéristiques est utilisée pour caractériser l'indépendance. Des tests généralisés basés sur le critère d'indépendance de Hilbert-Schmidt et sur la distance de covariance en sont obtenus. Une équivalence est également établie entre le test basé sur la distance de covariance et le test HSIC de noyau caractéristique d'une distribution stable avec des paramètres d'échelle suffisamment petits. La convergence faible du test HSIC est obtenue. Un calcul rapide et précis des valeurs-p des tests développés utilise une distribution de Pearson de type III comme approximation de la distribution exacte des tests. Un résultat fascinant est l'obtention des trois premiers moments exacts de la distribution de permutation des statistiques de dépendance. Une méthodologie similaire a été développée pour le test d'indépendance sérielle d'une suite. Des applications à des données réelles environnementales et financières sont effectuées. / The main result establishes the equivalence in terms of power between the alpha-distance covariance test and the Hilbert-Schmidt independence criterion (HSIC) test with the characteristic kernel of a stable probability distribution of index alpha with sufficiently small scale parameters. Large-scale simulations reveal the superiority of these two tests over other tests based on the empirical independence copula process. They also establish the usefulness of the lesser known Pearson type III approximation to the exact permutation distribution. This approximation yields tests with more accurate type I error rates than the gamma approximation usually used for HSIC, especially when dimensions of the two vectors are large. A new method for scale parameter selection in HSIC tests is proposed which improves power performance in three simulations, two of which are from machine learning. The problem of testing mutual independence between many random vectors is addressed. The closely related problem of testing serial independence of a multivariate stationary sequence is also considered. The Möbius transformation of characteristic functions is used to characterize independence. A generalization to p vectors of the alpha -distance covariance test and the Hilbert-Schmidt independence criterion (HSIC) test with the characteristic kernel of a stable probability distributionof index alpha is obtained. It is shown that an HSIC test with sufficiently small scale parameters is equivalent to an alpha -distance covariance test. Weak convergence of the HSIC test is established. A very fast and accurate computation of p-values uses the Pearson type III approximation which successfully approaches the exact permutation distribution of the tests. This approximation relies on the exact first three moments of the permutation distribution of any test which can be expressed as the sum of all elements of a componentwise product of p doubly-centered matrices. The alpha -distance covariance test and the HSIC test are both of this form. A new selection method is proposed for the scale parameter of the characteristic kernel of the HSIC test. It is shown in a simulation that this adaptive HSIC test has higher power than the alpha-distance covariance test when data are generated from a Student copula. Applications are given to environmental and financial data.