721 |
Learning with unlabeled data. / 在未標記的數據中的機器學習 / CUHK electronic theses & dissertations collection / Zai wei biao ji de shu ju zhong de ji qi xue xiJanuary 2009 (has links)
In the first part, we deal with the unlabeled data that are in good quality and follow the conditions of semi-supervised learning. Firstly, we present a novel method for Transductive Support Vector Machine (TSVM) by relaxing the unknown labels to the continuous variables and reducing the non-convex optimization problem to a convex semi-definite programming problem. In contrast to the previous relaxation method which involves O (n2) free parameters in the semi-definite matrix, our method takes advantage of reducing the number of free parameters to O (n), so that we can solve the optimization problem more efficiently. In addition, the proposed approach provides a tighter convex relaxation for the optimization problem in TSVM. Empirical studies on benchmark data sets demonstrate that the proposed method is more efficient than the previous semi-definite relaxation method and achieves promising classification results comparing with the state-of-the-art methods. Our second contribution is an extended level method proposed to efficiently solve the multiple kernel learning (MKL) problems. In particular, the level method overcomes the drawbacks of both the Semi-Infinite Linear Programming (SILP) method and the Subgradient Descent (SD) method for multiple kernel learning. Our experimental results show that the level method is able to greatly reduce the computational time of MKL over both the SD method and the SILP method. Thirdly, we discuss the connection between two fundamental assumptions in semi-supervised learning. More specifically, we show that the loss on the unlabeled data used by TSVM can be essentially viewed as an additional regularizer for the decision boundary. We further show that this additional regularizer induced by the TSVM is closely related to the regularizer introduced by the manifold regularization. Both of them can be viewed as a unified regularization framework for semi-supervised learning. / In the second part, we discuss how to employ the unlabeled data for building reliable classification systems in three scenarios: (1) only poorly-related unlabeled data are available, (2) good quality unlabeled data are mixed with irrelevant data and there are no prior knowledge on their composition, and (3) no unlabeled data are available but can be achieved from the Internet for text categorization. We build several frameworks to deal with the above cases. Firstly, we present a study on how to deal with the weakly-related unlabeled data, called the Supervised Self-taught Learning framework, which can transfer knowledge from the unlabeled data actively. The proposed model is able to select those discriminative features or representations, which are more appropriate for classification. Secondly, we also propose a novel framework that can learn from a mixture of unlabeled data, where good quality unlabeled data are mixed with unlabeled irrelevant samples. Moreover, we do not need the prior knowledge on which data samples are relevant or irrelevant. Consequently it is significantly different from the recent framework of semi-supervised learning with universum and the framework of Universum Support Vector Machine. As an important contribution, we have successfully formulated this new learning approach as a Semi-definite Programming problem, which can be solved in polynomial time. A series of experiments demonstrate that this novel framework has advantages over the semi-supervised learning on both synthetic and real data in many facets. Finally, for third scenario, we present a general framework for semi-supervised text categorization that collects the unlabeled documents via Web search engines and utilizes them to improve the accuracy of supervised text categorization. Extensive experiments have demonstrated that the proposed semi-supervised text categorization framework can significantly improve the classification accuracy. Specifically, the classification error is reduced by 30% averaged on the nine data sets when using Google as the search engine. / We consider the problem of learning from both labeled and unlabeled data through the analysis on the quality of the unlabeled data. Usually, learning from both labeled and unlabeled data is regarded as semi-supervised learning, where the unlabeled data and the labeled data are assumed to be generated from the same distribution. When this assumption is not satisfied, new learning paradigms are needed in order to effectively explore the information underneath the unlabeled data. This thesis consists of two parts: the first part analyzes the fundamental assumptions of semi-supervised learning and proposes a few efficient semi-supervised learning models; the second part discusses three learning frameworks in order to deal with the case that unlabeled data do not satisfy the conditions of semi-supervised learning. / Xu, Zenglin. / Advisers: Irwin King; Michael R. Lyu. / Source: Dissertation Abstracts International, Volume: 70-09, Section: B, page: . / Thesis (Ph.D.)--Chinese University of Hong Kong, 2009. / Includes bibliographical references (leaves 158-179). / Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Electronic reproduction. [Ann Arbor, MI] : ProQuest Information and Learning, [200-] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Abstracts in English and Chinese. / School code: 1307.
|
722 |
Generalized regularized learning. / 廣義正則化學習 / CUHK electronic theses & dissertations collection / Guang yi zheng ze hua xue xiJanuary 2007 (has links)
A classical algorithm in classification is the support vector machine (SVM) algorithm. Based on Vapnik's statistical learning theory, it tries to find a linear boundary with maximum margin to separate the given data into different classes. In non-separable case, SVM uses a kernel trick to map the data onto a feature space and finds a linear boundary in the new space. / Different algorithms are derived from the framework. When the empirical error is defined by a quadratic loss, we have generalized regularized least-squares learning algorithm. When the idea is applied to SVM, we obtain semi-parametric SVM algorithm. Besides, we derive the third algorithm which generalizes the kernel logistic regression algorithm. / How to choose non-regularized features? We give some empirical studies. We use dimensionality reduction techniques in text categorization, extract some non-regularized intrinsic features for the high dimensional data, and report improved results. / Instead of understanding SVM's behavior from Vapnik's theory, our work follows regularized learning viewpoint. In regularized learning, people try to find a solution from a function space which has small empirical error in explaining the input-output relationship for training data, yet keeping the simplicity of the solution. / To provide the simplicity, the complexity of the solution is penalized, which involves all features in the function space. An equal penalty, as in standard regularized learning, is reasonable without knowing the significance of individual features. But how about if we have prior knowledge that some features are more important than others? Instead of penalizing all features, we study a generalized regularized learning framework where part of the function space is not penalized, and derive its corresponding solution. / Two generalized algorithms need to solve positive definite linear systems to get the parameters. How to solve a large-scale linear system efficiently? Different from previous work in machine learning where people generally resort to conjugate gradient method, our work proposes to use a domain decomposition approach. New interpretations and improved results are reported accordingly. / Li, Wenye. / "September 2007." / Advisers: Kwong-Sak Leung; Kin-Hong Lee. / Source: Dissertation Abstracts International, Volume: 69-08, Section: B, page: 4850. / Thesis (Ph.D.)--Chinese University of Hong Kong, 2007. / Includes bibliographical references (p. 101-109). / Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Electronic reproduction. [Ann Arbor, MI] : ProQuest Information and Learning, [200-] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Abstracts in English and Chinese. / School code: 1307.
|
723 |
Fast Graph Laplacian regularized kernel learning via semidefinite-quadratic-linear programming.January 2011 (has links)
Wu, Xiaoming. / Thesis (M.Phil.)--Chinese University of Hong Kong, 2011. / Includes bibliographical references (p. 30-34). / Abstracts in English and Chinese. / Abstract --- p.i / Acknowledgement --- p.iv / Chapter 1 --- Introduction --- p.1 / Chapter 2 --- Preliminaries --- p.4 / Chapter 2.1 --- Kernel Learning Theory --- p.4 / Chapter 2.1.1 --- Positive Semidefinite Kernel --- p.4 / Chapter 2.1.2 --- The Reproducing Kernel Map --- p.6 / Chapter 2.1.3 --- Kernel Tricks --- p.7 / Chapter 2.2 --- Spectral Graph Theory --- p.8 / Chapter 2.2.1 --- Graph Laplacian --- p.8 / Chapter 2.2.2 --- Eigenvectors of Graph Laplacian --- p.9 / Chapter 2.3 --- Convex Optimization --- p.10 / Chapter 2.3.1 --- From Linear to Conic Programming --- p.11 / Chapter 2.3.2 --- Second-Order Cone Programming --- p.12 / Chapter 2.3.3 --- Semidefinite Programming --- p.12 / Chapter 3 --- Fast Graph Laplacian Regularized Kernel Learning --- p.14 / Chapter 3.1 --- The Problems --- p.14 / Chapter 3.1.1 --- MVU --- p.16 / Chapter 3.1.2 --- PCP --- p.17 / Chapter 3.1.3 --- Low-Rank Approximation: from SDP to QSDP --- p.18 / Chapter 3.2 --- Previous Approach: from QSDP to SDP --- p.20 / Chapter 3.3 --- Our Formulation: from QSDP to SQLP --- p.21 / Chapter 3.4 --- Experimental Results --- p.23 / Chapter 3.4.1 --- The Results --- p.25 / Chapter 4 --- Conclusion --- p.28 / Bibliography --- p.30
|
724 |
Distributionally Robust Optimization and its Applications in Machine LearningKang, Yang January 2017 (has links)
The goal of Distributionally Robust Optimization (DRO) is to minimize the cost of running a stochastic system, under the assumption that an adversary can replace the underlying baseline stochastic model by another model within a family known as the distributional uncertainty region. This dissertation focuses on a class of DRO problems which are data-driven, which generally speaking means that the baseline stochastic model corresponds to the empirical distribution of a given sample.
One of the main contributions of this dissertation is to show that the class of data-driven DRO problems that we study unify many successful machine learning algorithms, including square root Lasso, support vector machines, and generalized logistic regression, among others. A key distinctive feature of the class of DRO problems that we consider here is that our distributional uncertainty region is based on optimal transport costs. In contrast, most of the DRO formulations that exist to date take advantage of a likelihood based formulation (such as Kullback-Leibler divergence, among others). Optimal transport costs include as a special case the so-called Wasserstein distance, which is popular in various statistical applications.
The use of optimal transport costs is advantageous relative to the use of divergence-based formulations because the region of distributional uncertainty contains distributions which explore samples outside of the support of the empirical measure, therefore explaining why many machine learning algorithms have the ability to improve generalization. Moreover, the DRO representations that we use to unify the previously mentioned machine learning algorithms, provide a clear interpretation of the so-called regularization parameter, which is known to play a crucial role in controlling generalization error. As we establish, the regularization parameter corresponds exactly to the size of the distributional uncertainty region.
Another contribution of this dissertation is the development of statistical methodology to study data-driven DRO formulations based on optimal transport costs. Using this theory, for example, we provide a sharp characterization of the optimal selection of regularization parameters in machine learning settings such as square-root Lasso and regularized logistic regression.
Our statistical methodology relies on the construction of a key object which we call the robust Wasserstein profile function (RWP function). The RWP function similar in spirit to the empirical likelihood profile function in the context of empirical likelihood (EL). But the asymptotic analysis of the RWP function is different because of a certain lack of smoothness which arises in a suitable Lagrangian formulation.
Optimal transport costs have many advantages in terms of statistical modeling. For example, we show how to define a class of novel semi-supervised learning estimators which are natural companions of the standard supervised counterparts (such as square root Lasso, support vector machines, and logistic regression). We also show how to define the distributional uncertainty region in a purely data-driven way. Precisely, the optimal transport formulation allows us to inform the shape of the distributional uncertainty, not only its center (which given by the empirical distribution). This shape is informed by establishing connections to the metric learning literature. We develop a class of metric learning algorithms which are based on robust optimization. We use the robust-optimization-based metric learning algorithms to inform the distributional uncertainty region in our data-driven DRO problem. This means that we endow the adversary with additional which force him to spend effort on regions of importance to further improve generalization properties of machine learning algorithms.
In summary, we explain how the use of optimal transport costs allow constructing what we call double-robust statistical procedures. We test all of the procedures proposed in this paper in various data sets, showing significant improvement in generalization ability over a wide range of state-of-the-art procedures.
Finally, we also discuss a class of stochastic optimization algorithms of independent interest which are particularly useful to solve DRO problems, especially those which arise when the distributional uncertainty region is based on optimal transport costs.
|
725 |
Multiscale Modeling of Adsorbate Interactions on Transition Metal Alloy SurfacesBoes, Jacob Russell 01 April 2017 (has links)
Transition metals represent some of the first catalysts used in industrial processes and are still used today to produce many of the most needed chemicals. Adopting from ancient metallurgical techniques, it followed that the performance of these basic transition metals can be refined by adding multiple components. Since that time, improvements to these alloy catalysts has been mostly incremental due to the difficulty of producing new catalysts experimentally and a lack of fundamental understanding of the underlying physics. More recently, computational chemistry has proven itself an increasingly effective means for identifying these underlying physics. Through the use of d-band interactions of adsorbates with the surface, basic adsorption characteristics can be predicted across transition metals with limited initial information. However, although these models function well as high-level screening tools, much work is yet to be done before optimal catalysts can be comfortably designed from properties which experimentalists can directly control. This remains particularly challenging for alloy modeling, primarily due to the large number of possible atomic configurations, even for two metal systems. This work focuses on developing the methods for modeling optimal reaction properties at the surface of a transition metal alloy. Based on thermodynamic equilibrium between the surface, bulk, and gas reservoir, a model for the prediction of segregation under vacuum and adsorbate conditions can be predicted. Furthermore, by relating strain in the bulk lattice constant to the adsorption energies of varying local active sites, the optimal surface compositions can be related to bulk composition; a feature which can easily be selected for. Although useful for identifying trends across bulk composition space, these methods are limited to a small subset of active site configurations. To capture the complexity of more sophisticated processes, such as segregation, higher-timescale methods are required. Traditional computational tools are often too expensive to implement for these methods, and as such, they are usually completed with less-accurate potentials. In this work, we demonstrate that machine learning techniques have improved accuracy compared to physical potentials. We then go on to demonstrate how this improved accuracy can lead to experimentally accurate predictions of segregation.
|
726 |
Kernel based learning methods for pattern and feature analysisWu, Zhili 01 January 2004 (has links)
No description available.
|
727 |
On feature selection, kernel learning and pairwise constraints for clustering analysisZeng, Hong 01 January 2009 (has links)
No description available.
|
728 |
Modélisation du défaut d'excentration dans une machine asynchrone : application au diagnostic et à la commande de deux machines spécifiquesAndriamalala, Rijaniaina Njaksasoa 02 July 2009 (has links)
Cette thèse étudie le problème d'excentration rotorique et le défaut d'alimentation dans une Machine Asynchrone Double-Etoile. En outre, elle analyse la commande de deux machines asynchrones connectées en série ainsi que la détection de défaut d'onduleur dans ce cadre de connexion. Elle propose également une reconfiguration de commande permettant de tolérer le default détecté. Le travail débute par des méthodes de modélisation d'une machine asynchrone polyphasée excentrée. La première méthode prend en considération les harmoniques de bobinage mais néglige les effets d'encoches. La deuxième se limite au premier harmonique de bobinage mais tient compte des harmoniques d'encoches. A partir de ces deux modèles, nous avons pu caractériser analytiquement les signatures d'excentration. En simulation, les deux modèles donnent les mêmes spectres à basse fréquence. Toutefois, le deuxième modèle offre des raies supplémentaires à haute fréquence. Ces raies résultent de la combinaison de l'excentration et des harmoniques d'encoches. Nous entamons ensuite la détection de défauts d'alimentation ou d'excentration lorsque la machine double-étoile est alimentée par un onduleur. Pour éliminer le défaut de court-circuit ou le défaut d'ouverture de circuit, nous proposons à la fois une modification de la structure du convertisseur et une adaptation de l'algorithme de commande. Les résultats de simulation sont prometteurs. Les courants retrouvent le régime équilibré et la même amplitude que celle d'avant le défaut. La vitesse se stabilise à nouveau après une courte perturbation assez forte. En ce qui concerne le problème d'excentration, les prédictions analytiques se justifient même en cas d'alimentation par onduleur. Dans le cas d’une alimentation par commande vectorielle, les variables de commande peuvent également servir d'outils de diagnostic d'excentration. Enfin, nous abordons la commande découplée d'une machine hexaphasée et d’une machine triphasée connectées en série. Le découplage de la commande de deux moteurs est tout d’abord examiné analytiquement. Des tests de simulation sont ensuite effectués pour confirmer l’efficacité du découplage. Nous avons montré par simulation qu'il est aussi possible d'éliminer les perturbations dues à un défaut d'un interrupteur de puissance grâce à une structure du convertisseur et à une adaptation de l'algorithme de commande. La plupart des prédictions par simulation on été justifiées expérimentalement. / This thesis investigates various fault and detection issues in a Dual-Stator Winding Induction Machine Drive including rotor eccentricity problems and inverter switch faults. In addition, the control of six-phase series-connected two-motor drives and the related fault detection and fault tolerant strategy issues are studied as well. The work starts with new modeling methods for an eccentric multiphase induction machine. The first proposed method considers only the winding harmonics and neglects the slotting effects. Then, a second method is proposed, considering the first winding harmonic and the slotting ones. From both modeling techniques, eccentricity signatures are extracted. Simulation results show that both modeling techniques provide identical spectra at low frequency; however, the second technique gives additional high frequency sidebands. These sidebands are the results of the interaction between the eccentricity and the slot harmonics. Eccentricity and inverter faults in a Dual-Stator Winding Induction Machine Drive are subsequently investigated. The inverter topology and the control algorithm are reconfigured to deal with short-circuit and open circuit faults on the inverter side so that the stator currents become balanced again and reach their pre-fault magnitude. Simulation results show promising results. The speed is stabilized after a short disturbance due to the fault. Besides, analytical method has been successfully used to predict eccentricity fault, although the machine was inverter fed. Control variables have been effectively used as diagnosis tools for eccentricity fault in a vector controlled machine. Additionally, decoupled control of six-phase and three-phase machines connected in series has been investigated. Firstly, decoupling control using analytical method is predicted. Several simulations are then carried out to confirm the decoupling effectiveness. For this special drive, elimination of the disturbances due to a switch fault is also possible thanks to an appropriate converter topology and adaptation of the control algorithm. Most of simulation predictions are confirmed by experimental results.
|
729 |
Studying the ability of finding single and interaction effects with Random Forest, and its application in psychiatric geneticsNeira Gonzalez, Lara Andrea January 2018 (has links)
Psychotic disorders such as schizophrenia and bipolar disorder have a strong genetic component. The aetiology of psychoses is known to be complex, including additive effects from multiple susceptibility genes, interactions between genes, environmental risk factors, and gene by environment interactions. With the development of new technologies such as genome-wide association studies and imputation of ungenotyped variants, the amount of genomic data has increased dramatically leading to the necessary use of Machine Learning techniques. Random Forest has been widely used to study the underlying genetic factors of psychiatric disorders such as epistasis and gene-gene interactions. Several authors have investigated the ability of this algorithm in finding single and interaction effects, but have reported contradictory results. Therefore, in order to examine Random Forest ability of detecting single and interaction effects based on different variable importance measures, I conducted a simulation study assessing whether the algorithm was able to detect single and interaction models under different correlation conditions. The results suggest that the optimal Variable Importance Measures to use in real situations under correlation is the unconditional unscaled permutation variable importance measure. Several studies have shown bias in one of the most popular variable importance measures, the Gini importance. Hence, in a second simulation study I study whether the Gini variable importance is influenced by the variability of predictors, the precision of measuring them, and the variability of the error. Evidence of other biases in this variable importance was found. The results from the first simulation study were used to study whether genes related to 29 molecular biomarkers, which have been associated with schizophrenia, influence risk for schizophrenia in a case-control study of 26476 cases and 31804 controls from 39 different European ancestry cohorts. Single effects from ACAT2 and TNC genes were detected to contribute risk for schizophrenia. ACAT2 is a gene in the chromosome 6 which is related to energy metabolism. Transcriptional differences have been shown in schizophrenia brain tissue studies. TNC is expressed in the brain where is involved in the migration of the neurons and axons. In addition, we also used the simulation results to examine whether interactions between genes associated with abnormal emotion/affect behaviour influence risk for psychosis and cognition in humans, in a case-control study of 2049 cases and 1794 controls. Before correcting for multiple testing, significant interactions between CRHR1 and ESR1, and between MAPT and ESR1, and among CRHR1, ESR1 and TOM1L2, and among MAPT, ESR1 and TOM1L2 were observed in abnormal fear/anxiety-related behaviour pathway. There was no evidence for epistasis after Bonferroni correction.
|
730 |
Entity-based coherence in statistical machine translation : a modelling and evaluation perspectiveWetzel, Dominikus Emanuel January 2018 (has links)
Natural language documents exhibit coherence and cohesion by means of interrelated structures both within and across sentences. Sentences do not stand in isolation from each other and only a coherent structure makes them understandable and sound natural to humans. In Statistical Machine Translation (SMT) only little research exists on translating a document from a source language into a coherent document in the target language. The dominant paradigm is still one that considers sentences independently from each other. There is both a need for a deeper understanding of how to handle specific discourse phenomena, and for automatic evaluation of how well these phenomena are handled in SMT. In this thesis we explore an approach how to treat sentences as dependent on each other by focussing on the problem of pronoun translation as an instance of a discourse-related non-local phenomenon. We direct our attention to pronoun translation in the form of cross-lingual pronoun prediction (CLPP) and develop a model to tackle this problem. We obtain state-of-the-art results exhibiting the benefit of having access to the antecedent of a pronoun for predicting the right translation of that pronoun. Experiments also showed that features from the target side are more informative than features from the source side, confirming linguistic knowledge that referential pronouns need to agree in gender and number with their target-side antecedent. We show our approach to be applicable across the two language pairs English-French and English-German. The experimental setting for CLPP is artificially restricted, both to enable automatic evaluation and to provide a controlled environment. This is a limitation which does not yet allow us to test the full potential of CLPP systems within a more realistic setting that is closer to a full SMT scenario. We provide an annotation scheme, a tool and a corpus that enable evaluation of pronoun prediction in a more realistic setting. The annotated corpus consists of parallel documents translated by a state-of-the-art neural machine translation (NMT) system, where the appropriate target-side pronouns have been chosen by annotators. With this corpus, we exhibit a weakness of our current CLPP systems in that they are outperformed by a state-of-the-art NMT system in this more realistic context. This corpus provides a basis for future CLPP shared tasks and allows the research community to further understand and test their methods. The lack of appropriate evaluation metrics that explicitly capture non-local phenomena is one of the main reasons why handling non-local phenomena has not yet been widely adopted in SMT. To overcome this obstacle and evaluate the coherence of translated documents, we define a bilingual model of entity-based coherence, inspired by work on monolingual coherence modelling, and frame it as a learning-to-rank problem. We first evaluate this model on a corpus where we artificially introduce coherence errors based on typical errors CLPP systems make. This allows us to assess the quality of the model in a controlled environment with automatically provided gold coherence rankings. Results show that this model can distinguish with high accuracy between a human-authored translation and one with coherence errors, that it can also distinguish between document pairs from two corpora with different degrees of coherence errors, and that the learnt model can be successfully applied when the test set distribution of errors comes from a different one than the one from the training data, showing its generalization potentials. To test our bilingual model of coherence as a discourse-aware SMT evaluation metric, we apply it to more realistic data. We use it to evaluate a state-of-the-art NMT system against post-editing systems with pronouns corrected by our CLPP systems. For verifying our metric, we reuse our annotated parallel corpus and consider the pronoun annotations as proxy for human document-level coherence judgements. Experiments show far lower accuracy in ranking translations according to their entity-based coherence than on the artificial corpus, suggesting that the metric has difficulties generalizing to a more realistic setting. Analysis reveals that the system translations in our test corpus do not differ in their pronoun translations in almost half of the document pairs. To circumvent this data sparsity issue, and to remove the need for parameter learning, we define a score-based SMT evaluation metric which directly uses features from our bilingual coherence model.
|
Page generated in 0.0863 seconds