No description available.

Sequential supervised learning problems involve assigning a class label to each item in a sequence. Examples include part-of-speech tagging and text-to-speech mapping. A very general-purpose strategy for solving such problems is to construct a recurrent sliding window (RSW) classifier, which maps some window of the input sequence plus some number of previously-predicted items into a prediction for the next item in the sequence. This paper describes a general purpose implementation of RSW classifiers and discusses the highly practical issue of how to choose the size of the input window and the number of previous predictions to incorporate. Experiments on two real-world domains show that the optimal choices vary from one learning algorithm to another. They also depend on the evaluation criterion (number of correctly-predicted items versus number of correctly-predicted whole sequences). We conclude that window sizes must be chosen by cross-validation. The results have implications for the choice of window sizes for other models including hidden Markov models and conditional random fields. / Graduation date: 2004

Data can be represented in multiple views. Traditional multi-view learning methods (i.e., co-training, multi-task learning) focus on improving learning performance using information from the auxiliary view, although information from the target view is sufficient for learning task. However, this work addresses a semi-supervised case of multi-view learning, the surrogate supervision multi-view learning, where labels are available on limited views and a classifier is obtained on the target view where labels are missing. In surrogate multi-view learning, one cannot obtain a classifier without information from the auxiliary view. To solve this challenging problem, we propose discriminative and generative approaches. / Graduation date: 2013

L’objectif final étant de développer des modèles d’ordre réduit pour les applications de combustion, des techniques d’apprentissage automatique non supervisées et supervisées ont été testées et combinées dans les travaux de la présente thèse pour l’extraction de caractéristiques et la construction de modèles d’ordre réduit. Ainsi, l’application de techniques pilotées par les données pour la détection des caractéristiques d’ensembles de données de combustion turbulente (simulation numérique directe) a été étudiée sur deux flammes H2 / CO: une évolution spatiale (DNS1) et une jet à évolution temporelle (DNS2). Des méthodes telles que l’analyse en composantes principales (ACP), l’analyse en composantes principales locales (LPCA), la factorisation matricielle non négative (NMF) et les autoencodeurs ont été explorées à cette fin. Il a été démontré que divers facteurs pouvaient affecter les performances de ces méthodes, tels que les critères utilisés pour le centrage et la mise à l’échelle des données d’origine ou le choix du nombre de dimensions dans les approximations de rang inférieur. Un ensemble de lignes directrices a été présenté qui peut aider le processus d’identification de caractéristiques physiques significatives à partir de données de flux réactifs turbulents. Des méthodes de compression de données telles que l’analyse en composantes principales (ACP) et les variations ont été combinées à des méthodes d’interpolation telles que le krigeage, pour la construction de modèles ordonnées à prix réduits et calculables pour la prédiction de l’état d’un système de combustion dans des conditions de fonctionnement inconnues ou des combinaisons de modèles valeurs de paramètre d’entrée. La méthodologie a d’abord été testée pour la prévision des flammes 1D avec un nombre croissant de paramètres d’entrée (rapport d’équivalence, composition du carburant et température d’entrée), avec des variantes de l’approche PCA classique, à savoir PCA contrainte et PCA locale, appliquée aux cas de combustion la première fois en combinaison avec une technique d’interpolation. Les résultats positifs de l’étude ont conduit à l’application de la méthodologie proposée aux flammes 2D avec deux paramètres d’entrée, à savoir la composition du combustible et la vitesse d’entrée, qui ont donné des résultats satisfaisants. Des alternatives aux méthodes non supervisées et supervisées choisies ont également été testées sur les mêmes données 2D. L’utilisation de la factorisation matricielle non négative (FNM) pour l’approximation de bas rang a été étudiée en raison de la capacité de la méthode à représenter des données à valeur positive, ce qui permet de ne pas enfreindre des lois physiques importantes telles que la positivité des fractions de masse d’espèces chimiques et comparée à la PCA. Comme méthodes supervisées alternatives, la combinaison de l’expansion du chaos polynomial (PCE) et du Kriging et l’utilisation de réseaux de neurones artificiels (RNA) ont été testées. Les résultats des travaux susmentionnés ont ouvert la voie au développement d’un jumeau numérique d’un four à combustion à partir d’un ensemble de simulations 3D. La combinaison de PCA et de Kriging a également été utilisée dans le contexte de la quantification de l’incertitude (UQ), en particulier dans le cadre de collaboration de données lié (B2B-DC), qui a conduit à l’introduction de la procédure B2B-DC à commande réduite. Comme pour la première fois, le centre de distribution B2B a été développé en termes de variables latentes et non en termes de variables physiques originales. / With the final objective being to developreduced-order models for combustion applications,unsupervised and supervised machine learningtechniques were tested and combined in the workof the present Thesis for feature extraction and theconstruction of reduced-order models. Thus, the applicationof data-driven techniques for the detection offeatures from turbulent combustion data sets (directnumerical simulation) was investigated on two H2/COflames: a spatially-evolving (DNS1) and a temporallyevolvingjet (DNS2). Methods such as Principal ComponentAnalysis (PCA), Local Principal ComponentAnalysis (LPCA), Non-negative Matrix Factorization(NMF) and Autoencoders were explored for this purpose.It was shown that various factors could affectthe performance of these methods, such as the criteriaemployed for the centering and the scaling of theoriginal data or the choice of the number of dimensionsin the low-rank approximations. A set of guidelineswas presented that can aid the process ofidentifying meaningful physical features from turbulentreactive flows data. Data compression methods suchas Principal Component Analysis (PCA) and variationswere combined with interpolation methods suchas Kriging, for the construction of computationally affordablereduced-order models for the prediction ofthe state of a combustion system for unseen operatingconditions or combinations of model input parametervalues. The methodology was first tested forthe prediction of 1D flames with an increasing numberof input parameters (equivalence ratio, fuel compositionand inlet temperature), with variations of the classicPCA approach, namely constrained PCA and localPCA, being applied to combustion cases for the firsttime in combination with an interpolation technique.The positive outcome of the study led to the applicationof the proposed methodology to 2D flames withtwo input parameters, namely fuel composition andinlet velocity, which produced satisfactory results. Alternativesto the chosen unsupervised and supervisedmethods were also tested on the same 2D data.The use of non-negative matrix factorization (NMF) forlow-rank approximation was investigated because ofthe ability of the method to represent positive-valueddata, which helps the non-violation of important physicallaws such as positivity of chemical species massfractions, and compared to PCA. As alternative supervisedmethods, the combination of polynomial chaosexpansion (PCE) and Kriging and the use of artificialneural networks (ANNs) were tested. Results from thementioned work paved the way for the developmentof a digital twin of a combustion furnace from a setof 3D simulations. The combination of PCA and Krigingwas also employed in the context of uncertaintyquantification (UQ), specifically in the bound-to-bounddata collaboration framework (B2B-DC), which led tothe introduction of the reduced-order B2B-DC procedureas for the first time the B2B-DC was developedin terms of latent variables and not in terms of originalphysical variables.

This thesis is an investigation into Supervised Learning (SL) in Genetic Programming (GP). With its flexible tree-structured representation, GP is a type of Genetic Algorithm, using the Darwinian idea of natural selection and genetic recombination, evolving populations of solutions over many generations to solve problems. SL is a common approach in Machine Learning where the problem is presented as a set of examples. A good or fit solution is one which can successfully deal with all of the examples. In common with most Machine Learning approaches, GP has been used to solve many trivial problems. When applied to larger and more complex problems, however, several difficulties become apparent. When focusing on the basic features of GP, this thesis highlights the immense size of the GP search space, and describes an approach to measure this space. A stupendously flexible but frustratingly useless representation, Anarchically Automatically Defined Functions, is described. Some difficulties associated with the normal use of the GP operator Crossover (perhaps the most common method of combining GP trees to produce new trees) are demonstrated in the simple MAX problem. Crossover can lead to irreversible sub-optimal GP performance when used in combination with a restriction on tree size. There is a brief study of tournament selection which is a common method of selecting fit individuals from a GP population to act as parents in the construction of the next generation. The main contributions of this thesis however are two approaches for avoiding the fitness evaluation bottleneck resulting from the use of SL in GP. to establish the capability of a GP individual using SL, it must be tested or evaluated against each example in the set of training examples.

Recent work has shown that AdaBoost can be viewed as an algorithm that maximizes the margin on the training data via functional gradient descent. Under this interpretation, the weight computed by AdaBoost, for each hypothesis generated, can be viewed as a step size parameter in a gradient descent search. Friedman has suggested that shrinking these step sizes could produce improved generalization and reduce overfitting. In a series of experiments, he showed that very small step sizes did indeed reduce overfitting and improve generalization for three variants of Gradient_Boost, his generic functional gradient descent algorithm. For this report, we tested whether reduced learning rates can also improve generalization in AdaBoost. We tested AdaBoost (applied to C4.5 decision trees) with reduced learning rates on 28 benchmark datasets. The results show that reduced learning rates provide no statistically significant improvement on these datasets. We conclude that reduced learning rates cannot be recommended for use with boosted decision trees on datasets similar to these benchmark datasets. / Graduation date: 2002

Thesis (M.S.)--Oregon State University, 2006. / Printout. Includes bibliographical references (leaves 42-46). Also available on the World Wide Web.

Examples are a powerful tool for teaching both humans and computers.In order to learn from examples, however, a student must first extractthe examples from its stream of perception. Snapshot learning is ageneral approach to this problem, in which relevant samples ofperception are used as examples. Learning from these examples can inturn improve the judgement of the snapshot mechanism, improving thequality of future examples. One way to implement snapshot learning isthe Top-Cliff heuristic, which identifies relevant samples using ageneralized notion of peaks. I apply snapshot learning with theTop-Cliff heuristic to solve a distributed learning problem and showthat the resulting system learns rapidly and robustly, and canhallucinate useful examples in a perceptual stream from a teacherlesssystem.

There is an increasing need of automation for routine tasks like sorting agricultural produce in large scale post-harvest processing. Among different kinds of sensors used for such automation tasks, near-infrared (NIR) technology provides a rapid and effective solution for quantitative analysis of quality indices in food products. As industries and farms are adopting modern data-driven technologies, there is a need for evaluation of the modelling tools to find the optimal solutions for problem solving. This study aims to understand the process of evaluation of the modelling tools, in view of near-infrared data obtained from green leafy vegetables. The first part of this study deals with prediction of the type of leafy green vegetable from the near-infrared reflectance spectra non-destructively taken from the leaf surface. Supervised classification methods used for the classification task were k-nearest neighbors (KNN), support vector machines (SVM), linear discriminant analysis (LDA) classifier, regularized discriminant analysis (RDA) classifier, naïve Bayes classifier, bagged trees, random forests, and ensemble discriminant subspace classifier. The second part of this study deals with prediction of total glucosinolate and total polyphenol contents in leaves using Partial Least Squares Regression (PLSR) and Principal Component Regression (PCR). Optimal combination of predictors were chosen by using recursive feature elimination. NIR spectra taken from 283 different samples were used for classification task. Accuracy rates of tuned classifiers were compared for a standard test set. The ensemble discriminant subspace classifier was found to yield the highest accuracy rates (89.41%) for the standard test set. Classifiers were also compared in terms of accuracy rates and F1 scores. Learning rates of classifiers were compared with cross-validation accuracy rates for different proportions of dataset. Ensemble subspace discriminants, SVM, LDA and KNN were found to be similar in their cross-validation accuracy rates for different proportions of data. NIR spectra as well as reference values for total polyphenol content and total glucosinolate contents were taken from 40 samples for each analyses. PLSR model for total glucosinolate prediction built with spectra treated with Savitzky-Golay second derivative yielded a RMSECV of 0.67 μmol/g of fresh weight and cross-validation R2 value of 0.63. Similarly, PLSR model built with spectra treated with Savitzky-Golay first derivative yielded a RMSECV of 6.56 Gallic Acid Equivalent (GAE) mg/100g of fresh weight and cross-validation R-squared value of 0.74. Feature selection for total polyphenol prediction suggested that the region of NIR between 1300 - 1600 nm might contain important information about total polyphenol content in the green leaves.

In the first part, we deal with the unlabeled data that are in good quality and follow the conditions of semi-supervised learning. Firstly, we present a novel method for Transductive Support Vector Machine (TSVM) by relaxing the unknown labels to the continuous variables and reducing the non-convex optimization problem to a convex semi-definite programming problem. In contrast to the previous relaxation method which involves O (n2) free parameters in the semi-definite matrix, our method takes advantage of reducing the number of free parameters to O (n), so that we can solve the optimization problem more efficiently. In addition, the proposed approach provides a tighter convex relaxation for the optimization problem in TSVM. Empirical studies on benchmark data sets demonstrate that the proposed method is more efficient than the previous semi-definite relaxation method and achieves promising classification results comparing with the state-of-the-art methods. Our second contribution is an extended level method proposed to efficiently solve the multiple kernel learning (MKL) problems. In particular, the level method overcomes the drawbacks of both the Semi-Infinite Linear Programming (SILP) method and the Subgradient Descent (SD) method for multiple kernel learning. Our experimental results show that the level method is able to greatly reduce the computational time of MKL over both the SD method and the SILP method. Thirdly, we discuss the connection between two fundamental assumptions in semi-supervised learning. More specifically, we show that the loss on the unlabeled data used by TSVM can be essentially viewed as an additional regularizer for the decision boundary. We further show that this additional regularizer induced by the TSVM is closely related to the regularizer introduced by the manifold regularization. Both of them can be viewed as a unified regularization framework for semi-supervised learning. / In the second part, we discuss how to employ the unlabeled data for building reliable classification systems in three scenarios: (1) only poorly-related unlabeled data are available, (2) good quality unlabeled data are mixed with irrelevant data and there are no prior knowledge on their composition, and (3) no unlabeled data are available but can be achieved from the Internet for text categorization. We build several frameworks to deal with the above cases. Firstly, we present a study on how to deal with the weakly-related unlabeled data, called the Supervised Self-taught Learning framework, which can transfer knowledge from the unlabeled data actively. The proposed model is able to select those discriminative features or representations, which are more appropriate for classification. Secondly, we also propose a novel framework that can learn from a mixture of unlabeled data, where good quality unlabeled data are mixed with unlabeled irrelevant samples. Moreover, we do not need the prior knowledge on which data samples are relevant or irrelevant. Consequently it is significantly different from the recent framework of semi-supervised learning with universum and the framework of Universum Support Vector Machine. As an important contribution, we have successfully formulated this new learning approach as a Semi-definite Programming problem, which can be solved in polynomial time. A series of experiments demonstrate that this novel framework has advantages over the semi-supervised learning on both synthetic and real data in many facets. Finally, for third scenario, we present a general framework for semi-supervised text categorization that collects the unlabeled documents via Web search engines and utilizes them to improve the accuracy of supervised text categorization. Extensive experiments have demonstrated that the proposed semi-supervised text categorization framework can significantly improve the classification accuracy. Specifically, the classification error is reduced by 30% averaged on the nine data sets when using Google as the search engine. / We consider the problem of learning from both labeled and unlabeled data through the analysis on the quality of the unlabeled data. Usually, learning from both labeled and unlabeled data is regarded as semi-supervised learning, where the unlabeled data and the labeled data are assumed to be generated from the same distribution. When this assumption is not satisfied, new learning paradigms are needed in order to effectively explore the information underneath the unlabeled data. This thesis consists of two parts: the first part analyzes the fundamental assumptions of semi-supervised learning and proposes a few efficient semi-supervised learning models; the second part discusses three learning frameworks in order to deal with the case that unlabeled data do not satisfy the conditions of semi-supervised learning. / Xu, Zenglin. / Advisers: Irwin King; Michael R. Lyu. / Source: Dissertation Abstracts International, Volume: 70-09, Section: B, page: . / Thesis (Ph.D.)--Chinese University of Hong Kong, 2009. / Includes bibliographical references (leaves 158-179). / Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Electronic reproduction. [Ann Arbor, MI] : ProQuest Information and Learning, [200-] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Abstracts in English and Chinese. / School code: 1307.