Typescript, etc. / Digitized by Kansas State University Libraries

No description available.

Sequential supervised learning problems involve assigning a class label to each item in a sequence. Examples include part-of-speech tagging and text-to-speech mapping. A very general-purpose strategy for solving such problems is to construct a recurrent sliding window (RSW) classifier, which maps some window of the input sequence plus some number of previously-predicted items into a prediction for the next item in the sequence. This paper describes a general purpose implementation of RSW classifiers and discusses the highly practical issue of how to choose the size of the input window and the number of previous predictions to incorporate. Experiments on two real-world domains show that the optimal choices vary from one learning algorithm to another. They also depend on the evaluation criterion (number of correctly-predicted items versus number of correctly-predicted whole sequences). We conclude that window sizes must be chosen by cross-validation. The results have implications for the choice of window sizes for other models including hidden Markov models and conditional random fields. / Graduation date: 2004

No description available.

In this work, we present an unsupervised representation learning approach for learning rich spatiotemporal features from videos without the supervision from semantic labels. We propose to learn the spatiotemporal features by training a 3D convolutional neural network (CNN) using video completion as a surrogate task. Using a large collection of unlabeled videos, we train the CNN to predict the missing pixels of a spatiotemporal hole given the remaining parts of the video through minimizing per-pixel reconstruction loss. To achieve good reconstruction results using color videos, the CNN needs to have a certain level of understanding of the scene dynamics and predict plausible, temporally coherent contents. We further explore to jointly reconstruct both color frames and flow fields. By exploiting the statistical temporal structure of images, we show that the learned representations capture meaningful spatiotemporal structures from raw videos. We validate the effectiveness of our approach for CNN pre-training on action recognition and action similarity labeling problems. Our quantitative results demonstrate that our method compares favorably against learning without external data and existing unsupervised learning approaches. / Master of Science / The current supervised representation learning methods leverage large datasets of millions of labeled examples to learn semantically meaningful visual representations. Thousands of boring human hours are spent on manually labeling these datasets. But, do we need semantically labeled images to learn good visual representation? Humans learn visual representations using little or no semantic supervision but the existing approaches are mostly supervised. In this work, we propose an unsupervised visual representation learning algorithm to learn useful spatiotemporal features by formulating a video completion problem. To predict the missing pixels of the video, the model needs to have a high-level semantic understanding and motion patterns of people and objects. We demonstrate that video completion task effectively learns semantically meaningful spatiotemporal features from raw natural videos without semantic labels. The learned representation provide a good network weight initialization for applications with few training examples. We show significant performance gain over training the model from scratch and demonstrate improved performance in action recognition and action similarity labeling tasks when compared with competitive unsupervised learning algorithms.

Data can be represented in multiple views. Traditional multi-view learning methods (i.e., co-training, multi-task learning) focus on improving learning performance using information from the auxiliary view, although information from the target view is sufficient for learning task. However, this work addresses a semi-supervised case of multi-view learning, the surrogate supervision multi-view learning, where labels are available on limited views and a classifier is obtained on the target view where labels are missing. In surrogate multi-view learning, one cannot obtain a classifier without information from the auxiliary view. To solve this challenging problem, we propose discriminative and generative approaches. / Graduation date: 2013

The final thesis summarizes in its theoretical part basic knowledge of machine learning algorithms that involves supervised, semi-supervised, and unsupervised learning. Experiments with textual data in natural spoken language involving different machine learning methods and parameterization are carried out in its practical part. Conclusions made in the thesis may be of use to individuals that are at least slightly interested in this domain.

L’objectif final étant de développer des modèles d’ordre réduit pour les applications de combustion, des techniques d’apprentissage automatique non supervisées et supervisées ont été testées et combinées dans les travaux de la présente thèse pour l’extraction de caractéristiques et la construction de modèles d’ordre réduit. Ainsi, l’application de techniques pilotées par les données pour la détection des caractéristiques d’ensembles de données de combustion turbulente (simulation numérique directe) a été étudiée sur deux flammes H2 / CO: une évolution spatiale (DNS1) et une jet à évolution temporelle (DNS2). Des méthodes telles que l’analyse en composantes principales (ACP), l’analyse en composantes principales locales (LPCA), la factorisation matricielle non négative (NMF) et les autoencodeurs ont été explorées à cette fin. Il a été démontré que divers facteurs pouvaient affecter les performances de ces méthodes, tels que les critères utilisés pour le centrage et la mise à l’échelle des données d’origine ou le choix du nombre de dimensions dans les approximations de rang inférieur. Un ensemble de lignes directrices a été présenté qui peut aider le processus d’identification de caractéristiques physiques significatives à partir de données de flux réactifs turbulents. Des méthodes de compression de données telles que l’analyse en composantes principales (ACP) et les variations ont été combinées à des méthodes d’interpolation telles que le krigeage, pour la construction de modèles ordonnées à prix réduits et calculables pour la prédiction de l’état d’un système de combustion dans des conditions de fonctionnement inconnues ou des combinaisons de modèles valeurs de paramètre d’entrée. La méthodologie a d’abord été testée pour la prévision des flammes 1D avec un nombre croissant de paramètres d’entrée (rapport d’équivalence, composition du carburant et température d’entrée), avec des variantes de l’approche PCA classique, à savoir PCA contrainte et PCA locale, appliquée aux cas de combustion la première fois en combinaison avec une technique d’interpolation. Les résultats positifs de l’étude ont conduit à l’application de la méthodologie proposée aux flammes 2D avec deux paramètres d’entrée, à savoir la composition du combustible et la vitesse d’entrée, qui ont donné des résultats satisfaisants. Des alternatives aux méthodes non supervisées et supervisées choisies ont également été testées sur les mêmes données 2D. L’utilisation de la factorisation matricielle non négative (FNM) pour l’approximation de bas rang a été étudiée en raison de la capacité de la méthode à représenter des données à valeur positive, ce qui permet de ne pas enfreindre des lois physiques importantes telles que la positivité des fractions de masse d’espèces chimiques et comparée à la PCA. Comme méthodes supervisées alternatives, la combinaison de l’expansion du chaos polynomial (PCE) et du Kriging et l’utilisation de réseaux de neurones artificiels (RNA) ont été testées. Les résultats des travaux susmentionnés ont ouvert la voie au développement d’un jumeau numérique d’un four à combustion à partir d’un ensemble de simulations 3D. La combinaison de PCA et de Kriging a également été utilisée dans le contexte de la quantification de l’incertitude (UQ), en particulier dans le cadre de collaboration de données lié (B2B-DC), qui a conduit à l’introduction de la procédure B2B-DC à commande réduite. Comme pour la première fois, le centre de distribution B2B a été développé en termes de variables latentes et non en termes de variables physiques originales. / With the final objective being to developreduced-order models for combustion applications,unsupervised and supervised machine learningtechniques were tested and combined in the workof the present Thesis for feature extraction and theconstruction of reduced-order models. Thus, the applicationof data-driven techniques for the detection offeatures from turbulent combustion data sets (directnumerical simulation) was investigated on two H2/COflames: a spatially-evolving (DNS1) and a temporallyevolvingjet (DNS2). Methods such as Principal ComponentAnalysis (PCA), Local Principal ComponentAnalysis (LPCA), Non-negative Matrix Factorization(NMF) and Autoencoders were explored for this purpose.It was shown that various factors could affectthe performance of these methods, such as the criteriaemployed for the centering and the scaling of theoriginal data or the choice of the number of dimensionsin the low-rank approximations. A set of guidelineswas presented that can aid the process ofidentifying meaningful physical features from turbulentreactive flows data. Data compression methods suchas Principal Component Analysis (PCA) and variationswere combined with interpolation methods suchas Kriging, for the construction of computationally affordablereduced-order models for the prediction ofthe state of a combustion system for unseen operatingconditions or combinations of model input parametervalues. The methodology was first tested forthe prediction of 1D flames with an increasing numberof input parameters (equivalence ratio, fuel compositionand inlet temperature), with variations of the classicPCA approach, namely constrained PCA and localPCA, being applied to combustion cases for the firsttime in combination with an interpolation technique.The positive outcome of the study led to the applicationof the proposed methodology to 2D flames withtwo input parameters, namely fuel composition andinlet velocity, which produced satisfactory results. Alternativesto the chosen unsupervised and supervisedmethods were also tested on the same 2D data.The use of non-negative matrix factorization (NMF) forlow-rank approximation was investigated because ofthe ability of the method to represent positive-valueddata, which helps the non-violation of important physicallaws such as positivity of chemical species massfractions, and compared to PCA. As alternative supervisedmethods, the combination of polynomial chaosexpansion (PCE) and Kriging and the use of artificialneural networks (ANNs) were tested. Results from thementioned work paved the way for the developmentof a digital twin of a combustion furnace from a setof 3D simulations. The combination of PCA and Krigingwas also employed in the context of uncertaintyquantification (UQ), specifically in the bound-to-bounddata collaboration framework (B2B-DC), which led tothe introduction of the reduced-order B2B-DC procedureas for the first time the B2B-DC was developedin terms of latent variables and not in terms of originalphysical variables.

This thesis is an investigation into Supervised Learning (SL) in Genetic Programming (GP). With its flexible tree-structured representation, GP is a type of Genetic Algorithm, using the Darwinian idea of natural selection and genetic recombination, evolving populations of solutions over many generations to solve problems. SL is a common approach in Machine Learning where the problem is presented as a set of examples. A good or fit solution is one which can successfully deal with all of the examples. In common with most Machine Learning approaches, GP has been used to solve many trivial problems. When applied to larger and more complex problems, however, several difficulties become apparent. When focusing on the basic features of GP, this thesis highlights the immense size of the GP search space, and describes an approach to measure this space. A stupendously flexible but frustratingly useless representation, Anarchically Automatically Defined Functions, is described. Some difficulties associated with the normal use of the GP operator Crossover (perhaps the most common method of combining GP trees to produce new trees) are demonstrated in the simple MAX problem. Crossover can lead to irreversible sub-optimal GP performance when used in combination with a restriction on tree size. There is a brief study of tournament selection which is a common method of selecting fit individuals from a GP population to act as parents in the construction of the next generation. The main contributions of this thesis however are two approaches for avoiding the fitness evaluation bottleneck resulting from the use of SL in GP. to establish the capability of a GP individual using SL, it must be tested or evaluated against each example in the set of training examples.

Recent work has shown that AdaBoost can be viewed as an algorithm that maximizes the margin on the training data via functional gradient descent. Under this interpretation, the weight computed by AdaBoost, for each hypothesis generated, can be viewed as a step size parameter in a gradient descent search. Friedman has suggested that shrinking these step sizes could produce improved generalization and reduce overfitting. In a series of experiments, he showed that very small step sizes did indeed reduce overfitting and improve generalization for three variants of Gradient_Boost, his generic functional gradient descent algorithm. For this report, we tested whether reduced learning rates can also improve generalization in AdaBoost. We tested AdaBoost (applied to C4.5 decision trees) with reduced learning rates on 28 benchmark datasets. The results show that reduced learning rates provide no statistically significant improvement on these datasets. We conclude that reduced learning rates cannot be recommended for use with boosted decision trees on datasets similar to these benchmark datasets. / Graduation date: 2002