Doctor de Sistemas de Ingeniería / Recientemente los datos se han incrementado en todas las áreas del conocimiento, tanto en el número de instancias como en el de atributos. Bases de datos actuales pueden contar con decenas e incluso cientos de miles de variables con un alto grado de información tanto irrelevante como redundante. Esta gran cantidad de datos causa serios problemas a muchos algoritmos de minería de datos en términos de escalabilidad y rendimiento. Dentro de las áreas de investigación en selección de atributos se incluyen el análisis de chips de ADN, procesamiento de documentos provenientes de internet y modelos de administración de riesgo en el sector financiero. El objetivo de esta tarea es triple: mejorar el desempeño predictivo de los modelos, implementar soluciones más rápidas y menos costosas, y proveer de un mejor entendimiento del proceso subyacente que generó los datos. Dentro de las técnicas de minería de datos, el método llamado Support Vector Machines (SVMs) ha ganado popularidad gracias a su capacidad de generalización frente a nuevos objetos y de construir complejas funciones no lineales. Estas características permiten obtener mejores resultados que otros métodos predictivos. Sin embargo, una limitación de este método es que no está diseñado para identificar los atributos importantes para construir la regla discriminante. El presente trabajo tiene como objetivo desarrollar técnicas que permitan incorporar la selección de atributos en la formulación de SVMs no lineal, aportando eficiencia y comprensibilidad al método. Se desarrollaron dos metodologías: un algoritmo wrapper (HO-SVM) que utiliza el número de errores en un conjunto de validación como medida para decidir qué atributo eliminar en cada iteración, y un método embedded (KP-SVM) que optimiza la forma de un kernel Gaussiano no isotrópico, penalizando la utilización de atributos en la función de clasificación. Los algoritmos propuestos fueron probados en bases de datos de de diversa dimensionalidad, que van desde decenas a miles de atributos, y en problemas reales de asignación de créditos para entidades financieras nacionales. De los resultados se obtiene que SVMs no lineal con kernel Gaussiano muestra un mejor desempeño que con las funciones de kernel lineal y polinomial. Asimismo, los métodos de selección de atributos propuestos permiten mantener o incluso mejorar el desempeño predictivo de SVMs no lineal, logrando además una reducción significativa en la utilización de atributos. Para las bases de mayor dimensionalidad se reduce de miles a decenas de atributos seleccionados, logrando un desempeño predictivo significativamente mejor que los enfoques alternativos de selección de atributos para SVMs. Se concluye que los enfoques presentados representan la alternativa más efectiva dentro de las estudiadas para resolver el problema de selección de atributos en modelos de aprendizaje computacional. Como trabajo futuro se propone adaptar las metodologías propuestas para problemas con desbalance de clases, donde se requiere una evaluación distinta del desempeño del modelo considerando costos por error de clasificación asimétricos, una problemática común en aplicaciones como detección de fuga y riesgo crediticio.

Falls and fall-induced fractures are very common among the elderly. Hip fractures account for most of the deaths and costs of all the fall-induced fractures. This dissertation presents a novel MEMS based human airbag system used as a hip protector. A Micro Inertial Measurement Unit (muIMU) which is based on MEMS accelerometers and gyro sensors is developed as the motion sensing part of the system. The result using this muIMU based on Support Vector Machine (SVM) training to recognize falling-motions are presented, where we showed that selected eigenvector sets generated from 200 experimental data can be separated into falling and other motions completely. For real-time recognition, the SVM filter should be embedded to a high speed DSP system for fast computation and complex filter analyses. After the simulations for SVM filter and FFT were performed on a computer simulator (TI DSP320 C6713), we used DSK6713 (DSP Starter Kit) as our target board and integrated FFT and SVM filter on the chip. The whole algorithm works well with exist sensor data. Demo shows that our DSP system can successfully classify fall and non-fall states. At the same time, the system can trigger our airbag inflation mechanism when a fall occurs. The system was shown to open the airbag in real-time and protected the experimenter's hip area. / by Shi, Guangyi. / "March 2008." / Adviser: Wen Jung Li. / Source: Dissertation Abstracts International, Volume: 70-03, Section: B, page: 1855. / Thesis (Ph.D.)--Chinese University of Hong Kong, 2008. / Includes bibliographical references (p. 108-111). / Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Electronic reproduction. [Ann Arbor, MI] : ProQuest Information and Learning, [200-] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Abstracts in English and Chinese. / School code: 1307.

Regularization is a dominant theme in machine learning and statistics due to its prominent ability in providing an intuitive and principled tool for learning from high-dimensional data. As large-scale learning applications become popular, developing efficient algorithms and parsimonious models become promising and necessary for these applications. Aiming at solving large-scale learning problems, this thesis tackles the key research problems ranging from feature selection to learning with unlabeled data and learning data similarity representation. More specifically, we focus on the problems in three areas: online learning, semi-supervised learning, and multiple kernel learning. / The first part of this thesis develops a novel online learning framework to solve group lasso and multi-task feature selection. To the best our knowledge, the proposed online learning framework is the first framework for the corresponding models. The main advantages of the online learning algorithms are that (1) they can work on the applications where training data appear sequentially; consequently, the training procedure can be started at any time; (2) they can handle data up to any size with any number of features. The efficiency of the algorithms is attained because we derive closed-form solutions to update the weights of the corresponding models. At each iteration, the online learning algorithms just need O (d) time complexity and memory cost for group lasso, while they need O (d x Q) for multi-task feature selection, where d is the number of dimensions and Q is the number of tasks. Moreover, we provide theoretical analysis for the average regret of the online learning algorithms, which also guarantees the convergence rate of the algorithms. In addition, we extend the online learning framework to solve several related models which yield more sparse solutions. / The second part of this thesis addresses a general scenario of semi-supervised learning for the binary classification problern, where the unlabeled data may be a mixture of relevant and irrelevant data to the target binary classification task. Without specifying the relatedness in the unlabeled data, we develop a novel maximum margin classifier, named the tri-class support vector machine (3C-SVM), to seek an inductive rule that can separate these data into three categories: --1, +1, or 0. This is achieved by adopting a novel min loss function and following the maximum entropy principle. For the implementation, we approximate the problem and solve it by a standard concaveconvex procedure (CCCP). The approach is very efficient and it is possible to solve large-scale datasets. / The third part of this thesis focuses on multiple kernel learning (MKL) to solve the insufficiency of the L1-MKL and the Lp-MKL models. Hence, we propose a generalized MKL (GMKL) model by introducing an elastic net-type constraint on the kernel weights. More specifically, it is an MKL model with a constraint on a linear combination of the L1-norm and the square of the L2-norm on the kernel weights to seek the optimal kernel combination weights. Therefore, previous MKL problems based on the L1-norm or the L2-norm constraints can be regarded as its special cases. Moreover, our GMKL enjoys the favorable sparsity property on the solution and also facilitates the grouping effect. In addition, the optimization of our GMKL is a convex optimization problem, where a local solution is the globally optimal solution. We further derive the level method to efficiently solve the optimization problem. / Yang, Haiqin. / Advisers: Kuo Chin Irwin King; Michael Rung Tsong Iyu. / Source: Dissertation Abstracts International, Volume: 73-04, Section: B, page: . / Thesis (Ph.D.)--Chinese University of Hong Kong, 2011. / Includes bibliographical references (leaves 152-173). / Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Electronic reproduction. [Ann Arbor, MI] : ProQuest Information and Learning, [201-] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Abstract also in Chinese.

Orientador: João Ricardo Sato / Dissertação (mestrado) - Universidade Federal do ABC. Programa de Pós-Graduação em Neurociência e Cognição, 2013

Master of Science / Department of Computer Science / William H. Hsu / We present gestural music instruments and interfaces that aid musicians and audio engineers to express themselves efficiently. While we have mastered building a wide variety of physical instruments, the quest for virtual instruments and sound synthesis is on the rise. Virtual instruments are essentially software that enable musicians to interact with a sound module in the computer. Since the invention of MIDI (Musical Instrument Digital Interface), devices and interfaces to interact with sound modules like keyboards, drum machines, joysticks, mixing and mastering systems have been flooding the music industry. Research in the past decade gone one step further in interacting through simple musical gestures to create, shape and arrange music in real time. Machine learning is a powerful tool that can be smartly used to teach simple gestures to the interface. The ability to teach innovative gestures and shape the way a sound module behaves unleashes the untapped creativity of an artist. Timed music and multimedia programs such as Max/MSP/Jitter along with machine learning techniques open gateways to embodied musical experiences without physical touch. This master's report presents my research, observations and how this interdisciplinary field of research could be used to study wider neuroscience problems like embodied music cognition and human-computer interactions.

In this study, a Prediction Accuracy Based Hill Climbing Feature Selection Algorithm (AHCFS) is created and compared with an Error Rate Based Sequential Feature Selection Algorithm (ERFS) which is an existing Matlab algorithm. The goal of the study is to create a new piece of an algorithm that has potential to outperform the existing Matlab sequential feature selection algorithm in predicting the movement of S&P 500 (^GSPC) prices under certain circumstances. The two algorithms are tested based on historical data of ^GSPC, and Support Vector Machine (SVM) is employed by both as the classifier. A prediction without feature selection algorithm implemented is carried out and used as a baseline for comparison between the two algorithms. The prediction horizon set in this study for both algorithms varies from one to 60 days. The study results show that AHCFS reaches higher prediction accuracy than ERFS in the majority of the cases.

The objective of this study is to propose an efficient sampling-based RBDO using a new classification method to reduce the computational cost. In addition, accuracy improvement strategies for the Kriging method are proposed to reduce the number of expensive computer experiments. Current research effort involves: (1) developing a new classification method that is more efficient than conventional surrogate modeling methods while maintaining required accuracy level; (2) developing a sequential adaptive sampling method that inserts samples near the limit state function; (3) improving the efficiency of the RBDO process by using a fixed hyper-spherical local window with an efficient uniform sampling method and identification of active/violated constraints; and (4) improving the accuracy of the Kriging method by introducing several strategies. In the sampling-based RBDO, only accurate classification information is needed instead of accurate response surface. On the other hand, in general, surrogates are constructed using all available DoE samples instead of focusing on the limit state function. Therefore, the computational cost of surrogates can be relatively expensive; and the accuracy of the limit state (or decision) function can be sacrificed in return for reducing the error on unnecessary regions away from the limit state function. On the contrary, the support vector machine (SVM), which is a classification method, only uses support vectors, which are located near the limit state function, to focus on the decision function. Therefore, the SVM is very efficient and ideally applicable to sampling-based RBDO, if the accuracy of SVM is improved by inserting virtual samples near the limit state function. The proposed sequential sampling method inserts new samples near the limit state function so that the number of DoE samples is minimized. In many engineering problems, expensive computer simulations are used and thus the total computational cost needs to be reduced by using less number of DoE samples. Several efficiency strategies such as: (1) launching RBDO at a deterministic optimum design, (2) hyper-spherical local windows with an efficient uniform sampling method, (3) filtering of constraints, (4) sample reuse, (5) improved virtual sample generation, are used for the proposed sampling-based RBDO using virtual SVM. The number of computer experiments is also reduced by implementing accuracy improvement strategies for the Kriging method. Since the Kriging method is used for generating virtual samples and generating response surface of the cost function, the number of computer experiments can be reduced by introducing: (1) accurate correlation parameter estimation, (2) penalized maximum likelihood estimation (PMLE) for small sample size, (3) correlation model selection by MLE, and (4) mean structure selection by cross-validation (CV) error.

Les compartiments cellulaires, de par les frontières membranaires qui les définissent, permettent l'accomplissement de taches métaboliques diverses au sein de la cellule. Cette spécialisation en domaines intracellulaires induit donc une différentiation dans la fonction des protéines qui les composent. Le grand nombre de gènes orphelins produits ces dernières années par les projets de séquençage motive la mise au point de méthodes efficaces pour la prédiction ab-initio de la localisation cellulaire des protéines.<br /><br />Ainsi la majorité de ce travail de thèse s'intéresse au problème de la prédiction du compartiment cellulaire d'une protéine à partir de sa séquence primaire.<br /><br />Nous nous sommes attachés à proposer des alternatives descriptives aux méthodes existantes de prédiction de la localisation cellulaire en utilisant : (1) de nouveaux descripteurs issus de la séquence nucléique, (2) une approche par chaînes de Markov cachées (CMC) et arbres de décision. L'approche par CMC est justifiée biologiquement a posteriori car elle permet la modélisation de signaux d'adressage conjointement à la prise en compte de la composition globale. En outre, l'étape de classification hiérarchique par arbre améliore nettement les résultats de classification. Les résultats obtenues lors des comparaisons avec les méthodes existantes et utilisant des descripteurs fondés sur la composition globale possèdent des performances similaires.

Les SVM (Support Vector Machines) sont de nouvelles techniques d'apprentissage statistique proposées par V.Vapnik en 1995. Elles permettent d'aborder des problèmes très divers comme le classement, la régression, la fusion, etc... Depuis leur introduction dans le domaine de la Reconnaissance de Formes (RdF), plusieurs travaux ont pu montrer l'efficacité de ces techniques principalement en traitement d'image. L'idée essentielle des SVM consiste à projeter les données de l'espace d'entrée (appartenant à deux classes différentes) non-linéairement séparables dans un espace de plus grande dimension appelé espace de caractéristiques de façon à ce que les données deviennent linéairement séparables. Dans cet espace, la technique de construction de l'hyperplan optimal est utilisée pour calculer la fonction de classement séparant les deux classes. Dans ce travail de thèse, nous avons étudié les SVM comme techniques de classement pour la Vérification Automatique du Locuteur (VAL) en mode dépendant et indépendant du texte. Nous avons également étudié les SVM pour des tâches de fusion en réalisant des expériences concernant deux types de fusion, la fusion de méthodes et la fusion de modes. Dans le cadre du projet PICASSO, nous avons proposé un système de VAL en mode dépendant du texte utilisant les SVM dans une application de mots de passe publics. Dans ce système, une nouvelle modélisation basée sur la transcription phonétique des mots de passe a été proposée pour construire les vecteurs d'entrée pour notre classifieur SVM. En ce qui concerne notre étude des SVM en VAL en mode indépendant du texte, nous avons proposé des systèmes hybrides GMM-SVM. Dans ces systèmes, trois nouvelles représentations de données ont été proposées permettant de réunir l'efficacité des GMM en modélisation et les performances des SVM en décision. Ce travail entre dans le cadre de nos participations aux évaluations internationales NIST. Dans le cadre du projet BIOMET sur l'authentification biométrique mené par le GET (Groupe des Écoles de Télécommunications), nous avons étudié les SVM pour deux tâches de fusion. La première concerne la fusion de méthodes où nous avons fusionné les scores obtenus par les participants à la tâche ``One Speaker Detection'' aux évaluations NIST'2001. La seconde concerne la fusion de modes menée sur les scores obtenus sur les quatre différentes modalités de la base de données M2VTS. Les études que nous avons réalisées représentent une des premières tentatives d'appliquer les SVM dans le domaine de la VAL. Les résultats obtenus montrent que les SVM sont des techniques très efficaces et surtout très prometteuses que ce soit pour le classement ou la fusion.

<p>The subject of this thesis is the application of Support Vector Machines on two totally different applications, facial expressions recognition and remote sensing.</p><p>The basic idea of kernel algorithms is to transpose input data in a higher dimensional space, the feature space, in which linear operations on the data can be processed more easily. These operations in the feature space can be expressed in terms of input data thanks to the kernel functions. Support Vector Machines is a classifier using this kernel method by computing, in the feature space and on basis of examples of the different classes, hyperplanes that separate the classes. The hyperplanes in the feature space correspond to non linear surfaces in the input space.</p><p>Concerning facial expressions, the aim is to train and test a classifier able to recognise, on basis of some pictures of faces, which emotion (among these six ones: anger, disgust, fear, joy, sad, and surprise) that is expressed by the person in the picture. In this application, each picture has to be seen has a point in an N-dimensional space where N is the number of pixels in the image.</p><p>The second application is the detection of camouflage nets hidden in vegetation using a hyperspectral image taken by an aircraft. In this case the classification is computed for each pixel, represented by a vector whose elements are the different frequency bands of this pixel.</p>