Classification tasks are an integral part of science, industry, medicine, and business; being such a pervasive technique, its smallest improvement is valuable. Artificial Neural Network (ANN) is one of the strongest techniques used in many disciplines for classification. The ANN technique suffers from drawbacks such as intransparency in spite of its high prediction power. In this dissertation, motivated by learning styles in human brains, ANN’s shortcomings are assuaged and its learning power is improved. Self-Organizing Map (SOM), an ANN variation which has strong unsupervised power, and Feedforward ANN, traditionally used for classification tasks, are hybridized to solidify their benefits and help remove their limitations. These benefits are in two directions: enhancing ANN’s learning power, and improving decision-making. First, the proposed method, named Self-Organizing Error-Driven (SOED) Artificial Neural Network (ANN), shows significant improvements in comparison with usual ANNs. We show SOED is a more accurate, more reliable, and more transparent technique through experimentation with five famous benchmark datasets. Second, the hybridization creates space for inclusion of decision-making goals at the level of ANN’s learning. This gives the classifier the opportunity to handle the inconclusiveness of the data smarter and in the direction of decision-making goals. Through three case studies, naming 1) churn decision analytics, 2) breast cancer diagnosis, and 3) quality control decision making through thermal monitoring of additive manufacturing processes, this novel and cost-sensitive aspect of SOED has been explored and lead to much quantified improvement in decision-making.

No description available.

In the environment where new large scale problems are emerging in various disciplines and pervasive computing applications are becoming more common, there is an urgent need for machine learning algorithms that could process increasing amounts of data using comparatively smaller computing resources in a computational efficient way. Previous research has resulted in many successful learning algorithms that scale linearly or even sub-linearly with sample size and dimension, both in runtime and in space. However, linear or even sub-linear space scaling is often not sufficient, because it implies an unbounded growth in memory with sample size. This clearly opens another challenge: how to learn from large, or practically infinite, data sets or data streams using memory limited resources. Online learning is an important learning scenario in which a potentially unlimited sequence of training examples is presented one example at a time and can only be seen in a single pass. This is opposed to offline learning where the whole collection of training examples is at hand. The objective is to learn an accurate prediction model from the training stream. Upon on repetitively receiving fresh example from stream, typically, online learning algorithms attempt to update the existing model without retraining. The invention of the Support Vector Machines (SVM) attracted a lot of interest in adapting the kernel methods for both offline and online learning. Typical online learning for kernel classifiers consists of observing a stream of training examples and their inclusion as prototypes when specified conditions are met. However, such procedure could result in an unbounded growth in the number of prototypes. In addition to the danger of the exceeding the physical memory, this also implies an unlimited growth in both update and prediction time. To address this issue, in my dissertation I propose a series of kernel-based budgeted online algorithms, which have constant space and constant update and prediction time. This is achieved by maintaining a fixed number of prototypes under the memory budget. Most of the previous works on budgeted online algorithms focus on kernel perceptron. In the first part of the thesis, I review and discuss these existing algorithms and then propose a kernel perceptron algorithm which removes the prototype with the minimal impact on classification accuracy to maintain the budget. This is achieved by dual use of cached prototypes for both model presentation and validation. In the second part, I propose a family of budgeted online algorithms based on the Passive-Aggressive (PA) style. The budget maintenance is achieved by introducing an additional constraint into the original PA optimization problem. A closed-form solution was derived for the budget maintenance and model update. In the third part, I propose a budgeted online SVM algorithm. The proposed algorithm guarantees that the optimal SVM solution is maintained on all the prototype examples at any time. To maximize the accuracy, prototypes are constructed to approximate the data distribution near the decision boundary. In the fourth part, I propose a family of budgeted online algorithms for multi-class classification. The proposed algorithms are the recently proposed SVM training algorithm Pegasos. I prove that the gap between the budgeted Pegasos and the optimal SVM solution directly depends on the average model degradation due to budget maintenance. Following the analysis, I studied greedy multi-class budget maintenance methods based on removal, projection and merging of SVs. In each of these four parts, the proposed algorithms were experimentally evaluated against the state-of-art competitors. The results show that the proposed budgeted online algorithms outperform the competitive algorithm and achieve accuracy comparable to non-budget counterparts while being extremely computationally efficient. / Computer and Information Science

<p> This thesis presents a novel hybrid methodology using Recurrent Neural Network and Dynamic Time Warping to solve the mode estimation problem of a radar warning receiver (RWR). The RWR is an electronic support (ES) system with the primary objective to estimate the threat posed by an unfriendly (hostile) radar in an electronic warfare (EW) environment. One such radar is the multi-function radar (MFR), which employs complex signal architecture to perform multiple tasks. As the threat posed by the radar directly depends on its current mode of operation, it is vital to estimate and track the mode of the radar. The proposed method uses a recurrent neural network (echo state network and recurrent multi-layer perceptron) trained in a supervised manner, with the dynamic time warping algorithm as the post processor to estimate the mode of operation. A grid filter in Bayesian framework is then applied to the dynamic time warp estimate to provide an accurate posterior estimate of the operational mode of the MFR. This novel approach is tested on an EW scenario via simulation by employing a hypothetical MFR. Based on the simulation results, we conclude that the hybrid echo state network is more suitable than its recurrent multi-layer perceptron counterpart for the mode estimation problem of a RWR.</p> / Thesis / Master of Applied Science (MASc)

La compréhension automatique de la parole est au confluent des deux grands domaines que sont la reconnaissance automatique de la parole et l'apprentissage automatique. Un des problèmes majeurs dans ce domaine est l'obtention d'un corpus de données conséquent afin d'obtenir des modèles statistiques performants. Les corpus de parole pour entraîner des modèles de compréhension nécessitent une intervention humaine importante, notamment dans les tâches de transcription et d'annotation sémantique. Leur coût de production est élevé et c'est la raison pour laquelle ils sont disponibles en quantité limitée.Cette thèse vise principalement à réduire ce besoin d'intervention humaine de deux façons : d'une part en réduisant la quantité de corpus annoté nécessaire à l'obtention d'un modèle grâce à des techniques d'apprentissage semi-supervisé (Self-Training, Co-Training et Active-Learning) ; et d'autre part en tirant parti des réponses de l'utilisateur du système pour améliorer le modèle de compréhension.Ce dernier point touche à un second problème rencontré par les systèmes de compréhension automatique de la parole et adressé par cette thèse : le besoin d'adapter régulièrement leurs modèles aux variations de comportement des utilisateurs ou aux modifications de l'offre de services du système / Two wide research fields named Speech Recognition and Machine Learning meet with the Automatic Speech Language Understanding. One of the main problems in this domain is to obtain a sufficient corpus to train an efficient statistical model. Such speech corpora need a lot of human involvement to transcript and semantically annotate them. Their production cost is therefore quite high and they are difficultly available.This thesis mainly aims at reducing the need of human intervention in two ways: firstly, reducing the amount of corpus needed to build a model thanks to some semi-supervised learning methods (Self-Training, Co-Training and Active-Learning); And lastly, using the answers of the system end-user to improve the comprehension model.This last point addresses another problem related to automatic speech understanding systems: the need to adapt their models to the fluctuation of end-user habits or to the modification of the services list offered by the system

Sistemas de análise de gestos têm se destacado por suas contribuições para a interação entre humanos, humanos e máquinas, e humanos e ambiente. Nessa interação, a gesticulação natural é vista como parte do sistema linguístico que suporta a comunicação, e qualquer sistema de informação que objetiva usar interação para suporte à decisão deveria ser capaz de interpretá-la. Essa interpretação pode ser realizada por meio da segmentação das fases do gesto. Para resolver essa tarefa, o estabelecimento de uma representação de dados eficiente para os gestos é um ponto crítico. A representação escolhida e sua associação a técnicas de análise podem ou não favorecer a solução sob implementação. Neste trabalho, formas de representação de gestos são submetidas aos algoritmos de reconhecimento de padrões MLP e SOM para elaborar um ambiente propício à identificação das representações mais discriminantes, quais aspectos as diferentes representações descrevem com eficiência, e como elas podem ser combinadas para melhorar a segmentação das fases do gesto. Para construção das representações multidimensionais são usados aspectos espaciais e temporais combinados com a normalização dos dados e a aplicação do filtro wavelet na busca pela representação mais discriminante para o reconhecimento das fases do gesto. Ambos os algoritmos alcançaram bons resultados com o uso dos aspectos temporais. O MLP conseguiu classificar todas as fases do gesto em configurações de representação contendo dados sobre todos os membros monitorados. O SOM apresentou boa capacidade para formar grupos contendo dados de uma mesma fase do gesto mesmo com o uso de poucas características na construção da representação, porém não foi possível identificar a proposta de uma nova fase do gesto com o aprendizado não supervisionado / Gestures analysis systems have stood out for their contributions to the interaction between humans, humans and machines, and humans and environments. In this interaction, natural gesticulation is seen as part of a linguistic system that supports the communication, and all information systems aiming at the use of such an interaction in making decisions should be able to interpret it. Such an interpretation can be carried out through the gesture phases segmentation. In order to solve this task, the establishment of an efficient data representation for gestures is a critical issue. The chosen representation as well as its combination with techniques for analysis can or can not favor the solution being developed. In this work, different forms representation for gestures are applied to pattern recognition algorithms MLP and SOM to create an adequate environment to identify the more discriminative representations, which aspect the different representations describe with more efficiency, and how they can be combined in order to improve gesture phases segmentation. To construct the multidimensional representations we use spatial and temporal aspects combined with the normalization of the data and the application of the wavelet filter in the search for the most discriminating representation for the recognition of the gesture phases. Both algorithms achieved good results with the use of temporal aspects. MLP was able to classify all gesture phases using representation settings containing data about all monitored members. SOM presented good ability to form groups containing data of the same gesture phase even with the use of few characteristics in the construction of the representation, but it was not possible to identify the proposal of a new gesture phase with unsupervised learning

No presente trabalho duas metodologias da área de inteligência artificial (Redes Neurais e Algoritmos Genéticos) foram utilizadas para realizar um estudo Quimiossistemático da família Asteraceae. A família Asteraceae é uma das maiores famílias entre as Angiospermas, conta com aproximadamente 24.000 espécies. As espécies da família produzem grande diversidade de metabólitos secundários, entre os quais merecem destaque os terpenóides, poliacetilenos, flavonóides e cumarinas. Para um melhor entendimento da diversidade química da família construiu-se um Banco de Dados com as ocorrências de doze classes de metabólitos (monoterpenos, sesquiterpenos, sesquiterpenos lactonizados, diterpenos, triterpenos, cumarinas, flavonóides, poliacetilenos, benzofuranos, benzopiranos, acetofenonas e fenilpropanóides) produzidos pelas espécies da família. A partir desse banco três diferentes estudos foram realizados. No primeiro estudo, utilizando os mapas auto-organizáveis de Kohonen e o banco de dados químico classificado segundo duas das mais recentes filogenias da família foi possível realizar com sucesso separações de tribos e gêneros da família Asteraceae. Também foi possível indicar que a informação química concorda mais com a filogenia de Funk (Funk et al. 2009) do que com a filogenia de Bremer (Bremer 1994, 1996). No estudo seguinte, onde se objetivou a criação de modelos de previsão dos números de ocorrências das doze classes de metabólitos, utilizando o perceptron de múltiplas camadas com algoritmo de retropropagação de erro, o resultado foi insatisfatório. Apesar de em algumas classes de metabólitos a fase de treino da rede apresentar resultados satisfatórios, a fase de teste mostrou que os modelos criados não são capazes de realizar previsão para dados aos quais eles não foram submetidos na fase de treino, e portanto não são modelos adequados para realizar previsões. Finalmente, o terceiro estudo consistiu na criação de modelos de regressão linear utilizando como método de seleção de variáveis os algoritmos genéticos. Nesse estudo foi possível indicar que os monoterpenos e os sesquiterpenos são bastante relacionados biossinteticamente, também foi possível indicar que existem relações biossintéticas entre monoterpenos e diterpenos e entre sesquiterpenos e triterpenos / In this study two methods of artificial intelligence (neural network and genetic algorithms) were used to work out a Chemosystematic study of the Asteraceae family. The family Asteraceae is one of the largest families among the Angiosperms, having about 24,000 species. The species of the family produce a large diversity of secondary metabolites, and some worth mentioning are the terpenoids, polyacetylenes, flavonoids and coumarins. For a better understanding of the chemical diversity of the family a database was built up with the occurrences of twelve classes of metabolites (monoterpenes, sesquiterpenes, lactonizadossesquiterpenes, diterpenes, triterpenes, coumarins, flavonoids, polyacetylenes, Benzofurans, benzopyrans, acetophenones and phenylpropanoids) produced by species of the family. From this database three different studies were conducted. In the first study, using the Kohonen self-organized map and the chemical data classified according to two of the most recent phylogenies of the family, it was possible to successfully separatethe tribes and genera of the Asteraceae family. It was also possible to indicate that the chemical information agrees with the phylogeny of Funk (Funk et al. 2009) than with the phylogeny of Bremer (Bremer 1994, 1996). In the next study, which aims at creating models to predict the number of occurrences of the twelve classes of metabolites using multi-layer perceptron with backpropagation algorithm error, the result was found unsatisfactory. Although in some classes of metabolites the training phase of the network has satisfactory results, the test phase showed that the models created are not able to make prevision for data to which they were submitted in the training phase and thus are not suitable models for predictions. Finally, the third study was the creation of linear regression models using a genetic algorithm method of variable selection. This study could indicate that the monoterpenes and sesquiterpenes are closely related biosynthetically, and was also possible to indicate that there are biosynthetic relations between monoterpenes and diterpenes and between sesquiterpenes and triterpenes

Purpose – The purpose of the study is to expand the knowledge of how wheel motor currents can be combined with machine learning to be used in a collision detection system for autonomous robots, in order to decrease the number of external sensors and open new design opportunities and lowering production costs. Method – The study is conducted with design science research where two artefacts are developed in a cooperation with Globe Tools Group. The artefacts are evaluated in how they categorize data given by an autonomous robot in the two categories collision and non-collision. The artefacts are then tested by generated data to analyse their ability to categorize. Findings – Both artefacts showed a 100 % accuracy in detecting the collisions in the given data by the autonomous robot. In the second part of the experiment the artefacts show that they have different decision boundaries in how they categorize the data, which will make them useful in different applications. Implications – The study contributes to an expanding knowledge in how machine learning and wheel motor currents can be used in a collision detection system. The results can lead to lowering production costs and opening new design opportunities. Limitations – The data used in the study is gathered by an autonomous robot which only did frontal collisions on an artificial lawn. Keywords – Machine learning, K-Nearest Neighbour, Multilayer Perceptron, collision detection, autonomous robots, Collison detection based on current. / Syfte – Studiens syfte är att utöka kunskapen om hur hjulmotorstömmar kan kombineras med maskininlärning för att användas vid kollisionsdetektion hos autonoma robotar, detta för att kunna minska antalet krävda externa sensorer hos dessa robotar och på så sätt öppna upp design möjligheter samt minska produktionskostnader Metod – Studien genomfördes med design science research där två artefakter utvecklades i samarbete med Globe Tools Group. Artefakterna utvärderades sedan i hur de kategoriserade kollisioner utifrån en given datamängd som genererades från en autonom gräsklippare. Studiens experiment introducerade sedan in data som inte ingick i samma datamängd för att se hur metoderna kategoriserade detta. Resultat – Artefakterna klarade med 100% noggrannhet att detektera kollisioner i den giva datamängden som genererades. Dock har de två olika artefakterna olika beslutsregioner i hur de kategoriserar datamängderna till kollision samt icke-kollisioner, vilket kan ge dom olika användningsområden Implikationer – Examensarbetet bidrar till en ökad kunskap om hur maskininlärning och hjulmotorströmmar kan användas i ett kollisionsdetekteringssystem. Studiens resultat kan bidra till minskade kostnader i produktion samt nya design möjligheter Begränsningar – Datamängden som användes i studien samlades endast in av en autonom gräsklippare som gjorde frontalkrockar med underlaget konstgräs. Nyckelord – Maskininlärning, K-nearest neighbor, Multi-layer perceptron, kollisionsdetektion, autonoma robotar

CoordenaÃÃo de AperfeÃoamento de Pessoal de NÃvel Superior / Neste trabalho, à discutido o problema de reconhecimento de objetos utilizando imagens extraÃdas de um sensor industrial 3D. NÃs nos concentramos em 9 extratores de caracterÃsticas, dos quais 7 sÃo baseados nos momentos invariantes (Hu, Zernike, Legendre, Fourier-Mellin, Tchebichef, Bessel-Fourier e Gaussian-Hermite), um outro à baseado na Transformada de Hough e o Ãltimo na anÃlise de componentes independentes, e, 4 classificadores, Naive Bayes, k-Vizinhos mais PrÃximos, MÃquina de Vetor de Suporte e Rede Neural Artificial-Perceptron Multi-Camadas. Para a escolha do melhor extrator de caracterÃsticas, foram comparados os seus desempenhos de classificaÃÃo em termos de taxa de acerto e de tempo de extraÃÃo, atravÃs do classificador k-Vizinhos mais PrÃximos utilizando distÃncia euclidiana. O extrator de caracterÃsticas baseado nos momentos de Zernike obteve as melhores taxas de acerto, 98.00%, e tempo relativamente baixo de extraÃÃo de caracterÃsticas, 0.3910 segundos. Os dados gerados a partir deste, foram apresentados a diferentes heurÃsticas de classificaÃÃo. Dentre os classificadores testados, o classificador k-Vizinhos mais PrÃximos, obteve a melhor taxa mÃdia de acerto, 98.00% e, tempo mÃdio de classificaÃÃo relativamente baixo, 0.0040 segundos, tornando-se o classificador mais adequado para a aplicaÃÃo deste estudo. / In this work, the problem of recognition of objects using images extracted from a 3D industrial sensor is discussed. We focus in 9 feature extractors (where seven are based on invariant moments -Hu, Zernike, Legendre, Fourier-Mellin, Tchebichef, BesselâFourier and Gaussian-Hermite-, another is based on the Hough transform and the last one on independent component analysis), and 4 classifiers (Naive Bayes, k-Nearest Neighbor, Support Vector machines and Artificial Neural Network-Multi-Layer Perceptron). To choose the best feature extractor, their performance was compared in terms of classification accuracy rate and extraction time by the k-nearest neighbors classifier using euclidean distance. The feature extractor based on Zernike moments, got the best hit rates, 98.00 %, and relatively low time feature extraction, 0.3910 seconds. The data generated from this, were presented to different heuristic classification. Among the tested classifiers, the k-nearest neighbors classifier achieved the highest average hit rate, 98.00%, and average time of relatively low rank, 0.0040 seconds, thus making it the most suitable classifier for the implementation of this study.

Sistemas de análise de gestos têm se destacado por suas contribuições para a interação entre humanos, humanos e máquinas, e humanos e ambiente. Nessa interação, a gesticulação natural é vista como parte do sistema linguístico que suporta a comunicação, e qualquer sistema de informação que objetiva usar interação para suporte à decisão deveria ser capaz de interpretá-la. Essa interpretação pode ser realizada por meio da segmentação das fases do gesto. Para resolver essa tarefa, o estabelecimento de uma representação de dados eficiente para os gestos é um ponto crítico. A representação escolhida e sua associação a técnicas de análise podem ou não favorecer a solução sob implementação. Neste trabalho, formas de representação de gestos são submetidas aos algoritmos de reconhecimento de padrões MLP e SOM para elaborar um ambiente propício à identificação das representações mais discriminantes, quais aspectos as diferentes representações descrevem com eficiência, e como elas podem ser combinadas para melhorar a segmentação das fases do gesto. Para construção das representações multidimensionais são usados aspectos espaciais e temporais combinados com a normalização dos dados e a aplicação do filtro wavelet na busca pela representação mais discriminante para o reconhecimento das fases do gesto. Ambos os algoritmos alcançaram bons resultados com o uso dos aspectos temporais. O MLP conseguiu classificar todas as fases do gesto em configurações de representação contendo dados sobre todos os membros monitorados. O SOM apresentou boa capacidade para formar grupos contendo dados de uma mesma fase do gesto mesmo com o uso de poucas características na construção da representação, porém não foi possível identificar a proposta de uma nova fase do gesto com o aprendizado não supervisionado / Gestures analysis systems have stood out for their contributions to the interaction between humans, humans and machines, and humans and environments. In this interaction, natural gesticulation is seen as part of a linguistic system that supports the communication, and all information systems aiming at the use of such an interaction in making decisions should be able to interpret it. Such an interpretation can be carried out through the gesture phases segmentation. In order to solve this task, the establishment of an efficient data representation for gestures is a critical issue. The chosen representation as well as its combination with techniques for analysis can or can not favor the solution being developed. In this work, different forms representation for gestures are applied to pattern recognition algorithms MLP and SOM to create an adequate environment to identify the more discriminative representations, which aspect the different representations describe with more efficiency, and how they can be combined in order to improve gesture phases segmentation. To construct the multidimensional representations we use spatial and temporal aspects combined with the normalization of the data and the application of the wavelet filter in the search for the most discriminating representation for the recognition of the gesture phases. Both algorithms achieved good results with the use of temporal aspects. MLP was able to classify all gesture phases using representation settings containing data about all monitored members. SOM presented good ability to form groups containing data of the same gesture phase even with the use of few characteristics in the construction of the representation, but it was not possible to identify the proposal of a new gesture phase with unsupervised learning