Previous works on classification committees have shown that an efficient committee should consist of networks that are not only very accurate, but also diverse. In this work, aiming to explore trade-off between the diversity and accuracy of committee networks, the steps of neural network training, aggregation of the networks into a committee, and elimination of irrelevant input variables are integrated. To accomplish the elimination, an additional term to the Negative correlation learning error function, which forces input weights connected to the irrelevant input variables to decay, is added.

In the human visual system, one of the most prominent functions of the extensive feedback from the higher brain areas within and outside of the visual cortex is attentional modulation. The feedback helps the brain to concentrate its resources on visual features that are relevant for recognition, i. e. it iteratively selects certain aspects of the visual scene for refined processing by the lower areas until the inference process in the higher areas converges to a single hypothesis about this scene. In order to minimize a number of required selection-refinement iterations, one has to find a short sequence of maximally informative portions of the visual input. Since the feedback is not static, the selection process is adapted to a scene that should be recognized. To find a scene-specific subset of informative features, the adaptive selection process on every iteration utilizes results of previous processing in order to reduce the remaining uncertainty about the visual scene. This phenomenon inspired us to develop a computational algorithm solving a visual classification task that would incorporate such principle, adaptive feature selection. It is especially interesting because usually feature selection methods are not adaptive as they define a unique set of informative features for a task and use them for classifying all objects. However, an adaptive algorithm selects features that are the most informative for the particular input. Thus, the selection process should be driven by statistics of the environment concerning the current task and the object to be classified. Applied to a classification task, our adaptive feature selection algorithm favors features that maximally reduce the current class uncertainty, which is iteratively updated with values of the previously selected features that are observed on the testing sample. In information-theoretical terms, the selection criterion is the mutual information of a class variable and a feature-candidate conditioned on the already selected features, which take values observed on the current testing sample. Then, the main question investigated in this thesis is whether the proposed adaptive way of selecting features is advantageous over the conventional feature selection and in which situations. Further, we studied whether the proposed adaptive information-theoretical selection scheme, which is a computationally complex algorithm, is utilized by humans while they perform a visual classification task. For this, we constructed a psychophysical experiment where people had to select image parts that as they think are relevant for classification of these images. We present the analysis of behavioral data where we investigate whether human strategies of task-dependent selective attention can be explained by a simple ranker based on the mutual information, a more complex feature selection algorithm based on the conventional static mutual information and the proposed here adaptive feature selector that mimics a mechanism of the iterative hypothesis refinement. Hereby, the main contribution of this work is the adaptive feature selection criterion based on the conditional mutual information. Also it is shown that such adaptive selection strategy is indeed used by people while performing visual classification.

Actuellement, il y a une demande croissante pour le déploiement de robots mobile dans des lieux publics. Pour alimenter cette demande, plusieurs chercheurs ont déployé des systèmes robotiques de prototypes dans des lieux publics comme les hôpitaux, les supermarchés, les musées, et les environnements de bureau. Une principale préoccupation qui ne doit pas être négligé, comme des robots sortent de leur milieu industriel isolé et commencent à interagir avec les humains dans un espace de travail partagé, est une interaction sécuritaire. Pour un robot mobile à avoir un comportement interactif sécuritaire et acceptable - il a besoin de connaître la présence, la localisation et les mouvements de population à mieux comprendre et anticiper leurs intentions et leurs actions. Cette thèse vise à apporter une contribution dans ce sens en mettant l'accent sur les modalités de perception pour détecter et suivre les personnes à proximité d'un robot mobile. Comme une première contribution, cette thèse présente un système automatisé de détection des personnes visuel optimisé qui prend explicitement la demande de calcul prévue sur le robot en considération. Différentes expériences comparatives sont menées pour mettre clairement en évidence les améliorations de ce détecteur apporte à la table, y compris ses effets sur la réactivité du robot lors de missions en ligne. Dans un deuxiè contribution, la thèse propose et valide un cadre de coopération pour fusionner des informations depuis des caméras ambiant affixé au mur et de capteurs montés sur le robot mobile afin de mieux suivre les personnes dans le voisinage. La même structure est également validée par des données de fusion à partir des différents capteurs sur le robot mobile au cours de l'absence de perception externe. Enfin, nous démontrons les améliorations apportées par les modalités perceptives développés en les déployant sur notre plate-forme robotique et illustrant la capacité du robot à percevoir les gens dans les lieux publics supposés et respecter leur espace personnel pendant la navigation.

A suite of signal processing algorithms designed for extracting information from brain electrophysiology and movement signals, along with new insights gained by applying these tools to understanding parkinsonism, were presented in this dissertation. The approach taken does not assume any particular stimulus, underlying activity, or synchronizing event, nor does it assume any particular encoding scheme. Instead, novel signal processing applications of complex continuous wavelet transforms, cross-frequency-coupling, feature selection, and canonical correlation were developed to discover the most significant electrophysiologic changes in the basal ganglia and cortex of parkinsonian rhesus monkeys and how these changes are related to the motor signs of parkinsonism. The resulting algorithms effectively characterize the severity of parkinsonism and, when combined with motor signal decoding algorithms, allow technology-assisted multi-modal grading of the primary pathological signs. Based on these results, parallel data collection algorithms were implemented in real-time embedded software and off-the-shelf hardware to develop a new system to facilitate monitoring of the severity of Parkinson's disease signs and symptoms in human patients. Off -line analysis of data collected with the system was subsequently shown to allow discrimination between normal and simulated parkinsonian conditions. The main contributions of the work were in three areas: 1) Evidence of the importance of optimally selecting multiple, non-redundant features for understanding neural information, 2) Discovery of signi ficant correlations between certain pathological motor signs and brain electrophysiology in different brain regions, and 3) Implementation and human subject testing of multi-modal monitoring technology.

With growing demands for reliability, availability, safety and cost efficiency in modern machinery, accurate fault diagnosis is becoming of paramount importance so that potential failures can be better managed. Although various methods have been applied to machinery condition monitoring and fault diagnosis, the diagnostic accuracy that can be attained is far from satisfactory. As most machinery faults lead to increases in vibration levels, vibration monitoring has become one of the most basic and widely used methods to detect machinery faults. However, current vibration monitoring methods largely depend on signal processing techniques. This study is based on the recognition that a multi-parameter data fusion approach to diagnostics can produce more accurate results. Fuzzy measures and fuzzy integral data fusion theory can represent the importance of each criterion and express certain interactions among them. This research developed a novel, systematic and effective fuzzy measure and fuzzy integral data fusion approach for machinery fault diagnosis, which comprises feature set selection schema, feature level data fusion schema and decision level data fusion schema for machinery fault diagnosis. Different feature selection and fault diagnostic models were derived from these schemas. Two fuzzy measures and two fuzzy integrals were employed: the 2-additive fuzzy measure, the fuzzy measure, the Choquet fuzzy integral and the Sugeno fuzzy integral respectively. The models were validated using rolling element bearing and electrical motor experiments. Different features extracted from vibration signals were used to validate the rolling element bearing feature set selection and fault diagnostic models, while features obtained from both vibration and current signals were employed to assess electrical motor fault diagnostic models. The results show that the proposed schemas and models perform very well in selecting feature set and can improve accuracy in diagnosing both the rolling element bearing and electrical motor faults.

Σκοπός της συγκεκριμένης εργασίας είναι η έρευνα πάνω στον τομέα της αυτόματης παραγωγής έμπειρων συστημάτων, ανακαλύπτοντας γνώση μέσα σε σύνολα δεδομένων και αναπαριστώντας την με την μορφή κανόνων. Ουσιαστικά πρόκειται για μια μέθοδο επιτηρούμενης μάθησης όπως η εξόρυξη κανόνων ταξινόμησης, ωστόσο ο στόχος δεν είναι αποκλειστικά η ταξινόμηση, αλλά και η τήρηση σημαντικών προδιαγραφών ενός έμπειρου συστήματος όπως η επεξήγηση, η ενημέρωση για νέα δεδομένα κ.α. Στα πλαίσια της προπτυχιακής μου εργασίας αναπτύχθηκε ένα εργαλείο που είχε σκοπό την σύγκριση μεθόδων για συνδυασμό αβέβαιων συμπερασμάτων για το ίδιο γεγονός, στο μοντέλο των Συντελεστών Βεβαιότητας. Το εργαλείο έδινε την δυνατότητα να παραχθούν Έμπειρα Συστήματα (στη γλώσσα CLIPS) που χρησιμοποιούν τις παραπάνω μεθόδους. Σκοπός της παρούσας εργασίας ήταν η διερεύνηση του τομέα της μηχανικής μάθησης και η επέκταση του υπάρχοντος εργαλείου, ώστε να παράγει έμπειρα συστήματα με έναν πιο αυτόματο, αποδοτικό και λειτουργικό τρόπο. Πιο συγκεκριμένα τροποποιήθηκε η αρχιτεκτονική για την υποστήριξη μεταβλητών εξόδου με περισσότερες από δυο κλάσεις (Multiclass Classification). Επίσης έγινε επέκταση ώστε να μπορούν να εξαχθούν κανόνες για περισσότερες μεταβλητές του συνόλου δεδομένων (εκτός δηλαδή από την μεταβλητή εξόδου), για τις οποίες δεν χρειάζεται πλέον να γνωρίζει τιμές ο τελικός χρήστης του έμπειρου συστήματος. Η επέκταση αυτή δίνει την δυνατότητα να σχεδιαστούν πιο πολύπλοκες ιεραρχίες κανόνων, που ακολουθούν μια δενδρική δομή, εύκολα ερμηνεύσιμη από τον άνθρωπο. Το μοντέλο συντελεστών βεβαιότητας επανασχεδιάστηκε, ενώ πλέον προσφέρεται και ένας εναλλακτικός τρόπος υπολογισμού των συντελεστών βεβαιότητας των κανόνων ταξινόμησης ο οποίος βασίζεται στον ορισμό τους στο έμπειρο σύστημα MYCIN. Τα αποτελέσματα έδειξαν ότι σε μη ισορροπημένα σύνολα δεδομένων η μέθοδος αυτή ευνοεί την πρόβλεψη για την κλάση μειοψηφίας. Τεχνικές επιλογής υποσυνόλων χαρακτηριστικών, δίνουν την δυνατότητα αυτοματοποίησης σε μεγάλο βαθμό της διαδικασίας παραγωγής του έμπειρου συστήματος με τρόπο αποδοτικό. Άλλες προσθήκες είναι η δυνατότητα δημιουργίας συστημάτων που μπορούν να ενημερώνονται δυναμικά αξιοποιώντας νέα δεδομένα για το πρόβλημα, η παραγωγή κανόνων και συναρτήσεων για την αλληλεπίδραση με τον χρήστη, η παροχή γραφικού περιβάλλοντος για το παραγόμενο έμπειρο σύστημα κ.α. / The main objective of this thesis is to present a method for automatic generation of expert systems, by extracting knowledge from datasets and representing it in the form of production rules. We use a supervised machine learning method, resembling Classification Rule Mining, although classification is not our only goal. Important operational characteristics of expert systems, like explanation of conclusions and dynamic update of the knowledge base, are also taken into account. Our approach is implemented within an existing tool, initially developed by us to compare methods for combining uncertain conclusions about the same event, based on the uncertainty model of Certainty Factors. That tool could generate Expert Systems (in CLIPS language) that use the above methods. The main aim of this thesis is to do research mainly on the field of machine learning in order to enhance the above mentioned tool for generating Expert Systems in a more automatic, efficient and functional fashion. More specifically, the architecture has been modified to support output variables classified in more than two classes (Multiclass Classification). An extension of the system made it possible to generate classification rules for additional variables (apart from the output variable), for which the final user of the expert system cannot provide values. This gives the ability to design more complex rule hierarchies, which are represented in an easy-to-understand tree form. Furthermore, the certainty factors model has been revised and an additional method of computing them is offered, following the definitions in MYCIN’s model. Experimental results showed improved performance, especially for prediction of minority classes in imbalanced datasets. Feature ranking and subset selection techniques help to achieve the generation task in a more automatic and efficient way. Other enhancements include the ability to produce expert systems that dynamically update the certainty factors in their rules, the generation of rules and functions for interaction with the end-user and a graphical interface for the produced expert system.

Made available in DSpace on 2014-12-17T15:47:50Z (GMT). No. of bitstreams: 1 KarlianeMOV.pdf: 860257 bytes, checksum: 481ec0c73e057f9e2acea8211d919448 (MD5) Previous issue date: 2009-08-07 / Conselho Nacional de Desenvolvimento Cient?fico e Tecnol?gico / The objective of the researches in artificial intelligence is to qualify the computer to execute functions that are performed by humans using knowledge and reasoning. This work was developed in the area of machine learning, that it s the study branch of artificial intelligence, being related to the project and development of algorithms and techniques capable to allow the computational learning. The objective of this work is analyzing a feature selection method for ensemble systems. The proposed method is inserted into the filter approach of feature selection method, it s using the variance and Spearman correlation to rank the feature and using the reward and punishment strategies to measure the feature importance for the identification of the classes. For each ensemble, several different configuration were used, which varied from hybrid (homogeneous) to non-hybrid (heterogeneous) structures of ensemble. They were submitted to five combining methods (voting, sum, sum weight, multiLayer Perceptron and na?ve Bayes) which were applied in six distinct database (real and artificial). The classifiers applied during the experiments were k- nearest neighbor, multiLayer Perceptron, na?ve Bayes and decision tree. Finally, the performance of ensemble was analyzed comparatively, using none feature selection method, using a filter approach (original) feature selection method and the proposed method. To do this comparison, a statistical test was applied, which demonstrate that there was a significant improvement in the precision of the ensembles / As pesquisas em intelig?ncia artificial t?m como objetivo capacitar o computador a executar fun??es que s?o desempenhadas pelo ser humano usando conhecimento e racioc?nio. O presente trabalho foi desenvolvido dentro da ?rea de aprendizado de m?quina (AM), que ? um ramo de estudo da intelig?ncia artificial, sendo relacionado ao projeto e desenvolvimento de algoritmos e t?cnicas capazes de permitir o aprendizado computacional. O objetivo deste trabalho ? analisar um m?todo de sele??o de atributos em comit?s de classificadores. Esse m?todo, baseado em filtros, utilizou a vari?ncia e a correla??o de Spearman para ordenar os atributos e estrat?gias de recompensa e puni??o para medir a import?ncia de cada atributo na identifica??o das classes. Foram formados comit?s de classificadores tanto homog?neos quanto heterog?neos, e submetidos a cinco m?todos de combina??o de classificadores (voto, soma, soma ponderada, MLP e naive Bayes), os quais foram aplicados a seis bases de dados distintas (reais e artificiais). Os classificadores aplicados durante os experimentos foram k-nn, MLP, naive Bayes e ?rvore de decis?o. Por fim, foram analisados, comparativamente, o esempenho dos comit?s de classificadores utilizando nenhum m?todo de sele??o de atributos, utilizando um m?todo de sele??o de atributos padr?o baseado em filtro e o m?todo proposto (RecPun). Com base em um teste estat?stico, foi demonstrado que houve uma melhora significante na precis?o dos comit?s

Made available in DSpace on 2014-12-17T15:47:53Z (GMT). No. of bitstreams: 1 LigiaMMS_DISSERT.pdf: 1430923 bytes, checksum: 56e9de29de6907d7e9da54247e3af4ba (MD5) Previous issue date: 2010-10-14 / Coordena??o de Aperfei?oamento de Pessoal de N?vel Superior / Classifier ensembles are systems composed of a set of individual classifiers and a combination module, which is responsible for providing the final output of the system. In the design of these systems, diversity is considered as one of the main aspects to be taken into account since there is no gain in combining identical classification methods. The ideal situation is a set of individual classifiers with uncorrelated errors. In other words, the individual classifiers should be diverse among themselves. One way of increasing diversity is to provide different datasets (patterns and/or attributes) for the individual classifiers. The diversity is increased because the individual classifiers will perform the same task (classification of the same input patterns) but they will be built using different subsets of patterns and/or attributes. The majority of the papers using feature selection for ensembles address the homogenous structures of ensemble, i.e., ensembles composed only of the same type of classifiers. In this investigation, two approaches of genetic algorithms (single and multi-objective) will be used to guide the distribution of the features among the classifiers in the context of homogenous and heterogeneous ensembles. The experiments will be divided into two phases that use a filter approach of feature selection guided by genetic algorithm / Comit?s de classificadores s?o sistemas compostos por um conjunto de classificadores individuais e um m?dulo de combina??o, o qual ? respons?vel por fornecer a sa?da final do sistema. Para que esses sistemas apresentem melhor desempenho que um classificador simples, ? necess?rio que os componentes individuais n?o cometam erros nos mesmos padr?es. Por este motivo, a diversidade tem sido considerada um dos aspectos mais importantes no projeto desses sistemas, j? que n?o h? vantagem na combina??o de m?todos de classifica??o id?nticos. Uma forma de garantir diversidade ? atrav?s da constru??o de classificadores individuais a partir de diferentes conjuntos de treinamento (padr?es e/ou atributos). Nesse contexto, uma maneira de selecionar subconjuntos de atributos para os classificadores individuais ? atrav?s da utiliza??o de m?todos de sele??o de atributos. No entanto, na maioria das pesquisas, os m?todos de sele??o de atributos s?o aplicados apenas em comit?s de classificadores homog?neos, ou seja, comit?s compostos pelo mesmo tipo de classificador. Sendo assim, o objetivo deste trabalho ? analisar o comportamento desses m?todos na gera??o de comit?s de classificadores diversos, tanto homog?neos como heterog?neos. Para guiar a distribui??o dos atributos, entre os classificadores base, ser?o utilizadas duas abordagens de algoritmo gen?tico (mono-objetivo e multi-objetivo), usando diferentes fun??es de aptid?o. Para tanto, os experimentos ser?o divididos em duas fases, as quais usam uma abordagem filtro para a sele??o de atributos

Software quality ensures that applications that are developed are failure free. Some modern systems are intricate, due to the complexity of their information processes. Software fault prediction is an important quality assurance activity, since it is a mechanism that correctly predicts the defect proneness of modules and classifies modules that saves resources, time and developers’ efforts. In this study, a model that selects relevant features that can be used in defect prediction was proposed. The literature was reviewed and it revealed that process metrics are better predictors of defects in version systems and are based on historic source code over time. These metrics are extracted from the source-code module and include, for example, the number of additions and deletions from the source code, the number of distinct committers and the number of modified lines. In this research, defect prediction was conducted using open source software (OSS) of software product line(s) (SPL), hence process metrics were chosen. Data sets that are used in defect prediction may contain non-significant and redundant attributes that may affect the accuracy of machine-learning algorithms. In order to improve the prediction accuracy of classification models, features that are significant in the defect prediction process are utilised. In machine learning, feature selection techniques are applied in the identification of the relevant data. Feature selection is a pre-processing step that helps to reduce the dimensionality of data in machine learning. Feature selection techniques include information theoretic methods that are based on the entropy concept. This study experimented the efficiency of the feature selection techniques. It was realised that software defect prediction using significant attributes improves the prediction accuracy. A novel MICFastCR model, which is based on the Maximal Information Coefficient (MIC) was developed to select significant attributes and Fast Correlation Based Filter (FCBF) to eliminate redundant attributes. Machine learning algorithms were then run to predict software defects. The MICFastCR achieved the highest prediction accuracy as reported by various performance measures. / School of Computing / Ph. D. (Computer Science)

Title: Artificial Neural Networks and Their Usage For Knowledge Extraction Author: RNDr. Zuzana Petříčková Department: Department of Theoretical Computer Science and Mathema- tical Logic Supervisor: doc. RNDr. Iveta Mrázová, CSc., Department of Theoretical Computer Science and Mathematical Logic Abstract: The model of multi/layered feed/forward neural networks is well known for its ability to generalize well and to find complex non/linear dependencies in the data. On the other hand, it tends to create complex internal structures, especially for large data sets. Efficient solutions to demanding tasks currently dealt with require fast training, adequate generalization and a transparent and simple network structure. In this thesis, we propose a general framework for training of BP/networks. It is based on the fast and robust scaled conjugate gradient technique. This classical training algorithm is enhanced with analytical or approximative sensitivity inhibition during training and enforcement of a transparent in- ternal knowledge representation. Redundant hidden and input neurons are pruned based on internal representation and sensitivity analysis. The performance of the developed framework has been tested on various types of data with promising results. The framework provides a fast training algorithm,...