Many classification algorithms are designed on the assumption that the population of interest is stationary, i.e. it does not change over time. However, there are many real-world problems where this assumption is not appropriate. In this thesis, we develop a classifier for non-stationary populations which is based on a multiple logistic model for the conditional class probabilities and incorporates a linear combination of the outputs of a number of pre-determined component classifiers. The final classifier is able to adjust to changes in the population by sequential updating of the coefficients of the linear combination, which are the parameters of the model. The model we use is motivated by the relatively good classification performance which has been achieved by classification rules based on combining classifier outputs. However, in some cases such classifiers can also perform relatively poorly, and in general the mechanisms behind such results are little understood. For the model we propose, which is a generalisation of several existing models for stationary classification problems, we show there exists a simple relationship between the component classifiers which are used, the sign of the parameters and the decision boundaries of the final classifier. This relationship can be used to guide the choice of component classifiers, and helps with understanding the conditions necessary for the classifier to perform well. We compare several "on-line" algorithms for implementing the classification model, where the classifier is updated as new labelled observations become available. The predictive approach to classification is adopted, so each algorithm is based on updating the posterior distribution of the parameters as new information is received. Specifically, we compare a method which assumes the posterior distribution is Gaussian, a more general method developed for the class of Dynamic Generalised Linear Models, and a method based on a sequential Monte Carlo approximation of the posterior. The relationship between the model used for parameter evolution, the bias of the parameter estimates and the error of the classifier is explored.

A generic missile warner decision algorithm for airborne platforms with an emphasis on multiple classifier systems is proposed within the scope of this thesis. For developing the algorithm, simulation data are utilized. The simulation data are created in order to cover a wide range of real-life scenarios and for this purpose a scenario creation methodology is proposed. The scenarios are simulated by a generic missile warner simulator and tracked object data for each scenario are produced. Various feature extraction techniques are applied to the output data of the scenarios and feature sets are generated. Feature sets are examined by using various statistical methods. The performance of selected multiple classifier systems are evaluated for all feature sets and experimental results are presented.

This thesis is focused on the analysis and development of Ensembles of Neural Networks. An ensemble is a system in which a set of heterogeneous Artificial Neural Networks are generated in order to outperform the Single network based classifiers. However, this proposed thesis differs from others related to ensembles of neural networks [1, 2, 3, 4, 5, 6, 7] since it is organized as follows. In this thesis, firstly, an ensemble methods comparison has been introduced in order to provide a rank-based list of the best ensemble methods existing in the bibliography. This comparison has been split into two researches which represents two chapters of the thesis. Moreover, there is another important step related to the ensembles of neural networks which is how to combine the information provided by the neural networks in the ensemble. In the bibliography, there are some alternatives to apply in order to get an accurate combination of the information provided by the heterogeneous set of networks. For this reason, a combiner comparison has also been introduced in this thesis. Furthermore, Ensembles of Neural Networks is only a kind of Multiple Classifier System based on neural networks. However, there are other alternatives to generate MCS based on neural networks which are quite different to Ensembles. The most important systems are Stacked Generalization and Mixture of Experts. These two systems will be also analysed in this thesis and new alternatives are proposed. One of the results of the comparative research developed is a deep understanding of the field of ensembles. So new ensemble methods and combiners can be designed after analyzing the results provided by the research performed. Concretely, two new ensemble methods, a new ensemble methodology called Cross-Validated Boosting and two reordering algorithms are proposed in this thesis. The best overall results are obtained by the ensemble methods proposed. Finally, all the experiments done have been carried out on a common experimental setup. The experiments have been repeated ten times on nineteen different datasets from the UCI repository in order to validate the results. Moreover, the procedure applied to set up specific parameters is quite similar in all the experiments performed. It is important to conclude by remarking that the main contributions are: 1) An experimental setup to prepare the experiments which can be applied for further comparisons. 2) A guide to select the most appropriate methods to build and combine ensembles and multiple classifiers systems. 3) New methods proposed to build ensembles and other multiple classifier systems.

The growth in the use of machine learning in different areas has revealed challenging classification problems that require robust systems. Multiple Classier Systems (MCSs) have attracted interest from researchers as a method that could address such problems. Optimizing the fitness of an MCS improves its, robustness. The lack of an analysis for MCSs from a fitness perspective is identified. To fill this gap, an analytic model from this perspective is derived mathematically by extending the error analysis introduced by Brown and Kuncheva in 2010. The model relates the fitness of an MCS to the average accuracy, positive-diversity, and negative-diversity of the classifiers that constitute the MCS. The model is verified using a statistical analysis of a Monte-Carlo based simulation. This shows the significance of the indicated relationships by the model. This model provides guidelines for developing robust MCSs. It enables the selection of classifiers which compose an MCS with an improved fitness while improving computational cost by avoiding local calculations. The usefulness of the model for designing classification systems is investigated. A new measure consisting of the accuracy and positive-diversity is developed. This measure evaluates fitness while avoiding many calculations compared to the regular measures. A new system (Gadapt) is developed. Gadapt combines machine learning and genetic algorithms to define subsets of the feature space that closely match true class regions. It uses the new measure as a multi-objective criterion for a multi-objective genetic algorithm to identify the MCSs those create the subsets. The design of Gadapt is validated experimentally. The usefulness of the measure and the method of determining the subsets for the performance of Gadapt are examined based on five generated data sets that represent a wide range of problems. The robustness of Gadapt to small amounts of training data is evaluated in comparison with five existing systems on four benchmark data sets. The performance of Gadapt is evaluated in comparison with eleven existing systems on nine benchmark data sets. The analysis of the experiment results supports the validity of the Gadapt design and the outperforming of Gadapt on the existing systems in terms of robustness and performance.

Ferramentas de geoinformação possuem grande aplicabilidade na compreensão e no mapeamento de deslizamentos. Considerando-se a importância dos componentes do relevo e da cobertura do solo neste processo, torna-se essencial o estabelecimento de metodologias para a síntese de informações do relevo e para a identificação de cicatrizes de deslizamento, de maneira a facilitar o monitoramento de áreas de risco. O objetivo desta Tese é propor metodologias de processamento digital de imagens para o mapeamento e identificação de cicatrizes de deslizamento próximo a rodovias. Um deslizamento de grande porte com várias consequências econômicas, ocorrido no ano de 1999, às margens da Rodovia Anchieta, na bacia hidrográfica do Rio Pilões foi utilizado como área de estudo deste trabalho. Utilizando dados gratuitos, mapas de cobertura do solo e de compartimentação do relevo foram gerados e analisados conjuntamente para a identificação das áreas de potenciais cicatrizes na região das Rodovias Anchieta e Imigrantes. A análise do relevo foi realizada utilizando técnicas de classificação baseada em objeto. A identificação de áreas de cicatrizes de deslizamento foi realizada através da avaliação de duas estratégias metodológicas: uma utilizando o algoritmo de classificação supervisionada SVM (Support Vector Machine) aplicado ao índice de vegetação NDVI (Normalized Difference Vegetation Index) e outra que utilizando combinação entre diferentes classificadores para a composição de uma classificação final. Os resultados obtidos para o mapeamento do relevo mostraram que a metodologia proposta possui grande potencial para a descrição de feições do relevo, com maior nível de detalhamento, facilitando a identificação de áreas com grande potencial de ocorrência de deslizamentos. Ambas as metodologias de identificação de cicatrizes de deslizamento apresentaram bons resultados, sendo que a combinação entre os algoritmos SVM, Redes Neurais e Máxima Verossimilhança apresentou o resultado mais adequado com os objetivos do trabalho, atingindo erro de omissão inferior a 10% para a classe de deslizamento. A combinação dos dois produtos permitiu a análise e identificação de diversas áreas de potenciais cicatrizes de deslizamento associadas à rodovias na região de estudo. A metodologia proposta possui ampla replicabilidade, podendo ser utilizada para análises de risco associadas a assentamentos urbanos, empreendimentos lineares e para o planejamento territorial e ambiental. / Geoinformation tools have great applicability in understanding and mapping landslides. Considering the significance of releif components and land cover in this process, it is essential the establishment of methods for the synthesis of the relief information and identification landslides, aiming to facilitate areas risk monitoring. The objective of this Dissertation is to propose digital image processing methodologies for map and identify landslide near to highways. A large landslide with several economic consequences was used as a study area of this work, occurred in 1999, near the Highway Anchieta, in Piloes river basin. Using free data, land cover and relief subdivsion maps were generated and intersected to identify areas of potential landslides in the region of Highways Anchieta and Imigrantes. The relief analysis was performed using based on object classification techniques. The identification of the landslide was performed by evaluating two methodological strategies: one using the supervised classification algorithm SVM (Support Vector Machine) applied to the NDVI vegetation index (Normalized Difference Vegetation Index) and another using combination of different classifiers for the composition of a final classification. The results obtained for relief mapping showed that the proposed method has great potential for the description of the relief features, with greater detail, facilitating the identification of areas with high potential for occurrence of landslides. Both landslides identification methodologies showed good results, and the combination of SVM, Neural Network and Maximum Likelihood algorithms presented the most appropriate result, reaching omission error of less than 10% for the landslide class. The combination of the two products allowed the analysis and identification of several areas of potential landslide scars associated with roads in the study area. The proposed methodology has extensive replication and can be used for risk analysis associated with urban settlements, linear infrastructures and the territorial and environmental planning.

Ferramentas de geoinformação possuem grande aplicabilidade na compreensão e no mapeamento de deslizamentos. Considerando-se a importância dos componentes do relevo e da cobertura do solo neste processo, torna-se essencial o estabelecimento de metodologias para a síntese de informações do relevo e para a identificação de cicatrizes de deslizamento, de maneira a facilitar o monitoramento de áreas de risco. O objetivo desta Tese é propor metodologias de processamento digital de imagens para o mapeamento e identificação de cicatrizes de deslizamento próximo a rodovias. Um deslizamento de grande porte com várias consequências econômicas, ocorrido no ano de 1999, às margens da Rodovia Anchieta, na bacia hidrográfica do Rio Pilões foi utilizado como área de estudo deste trabalho. Utilizando dados gratuitos, mapas de cobertura do solo e de compartimentação do relevo foram gerados e analisados conjuntamente para a identificação das áreas de potenciais cicatrizes na região das Rodovias Anchieta e Imigrantes. A análise do relevo foi realizada utilizando técnicas de classificação baseada em objeto. A identificação de áreas de cicatrizes de deslizamento foi realizada através da avaliação de duas estratégias metodológicas: uma utilizando o algoritmo de classificação supervisionada SVM (Support Vector Machine) aplicado ao índice de vegetação NDVI (Normalized Difference Vegetation Index) e outra que utilizando combinação entre diferentes classificadores para a composição de uma classificação final. Os resultados obtidos para o mapeamento do relevo mostraram que a metodologia proposta possui grande potencial para a descrição de feições do relevo, com maior nível de detalhamento, facilitando a identificação de áreas com grande potencial de ocorrência de deslizamentos. Ambas as metodologias de identificação de cicatrizes de deslizamento apresentaram bons resultados, sendo que a combinação entre os algoritmos SVM, Redes Neurais e Máxima Verossimilhança apresentou o resultado mais adequado com os objetivos do trabalho, atingindo erro de omissão inferior a 10% para a classe de deslizamento. A combinação dos dois produtos permitiu a análise e identificação de diversas áreas de potenciais cicatrizes de deslizamento associadas à rodovias na região de estudo. A metodologia proposta possui ampla replicabilidade, podendo ser utilizada para análises de risco associadas a assentamentos urbanos, empreendimentos lineares e para o planejamento territorial e ambiental. / Geoinformation tools have great applicability in understanding and mapping landslides. Considering the significance of releif components and land cover in this process, it is essential the establishment of methods for the synthesis of the relief information and identification landslides, aiming to facilitate areas risk monitoring. The objective of this Dissertation is to propose digital image processing methodologies for map and identify landslide near to highways. A large landslide with several economic consequences was used as a study area of this work, occurred in 1999, near the Highway Anchieta, in Piloes river basin. Using free data, land cover and relief subdivsion maps were generated and intersected to identify areas of potential landslides in the region of Highways Anchieta and Imigrantes. The relief analysis was performed using based on object classification techniques. The identification of the landslide was performed by evaluating two methodological strategies: one using the supervised classification algorithm SVM (Support Vector Machine) applied to the NDVI vegetation index (Normalized Difference Vegetation Index) and another using combination of different classifiers for the composition of a final classification. The results obtained for relief mapping showed that the proposed method has great potential for the description of the relief features, with greater detail, facilitating the identification of areas with high potential for occurrence of landslides. Both landslides identification methodologies showed good results, and the combination of SVM, Neural Network and Maximum Likelihood algorithms presented the most appropriate result, reaching omission error of less than 10% for the landslide class. The combination of the two products allowed the analysis and identification of several areas of potential landslide scars associated with roads in the study area. The proposed methodology has extensive replication and can be used for risk analysis associated with urban settlements, linear infrastructures and the territorial and environmental planning.

Dans cette thèse, nous proposons plusieurs nouvelles techniques pour la classification d'images hyperspectrales basées sur l'apprentissage d'ensemble. Le cadre proposé introduit des innovations importantes par rapport aux approches précédentes dans le même domaine, dont beaucoup sont basées principalement sur un algorithme individuel. Tout d'abord, nous proposons d'utiliser la Forêt de Rotation (Rotation Forest) avec différentes techiniques d'extraction de caractéristiques linéaire et nous comparons nos méthodes avec les approches d'ensemble traditionnelles, tels que Bagging, Boosting, Sous-espace Aléatoire et Forêts Aléatoires. Ensuite, l'intégration des machines à vecteurs de support (SVM) avec le cadre de sous-espace de rotation pour la classification de contexte est étudiée. SVM et sous-espace de rotation sont deux outils puissants pour la classification des données de grande dimension. C'est pourquoi, la combinaison de ces deux méthodes peut améliorer les performances de classification. Puis, nous étendons le travail de la Forêt de Rotation en intégrant la technique d'extraction de caractéristiques locales et l'information contextuelle spatiale avec un champ de Markov aléatoire (MRF) pour concevoir des méthodes spatio-spectrale robustes. Enfin, nous présentons un nouveau cadre général, ensemble de sous-espace aléatoire, pour former une série de classifieurs efficaces, y compris les arbres de décision et la machine d'apprentissage extrême (ELM), avec des profils multi-attributs étendus (EMaPS) pour la classification des données hyperspectrales. Six méthodes d'ensemble de sous-espace aléatoire, y compris les sous-espaces aléatoires avec les arbres de décision, Forêts Aléatoires (RF), la Forêt de Rotation (RoF), la Forêt de Rotation Aléatoires (Rorf), RS avec ELM (RSELM) et sous-espace de rotation avec ELM (RoELM), sont construits par multiples apprenants de base. L'efficacité des techniques proposées est illustrée par la comparaison avec des méthodes de l'état de l'art en utilisant des données hyperspectrales réelles dans de contextes différents. / In this thesis, we propose several new techniques for the classification of hyperspectral remote sensing images based on multiple classifier system (MCS). Our proposed framework introduces significant innovations with regards to previous approaches in the same field, many of which are mainly based on an individual algorithm. First, we propose to use Rotation Forests with several linear feature extraction and compared them with the traditional ensemble approaches, such as Bagging, Boosting, Random subspace and Random Forest. Second, the integration of the support vector machines (SVM) with Rotation subspace framework for context classification is investigated. SVM and Rotation subspace are two powerful tools for high-dimensional data classification. Therefore, combining them can further improve the classification performance. Third, we extend the work of Rotation Forests by incorporating local feature extraction technique and spatial contextual information with Markov random Field (MRF) to design robust spatial-spectral methods. Finally, we presented a new general framework, Random subspace ensemble, to train series of effective classifiers, including decision trees and extreme learning machine (ELM), with extended multi-attribute profiles (EMAPs) for classifying hyperspectral data. Six RS ensemble methods, including Random subspace with DT (RSDT), Random Forest (RF), Rotation Forest (RoF), Rotation Random Forest (RoRF), RS with ELM (RSELM) and Rotation subspace with ELM (RoELM), are constructed by the multiple base learners. The effectiveness of the proposed techniques is illustrated by comparing with state-of-the-art methods by using real hyperspectral data sets with different contexts.