The Fuzzy–ARTMAP (FAM) algorithm has been proven to be one of the premier neural network architectures for classification problems. FAM can learn on line and is usually faster than other neural network approaches. Nevertheless the learning time of FAM can slow down considerably when the size of the training set increases into the hundreds of thousands. In this dissertation we apply data partitioning and network partitioning to the FAM algorithm in a sequential and parallel setting to achieve better convergence time and to efficiently train with large databases (hundreds of thousands of patterns). We implement our parallelization on a Beowulf clusters of workstations. This choice of platform requires that the process of parallelization be coarse grained. Extensive testing of all the approaches is done on three large datasets (half a million data points). One of them is the Forest Covertype database from Blackard and the other two are artificially generated Gaussian data with different percentages of overlap between classes. Speedups in the data partitioning approach reached the order of the hundreds without having to invest in parallel computation. Speedups on the network partitioning approach are close to linear on a cluster of workstations. Both methods allowed us to reduce the computation time of training the neural network in large databases from days to minutes. We prove formally that the workload balance of our network partitioning approaches will never be worse than an acceptable bound, and also demonstrate the correctness of these parallelization variants of FAM.

This research focuses on the learning of a class of behaviors defined as high-level behaviors. High-level behaviors are defined here as behaviors that can be executed using a sequence of identifiable behaviors. Represented by low-level contexts, these behaviors are known a priori to learning and can be modeled separately by a knowledge engineer. The learning task, which is achieved by observing an expert within simulation, then becomes the identification and representation of the low-level context sequence executed by the expert. To learn this sequence, this research proposes FAMTILE - the Fuzzy ARTMAP / Template-Based Interpretation Learning Engine. This algorithm attempts to achieve this learning task by constructing rules that govern the low-level context transitions made by the expert. By combining these rules with models for these low-level context behaviors, it is hypothesized that an intelligent model for the expert can be created that can adequately model his behavior. To evaluate FAMTILE, four testing scenarios were developed that attempt to achieve three distinct evaluation goals: assessing the learning capabilities of Fuzzy ARTMAP, evaluating the ability of FAMTILE to correctly predict expert actions and context choices given an observation, and creating a model of the expert's behavior that can perform the high-level task at a comparable level of proficiency.

This dissertation deals with the evolutionary optimization of ART neural network architectures. ART (adaptive resonance theory) was introduced by a Grossberg in 1976. In the last 20 years (1987-2007) a number of ART neural network architectures were introduced into the literature (Fuzzy ARTMAP (1992), Gaussian ARTMAP (1996 and 1997) and Ellipsoidal ARTMAP (2001)). In this dissertation, we focus on the evolutionary optimization of ART neural network architectures with the intent of optimizing the size and the generalization performance of the ART neural network. A number of researchers have focused on the evolutionary optimization of neural networks, but no research has been performed on the evolutionary optimization of ART neural networks, prior to 2006, when Daraiseh has used evolutionary techniques for the optimization of ART structures. This dissertation extends in many ways and expands in different directions the evolution of ART architectures, such as: (a) uses a multi-objective optimization of ART structures, thus providing to the user multiple solutions (ART networks) with varying degrees of merit, instead of a single solution (b) uses GA parameters that are adaptively determined throughout the ART evolution, (c) identifies a proper size of the validation set used to calculate the fitness function needed for ART's evolution, thus speeding up the evolutionary process, (d) produces experimental results that demonstrate the evolved ART's effectiveness (good accuracy and small size) and efficiency (speed) compared with other competitive ART structures, as well as other classifiers (CART (Classification and Regression Trees) and SVM (Support Vector Machines)). The overall methodology to evolve ART using a multi-objective approach, the chromosome representation of an ART neural network, the genetic operators used in ART's evolution, and the automatic adaptation of some of the GA parameters in ART's evolution could also be applied in the evolution of other exemplar based neural network classifiers such as the probabilistic neural network and the radial basis function neural network.

Orientador: Carlos Roberto Minussi / Banca: Francisco Villarreal Alvarado / Banca: Maria do Carmo Gomes da Silveira / Banca: Luciana Cambraia Leite / Banca: Ricardo Menezes Salgado / Resumo: Esta pesquisa visa o desenvolvimento de um método para análise da estabilidade transitória de sistemas de energia eletrica multimaquinas, por meio de uma rede neural ART-ARTMAP Fuzzy Euclidiana Modificada com Treinamento Continuado. Esta arquitetura apresenta tres diferenciais em e relação a outras já utilizadas para abordar tal problema: (1) a rede iniciada com apenas um neuronio ativado e vai se expandindo durante todo o o treinamento/análise, (2) possui um módulo de treinamento continuado e (3) a o possui um módulo de deteção de intruso. No primeiro diferencial, a redeé iniciada com um neuronio e vai se expandindo de acordo com a aquisição de conhecimento, isto faz com que esta se torne muito mais rápida e que o gasto computacional se torne mínimo. Com o módulo de treinamento continuado, a rede neural consegue armazenar novos dados sem a necessidade de realizar o retreinamento. Já o módulo de detecção de intruso faz com que, ao ser apresentada a rede uma configuração "estranha", a rede execute um treinamento específico para que esta configuração, com um número mínimo de entradas, seja incorporada definitivamente à rede neural. A aplicação para a rede proposta nesta pesquisa, foi a análise de estabilidade transitória, considerando-se o modelo clássico (estabilidade de primeira oscilação), para um sistema composto por 10 máquinas síncronas, 45 barras e 73 linhas de transmissão / Abstract: This doctoral research aims to develop a method to analyze the transient stability of multimachine eletric power systems, through a neural network Modified Euclidean Fuzzy ART-ARTMAP with Continuous Training. The architecture presented has three differences in relation to others used to deal with this problem: (1) the network starts with only one neuron activated and expands throughout the training/analysis, (2) has a continuous training module and (3) has an intrusion detection module. The first difference, is the fact that it starts with a neuron and expands according to knowledge acquisition of the network, and causes it to become much faster and the computational expenses becomes minimum. With continuous training mod- ule, the neural network can store the new data without the need for the retraining. The intrusion detection module causes, when presented to the network a strange configuration, the network to carry out a specific training for this configuration with a minimum total of inputs so that the configu- ration is definitely incorporated to the neural network. The application for this network, in this research, was to analyze the transient stability consid- ering the classical model (stability of first oscillation) to a system composed of 10 synchronous machines, 45 buses and 73 transmission lines / Doutor

Made available in DSpace on 2014-12-17T14:55:06Z (GMT). No. of bitstreams: 1 RobinsonLSA_TESE.pdf: 3429587 bytes, checksum: 6e34f5d59ebeb449eb13310ec5ff1eae (MD5) Previous issue date: 2012-11-05 / ART networks present some advantages: online learning; convergence in a few epochs of training; incremental learning, etc. Even though, some problems exist, such as: categories proliferation, sensitivity to the presentation order of training patterns, the choice of a good vigilance parameter, etc. Among the problems, the most important is the category proliferation that is probably the most critical. This problem makes the network create too many categories, consuming resources to store unnecessarily a large number of categories, impacting negatively or even making the processing time unfeasible, without contributing to the quality of the representation problem, i. e., in many cases, the excessive amount of categories generated by ART networks makes the quality of generation inferior to the one it could reach. Another factor that leads to the category proliferation of ART networks is the difficulty of approximating regions that have non-rectangular geometry, causing a generalization inferior to the one obtained by other methods of classification. From the observation of these problems, three methodologies were proposed, being two of them focused on using a most flexible geometry than the one used by traditional ART networks, which minimize the problem of categories proliferation. The third methodology minimizes the problem of the presentation order of training patterns. To validate these new approaches, many tests were performed, where these results demonstrate that these new methodologies can improve the quality of generalization for ART networks / As redes do tipo ART apresentam algumas vantagens: aprendizado online; converg?ncia em poucas ?pocas de treinamento; aprendizado incremental, etc. Contudo, alguns problemas existem: prolifera??o de categorias, sensibilidade a ordem de apresenta??o dos padr?es, escolha de um bom valor para o par?metro de vigil?ncia. O mais importante deles ? o problema da prolifera??o de categorias e ? provavelmente o mais cr?tico. Esse problema faz com que a rede crie v?rias categorias consumindo recursos (recursos para armazenar uma grande quantidade de categorias desnecess?rias impactando negativamente ou at? mesmo inviabilizando o tempo de processamento da rede) sem contribuir para a qualidade da representa??o do problema, ou seja, em muitos casos a quantidade excessiva de categorias geradas pelas redes ART faz com que a qualidade da generaliza??o da rede seja inferior. Outro fator que leva a prolifera??o de categorias das redes do tipo ART ? a dificuldade de aproximar regi?es de classes que tem geometria n?o retangular, ocasionando uma generaliza??o inferior a outros m?todos de classifica??o. A partir da observa??o desses problemas, foi desenvolvido esse trabalho que prop?e tr?s metodologias. Duas dessas metodologias utilizam uma geometria mais flex?vel do que a geometria regular retangular presente nas redes ART tradicionais e minimizam o problema da prolifera??o de categorias. A terceira metodologia minimiza o problema da ordem de apresenta??o dos padr?es e a prolifera??o de categorias. Com o objetivo de validar as novas abordagens, v?rios testes foram realizados. Os resultados obtidos nesses testes demonstram a viabilidade das metodologias propostas em reduzir o n?mero de categorias e melhorar a qualidade da generaliza??o. Em muitos desses testes a quantidade m?nima de categorias necess?rias para classificar corretamente as classes foi atingida ap?s o treinamento, o que demonstra uma significativa melhora em rela??o aos m?todos j? existentes. Al?m disso, devido a nova geometria das categorias, utilizando politopos convexos, a qualidade da generaliza??o melhorou em rala??o ao estado da arte

In this thesis, a Content Based Image Retrieval (CBIR) system to query the objects in an image database is proposed. Images are represented as collections of regions after being segmented with Normalized Cuts algorithm. MPEG-7 content descriptors are used to encode regions in a 239-dimensional feature space. User of the proposed CBIR system decides which objects to query and labels exemplar regions to train the system using a graphical interface. Fuzzy ARTMAP algorithm is used to learn the mapping between feature vectors and binary coded class identification numbers. Preliminary recognition experiments prove the power of fuzzy ARTMAP as a region classifier. After training, features of all regions in the database are extracted and classified. Simple index files enabling fast access to all regions from a given class are prepared to be used in the querying phase. To retrieve images containing a particular object, user opens an image and selects a query region together with a label in the graphical interface of our system. Then the system ranks all regions in the indexed set of the query class with respect to their L2 (Euclidean) distance to the query region and displays resulting images. During retrieval experiments, comparable class precisions with respect to exhaustive searching of the database are maintained which demonstrates e ectiveness of the classifier in narrowing down the search space.

Orientador: Fábio Roberto Chavarette / Resumo: Diante da constante necessidade de avanço tecnológico na agricultura para promover o aumento da produtividade e seguranças aos indivíduos envolvidos no processo, esta pesquisa apresenta o desenvolvimento de um sistema inteligente, utilizando redes neurais artificiais, aplicado ao monitoramento e análise de falhas estruturais em um trator. Simulou-se o trator por meio de um modelo numérico, representado através de equações diferenciais, o qual gera sinais conforme se alteram os parâmetros de velocidade do trator e a distância entre as saliências no solo. Para a análise, identificação e classificação dos dados simulados computacionalmente, foi utilizado uma rede neural do tipo ARTMAP-Fuzzy, que utiliza conceitos da Teoria da Ressonância Adaptativa, cujo algoritmo foi desenvolvido utilizando o programa Matlab. A principal aplicação deste sistema é inspecionar a estrutura do trator objetivando sua melhor conservação, indicando se o mesmo encontra-se em condições normais ou em caso de falha estrutural. Caso uma falha seja detectada, é possível classificar seu tipo. Os resultados evidenciados foram obtidos por meio de média simples entre as execuções do sistema, em virtude de se elevar a veracidade das informações demonstradas. Os resultados obtidos na aplicação da rede neural ao problema especificado mostraram-se ser satisfatórios. / Abstract: Faced with the constant need for technological advancement in agriculture to promote increased productivity and security to individuals involved in the process, this research presents the development of an intelligent system using artificial neural networks applied to the structural health monitoring and analysis of failure on a tractor. The tractor was simulated by means of a numerical model built by differential equations, which generates signals according to the parameters of tractor's speed and the distance between consecutive protrusions on the ground. For the analysis, identification and classification of computationally simulated data, a artificial neural network known as ARTMAP-fuzzy was used, which uses concepts of Adaptive Resonance Theory, whose algorithm was developed using Matlab. The main application of this system is to inspect the tractor structure aiming its better conservation, indicating whether it is in normal conditions or in case of structural failure. If the fault was detected, it is possible to classify the type of failure identified. The disclosed results were obtained by simple average between the executions of the system, because to increase the accuracy of the information shown. The results obtained in the application of artificial neural network to the specified problem proved to be satisfactory. / Mestre

The rapid growth of computers transformed the way in which information and data was stored. With this new paradigm of data access, comes the threat of this information being exposed to unauthorized and unintended users. Many systems have been developed which scrutinize the data for a deviation from the normal behavior of a user or system, or search for a known signature within the data. These systems are termed as Intrusion Detection Systems (IDS). These systems employ different techniques varying from statistical methods to machine learning algorithms. Intrusion detection systems use audit data generated by operating systems, application softwares or network devices. These sources produce huge amount of datasets with tens of millions of records in them. To analyze this data, data mining is used which is a process to dig useful patterns from a large bulk of information. A major obstacle in the process is that the traditional data mining and learning algorithms are overwhelmed by the bulk volume and complexity of available data. This makes these algorithms impractical for time critical tasks like intrusion detection because of the large execution time. Our approach towards this issue makes use of high performance data mining techniques to expedite the process by exploiting the parallelism in the existing data mining algorithms and the underlying hardware. We will show that how high performance and parallel computing can be used to scale the data mining algorithms to handle large datasets, allowing the data mining component to search a much larger set of patterns and models than traditional computational platforms and algorithms would allow. We develop parallel data mining algorithms by parallelizing existing machine learning techniques using cluster computing. These algorithms include parallel backpropagation and parallel fuzzy ARTMAP neural networks. We evaluate the performances of the developed models in terms of speedup over traditional algorithms, prediction rate and false alarm rate. Our results showed that the traditional backpropagation and fuzzy ARTMAP algorithms can benefit from high performance computing techniques which make them well suited for time critical tasks like intrusion detection.

The significant amount of data contained in digital images present barriers to methods of learning from the information they hold. Noise and the subjectivity of image evaluation further complicate such automated processes. In this thesis, we examine a particular area in which these difficulties are experienced. We attempt to control the parameters of a multi-step algorithm that processes visual information. A framework for approaching the parameter selection problem using reinforcement learning agents is presented as the main contribution of this research. We focus on the generation of state and action space, as well as task-dependent reward. We first discuss the automatic determination of fuzzy membership functions as a specific case of the above problem. Entropy of a fuzzy event is used as a reinforcement signal. Membership functions representing brightness have been automatically generated for several images. The results show that the reinforcement learning approach is superior to an existing simulated annealing-based approach. The framework has also been evaluated by optimizing ten parameters of the text detection for semantic indexing algorithm proposed by Wolf et al. Image features are defined and extracted to construct the state space. Generalization to reduce the state space is performed with the fuzzy ARTMAP neural network, offering much faster learning than in the previous tabular implementation, despite a much larger state and action space. Difficulties in using a continuous action space are overcome by employing the DIRECT method for global optimization without derivatives. The chosen parameters are evaluated using metrics of recall and precision, and are shown to be superior to the parameters previously recommended. We further discuss the interplay between intermediate and terminal reinforcement.

The significant amount of data contained in digital images present barriers to methods of learning from the information they hold. Noise and the subjectivity of image evaluation further complicate such automated processes. In this thesis, we examine a particular area in which these difficulties are experienced. We attempt to control the parameters of a multi-step algorithm that processes visual information. A framework for approaching the parameter selection problem using reinforcement learning agents is presented as the main contribution of this research. We focus on the generation of state and action space, as well as task-dependent reward. We first discuss the automatic determination of fuzzy membership functions as a specific case of the above problem. Entropy of a fuzzy event is used as a reinforcement signal. Membership functions representing brightness have been automatically generated for several images. The results show that the reinforcement learning approach is superior to an existing simulated annealing-based approach. The framework has also been evaluated by optimizing ten parameters of the text detection for semantic indexing algorithm proposed by Wolf et al. Image features are defined and extracted to construct the state space. Generalization to reduce the state space is performed with the fuzzy ARTMAP neural network, offering much faster learning than in the previous tabular implementation, despite a much larger state and action space. Difficulties in using a continuous action space are overcome by employing the DIRECT method for global optimization without derivatives. The chosen parameters are evaluated using metrics of recall and precision, and are shown to be superior to the parameters previously recommended. We further discuss the interplay between intermediate and terminal reinforcement.