This paper describes an isolated-word and speaker-independent Mandarin digit speech recognition system based on Backpropagation Neural Networks(BPNN). The recognition rate will achieve up to 95%. When the system was applied to a new user with adaptive modification method, the recognition rate will be higher than 99%. In order to implement the speech recognition system on Digital Signal Processors (DSP) we use a neuron-cancellation rule in accordance with BPNN. The system will cancel about 1/3 neurons and reduce 20%¡ã40% memory size under the rule. However, the recognition rate can still achiever up to 85%. For the output structure of the BPNN, we present a binary-code to supersede the one-to-one model. In addition, we use a new ideal about endpoint detection algorithm for the recoding signals. It can avoid disturbance without complex computations.

This research involves the development of an aircraft fuel consumption model to simplify Bela Collins of the MITRE Corporation aircraft fuelburn model in terms of level of computation and level of capability. MATLAB and its accompanying Neural Network Toolbox, has been applied to data from the base model to predict fuel consumption. The approach to the base model and neural network is detailed in this paper. It derives from the basic concepts of energy balance. Multivariate curve fitting techniques used in conjunction with aircraft performance data derive the aircraft specific constants. Aircraft performance limits are represented by empirical relationships that also utilize aircraft specific constants. It is based on generally known assumptions and approximations for commercial jet operations. It will simulate fuel consumption by adaptation of a specific aircraft using constants that represent the relationship of lift-to-drag and thrust-to-fuel flow. The neural network model invokes the output from MITRE1s algorithm and provides: (1) a comparison to the polynomial fuelburn function in the fuelburn post- processor of the FAA Airport and Airspace Simulation Model (SIMMOD), (2) an established sensitivity of system performance for a range of variables that effect fuel consumption, (3) a comparison of post fuel burn (fuel consumption algorithms) techniques to new techniques, and (4) the development of a trained demo neural network. With the powerful features of optimization, graphics, and hierarchical modeling, the MATLAB toolboxes proved to be effective in this modeling process. / Master of Science

To understand remotely sensed data, one must understand the relationship between radiative transfer models and their predictions of the interaction of solar radiation on geophysical media. If it can be established that these models are indeed accurate, some form of evaluation has to be performed on these models, for users to choose the model that suits their requirements. This thesis focuses on the implementation of a variable linear kernel model, its validation, and to study its application in the prediction of BRDF effects using two different neural networks-- the backpropogation and the radial basis function neural network and finally to draw conclusions on which neural network is best suited for this model. Based on these results the optimum number of kernels for this model is derived.

A feed forward architecture is suggested that increases the complexity of conventional neural network components through the implementation of a more complex scheme of interconnection. This is done with a view to increasing the range of application of the feed forward paradigm. The uniqueness of this new network design is illustrated by developing an extended taxonomy of accepted published constructs specific and similar to the higher order, product kernel approximations achievable using "parallel paths". Network topologies from this taxonomy are then compared to each other and the architectures containing parallel paths. In attempting this comparison, the context of the term "network topology" is reconsidered. The output of "channels" in these parallel paths are the products of a conventional connection as observed facilitating interconnection between two layers in a multilayered perceptron and the output of a network processing unit, a "control element", that can assume the identity of a number of pre-existing processing paradigms. The inherent property of universal approximation is tested by existence proof and the method found to be inconclusive. In so doing an argument is suggested to indicate that the parametric nature of the functions as determined by conditions upon initialization may only lead to conditional approximations. The property of universal approximation is neither, confirmed or denied. Universal approximation cannot be conclusively determined by the application of Stone Weierstrass Theorem, as adopted from real analysis. This novel implementation requires modifications to component concepts and the training algorithm. The inspiration for these modifications is related back to previously published work that also provides the basis of "proof of concept". By achieving proof of concept the appropriateness of considering network topology without assessing the impact of the method of training on this topology is considered and discussed in some detail. Results of limited testing are discussed with an emphasis on visualising component contributions to the global network output.

Feedforward neural networks have been widely studied and used in many applications in science and engineering. The training of this type of networks is mainly undertaken using the well-known backpropagation based learning algorithms. One major problem with this type of algorithms is the slow training convergence speed, which hinders their applications. In order to improve the training convergence speed of this type of algorithms, many researchers have developed different improvements and enhancements. However, the slow convergence problem has not been fully addressed. This thesis makes several contributions by proposing new backpropagation learning algorithms based on the terminal attractor concept to improve the existing backpropagation learning algorithms such as the gradient descent and Levenberg-Marquardt algorithms. These new algorithms enable fast convergence both at a distance from and in a close range of the ideal weights. In particular, a new fast convergence mechanism is proposed which is based on the fast terminal attractor concept. Comprehensive simulation studies are undertaken to demonstrate the effectiveness of the proposed backpropagataion algorithms with terminal attractors. Finally, three practical application cases of time series forecasting, character recognition and image interpolation are chosen to show the practicality and usefulness of the proposed learning algorithms with comprehensive comparative studies with existing algorithms.

Este trabalho a um estudo a respeito da aplicação de Redes Neurais Artificiais (RNAs), mais especificamente do modelo perceptron multi-camadas com aprendizado por retro-propagação de erros, a previsão de valores futuros de Series Temporais. 0 estudo foi realizado através da realização de previsões a partir de uma determinada arquitetura de rede neural, a qual é construída com base na analise estatística da serie, para três series reais. A primeira representa o Índice mensal de passageiros das linhas aéreas americanas entre janeiro de 1960 e dezembro de 1971, a segunda corresponde ao índice pluviométrico anual da cidade de Fortaleza no estado do Ceara entre 1849 e 1984, e a terceira trata do índice mensal de produção industrial do estado do Rio Grande do Sul entre janeiro de 1981 e julho de 1993. As duas primeiras series são exemplos clássicos utilizados no estudo dos modelos estatísticos aplicados a previsão de Series Temporais. Os resultados obtidos com as RNAs foram comparados aos progn6sticos realizados pelo método economêtrico que apresenta os melhores resultados para o problema da previsão de Series Temporais: o método da decomposição da serie em suas componentes básicas não-observáveis (tendência, sazonalidade, ciclo e irregular). Tais resultados mostraram que as RNAs podem apresentar excelentes níveis de precisão em seus prognósticos, indicando sua adaptação ao problema da previsão de valores futuros de Séries Temporais. / This work presents a study of the prediction power of Artificial Neural Networks (ANN) in comparison with prediction capability of traditional Time Series models, more specifically the Unobservable Components Models (UCM). The data used to perform the study was the monthly american airlines passengers, the annual rainfall in Fortaleza, Brazil and the monthly gross industrial output for the state of Rio Grande do Sul, Brazil. The results show that Artificial Neural Networks can outperform the forecasts of Unobservable Components Models.

Este trabalho a um estudo a respeito da aplicação de Redes Neurais Artificiais (RNAs), mais especificamente do modelo perceptron multi-camadas com aprendizado por retro-propagação de erros, a previsão de valores futuros de Series Temporais. 0 estudo foi realizado através da realização de previsões a partir de uma determinada arquitetura de rede neural, a qual é construída com base na analise estatística da serie, para três series reais. A primeira representa o Índice mensal de passageiros das linhas aéreas americanas entre janeiro de 1960 e dezembro de 1971, a segunda corresponde ao índice pluviométrico anual da cidade de Fortaleza no estado do Ceara entre 1849 e 1984, e a terceira trata do índice mensal de produção industrial do estado do Rio Grande do Sul entre janeiro de 1981 e julho de 1993. As duas primeiras series são exemplos clássicos utilizados no estudo dos modelos estatísticos aplicados a previsão de Series Temporais. Os resultados obtidos com as RNAs foram comparados aos progn6sticos realizados pelo método economêtrico que apresenta os melhores resultados para o problema da previsão de Series Temporais: o método da decomposição da serie em suas componentes básicas não-observáveis (tendência, sazonalidade, ciclo e irregular). Tais resultados mostraram que as RNAs podem apresentar excelentes níveis de precisão em seus prognósticos, indicando sua adaptação ao problema da previsão de valores futuros de Séries Temporais. / This work presents a study of the prediction power of Artificial Neural Networks (ANN) in comparison with prediction capability of traditional Time Series models, more specifically the Unobservable Components Models (UCM). The data used to perform the study was the monthly american airlines passengers, the annual rainfall in Fortaleza, Brazil and the monthly gross industrial output for the state of Rio Grande do Sul, Brazil. The results show that Artificial Neural Networks can outperform the forecasts of Unobservable Components Models.

Este trabalho a um estudo a respeito da aplicação de Redes Neurais Artificiais (RNAs), mais especificamente do modelo perceptron multi-camadas com aprendizado por retro-propagação de erros, a previsão de valores futuros de Series Temporais. 0 estudo foi realizado através da realização de previsões a partir de uma determinada arquitetura de rede neural, a qual é construída com base na analise estatística da serie, para três series reais. A primeira representa o Índice mensal de passageiros das linhas aéreas americanas entre janeiro de 1960 e dezembro de 1971, a segunda corresponde ao índice pluviométrico anual da cidade de Fortaleza no estado do Ceara entre 1849 e 1984, e a terceira trata do índice mensal de produção industrial do estado do Rio Grande do Sul entre janeiro de 1981 e julho de 1993. As duas primeiras series são exemplos clássicos utilizados no estudo dos modelos estatísticos aplicados a previsão de Series Temporais. Os resultados obtidos com as RNAs foram comparados aos progn6sticos realizados pelo método economêtrico que apresenta os melhores resultados para o problema da previsão de Series Temporais: o método da decomposição da serie em suas componentes básicas não-observáveis (tendência, sazonalidade, ciclo e irregular). Tais resultados mostraram que as RNAs podem apresentar excelentes níveis de precisão em seus prognósticos, indicando sua adaptação ao problema da previsão de valores futuros de Séries Temporais. / This work presents a study of the prediction power of Artificial Neural Networks (ANN) in comparison with prediction capability of traditional Time Series models, more specifically the Unobservable Components Models (UCM). The data used to perform the study was the monthly american airlines passengers, the annual rainfall in Fortaleza, Brazil and the monthly gross industrial output for the state of Rio Grande do Sul, Brazil. The results show that Artificial Neural Networks can outperform the forecasts of Unobservable Components Models.

Evolutionary algorithms have achieved great performance among a wide range of optimization problems. In this degree project, the network optimization problem has been reformulated and solved in an evolved way. A feasible evolutionary framework has been designed and implemented to train neural networks in supervised learning scenarios. Under the structure of evolutionary algorithms, a well-defined fitness function is applied to evaluate network parameters, and a carefully derived form of approximate gradients is used for updating parameters. Performance of the framework has been tested by training two different types of networks, linear affine networks and convolutional networks, for a flash call verification task.Under this application scenario, whether a flash call verification will be successful or not will be predicted by a network, which is inherently a binary classification problem. Furthermore, its performance has also been compared with traditional backpropagation optimizers from two aspects: accuracy and time consuming. The results show that this framework is able to push a network training process to converge into a certain level. During the training process, despite of noises and fluctuations, both accuracies and losses converge roughly under the same pattern as in backpropagation. Besides, the evolutionary algorithm seems to have higher updating efficiency per epoch at the first training stage before converging. While with respect to fine tuning, it doesn’t work as good as backpropagation in the final convergence period. / Evolutionära algoritmer uppnår bra prestanda för ett stort antal olika typer av optimeringsproblem. I detta examensprojekt har ett nätverksoptimeringsproblem lösts genom omformulering och vidareutveckling av angreppssättet. Ett förslag till ramverk har utformats och implementerats för att träna neuronnätverk i övervakade inlärningsscenarier. För evolutionära algoritmer används en väldefinierad träningsfunktion för att utvärdera nätverksparametrar, och en noggrant härledd form av approximerade gradienter används för att uppdatera parametrarna. Ramverkets prestanda har testats genom att träna två olika typer av linjära affina respektive konvolutionära neuronnätverk, för optimering av telefonnummerverifiering. I detta applikationsscenario förutses om en telefonnummerverifiering kommer att lyckas eller inte med hjälp av ett neuronnätverk som i sig är ett binärt klassificeringsproblem. Dessutom har dess prestanda också jämförts med traditionella backpropagationsoptimerare från två aspekter: noggrannhet och hastighet. Resultaten visar att detta ramverk kan driva en nätverksträningsprocess för att konvergera till en viss nivå. Trots brus och fluktuationer konvergerar både noggrannhet och förlust till ungefär under samma mönster som i backpropagation. Dessutom verkar den evolutionära algoritmen ha högre uppdateringseffektivitet per tidsenhet i det första träningsskedet innan den konvergerar. När det gäller finjustering fungerar det inte lika bra som backpropagation under den sista konvergensperioden.

Rekursive Auto-Assoziative Speicher (RAAM) sind spezielle Neuronale Netze (NN), die in der Lage sind, hierarchiche Strukturen zu verarbeiten. Bei der Simulation dieser Netze gibt es einige Besonderheiten, wie z.B. die dynamische Trainingsmenge, zu beachten. In der Arbeit werden diese und die daraus resultierenden angepassten Lernalgorithmen erörtert. Außerdem wird ein normaler Backpropagation-Simulator (Xerion) um die Fähigkeiten für die Simulation von RAAMs erweitert.