Ng Chi-fai. / Thesis submitted in: December 2000. / Thesis (M.Phil.)--Chinese University of Hong Kong, 2001. / Includes bibliographical references (leaves 69-73). / Abstracts in English and Chinese. / Abstract --- p.i / Acknowledgments --- p.iii / List of Figures --- p.viii / List of Tables --- p.ix / Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- Motivation and Objectives --- p.1 / Chapter 1.2 --- Principles of Feedforward Neural Network Approximation --- p.1 / Chapter 1.3 --- Contribution of The Thesis --- p.5 / Chapter 1.4 --- Outline of The Thesis --- p.5 / Chapter 2 --- Feedforward Neural Networks: An Approximator for Nonlinear Control Law --- p.8 / Chapter 2.1 --- Optimization Methods Applied in Feedforward Neural Network Approximation --- p.8 / Chapter 2.2 --- Example in Supervised Learning --- p.10 / Chapter 2.2.1 --- Problem Description --- p.10 / Chapter 2.2.2 --- Neural Network Configuration and Training --- p.12 / Chapter 2.2.3 --- Simulation Result --- p.13 / Chapter 3 --- Neural Based Approximation of Center Manifold Equations --- p.19 / Chapter 3.1 --- Solving Center Manifold Equations by Feedforward Neural Network Approx- imation --- p.19 / Chapter 3.2 --- Example --- p.21 / Chapter 3.2.1 --- Problem Description --- p.21 / Chapter 3.2.2 --- Simulation Result --- p.24 / Chapter 3.2.3 --- Discussion --- p.24 / Chapter 4 --- Connection of Center Manifold Equations to Output Regulation Problem --- p.29 / Chapter 4.1 --- Output Regulation Theory --- p.29 / Chapter 4.2 --- Reduction of Regulator Equation into Center Manifold Equations --- p.31 / Chapter 5 --- Application to the Control Design of Ball and Beam System --- p.34 / Chapter 5.1 --- Problem Description --- p.34 / Chapter 5.2 --- Neural Approximation Solution of Center Manifold Equations --- p.37 / Chapter 5.3 --- Simulation Results --- p.38 / Chapter 5.4 --- Discussion --- p.45 / Chapter 6 --- Neural Based Disturbance Rejection of Nonlinear Benchmark Problem (TORA System) --- p.48 / Chapter 6.1 --- Problem Description --- p.48 / Chapter 6.2 --- Neural based Approximation of the Center Manifold Equations of TORA System --- p.51 / Chapter 6.3 --- Simulation Results --- p.53 / Chapter 6.4 --- Discussion --- p.59 / Chapter 7 --- Conclusion --- p.62 / Chapter 7.1 --- Future Works --- p.63 / Chapter A --- Center Manifold Theory --- p.64 / Chapter B --- Relation between Center Manifold Equation and Output Regulation Prob- lem --- p.66 / Biography --- p.68 / References --- p.69

I conduct a study of learning in HMAX-like models, which are hierarchical models of visual processing in biological vision systems. Such models compute a new representation for an image based on the similarity of image sub-parts to a number of specific patterns, called prototypes. Despite being a central piece of the overall model, the issue of choosing the best prototypes for a given task is still an open problem. I study this problem, and consider the best way to increase task performance while decreasing the computational costs of the model. This work broadens our understanding of HMAX and related hierarchical models as tools for theoretical neuroscience, while simultaneously increasing the utility of such models as applied computer vision systems.

Object recognition in video has seen giant strides in accuracy improvements in the last few years, a testament to the computational capacity of deep convolutional neural networks. However, this computational capacity of software-based neural networks coincides with high power consumption compared to that of some spiking neural networks (SNNs), up to 300,000 times more energy per synaptic event in IBM's TrueNorth chip, for example. SNNs are also well-suited to exploit the precise timing of event-driven image sensors, which transmit asynchronous "events" only when the luminance of a pixel changes above or below a threshold value. The combination of event-based imagers and SNNs becomes a straightforward way to achieve low power consumption in object recognition tasks. This thesis compares different linear classifiers for two low-power, hardware-friendly, spiking, unsupervised neural network architectures, SSLCA and HFirst, in response to asynchronous event-based data, and explores their ability to learn and recognize patterns from two event-based image datasets, N-MNIST and CIFAR10-DVS. By performing a grid search of important SNN and classifier hyperparameters, we also explore how to improve classification performance of these architectures. Results show that a softmax regression classifier exhibits modest accuracy gains (0.73%) over the next-best performing linear support vector machine (SVM), and considerably outperforms a single layer perceptron (by 5.28%) when classification performance is averaged over all datasets and spiking neural network architectures with varied hyperparameters. Min-max normalization of the inputs to the linear classifiers aides in classification accuracy, except in the case of the single layer perceptron classifier. We also see the highest reported classification accuracy for spiking convolutional networks on N-MNIST and CIFAR10-DVS, increasing this accuracy from 97.77% to 97.82%, and 29.67% to 31.76%, respectively. These findings are relevant for any system employing unsupervised SNNs to extract redundant features from event-driven data for recognition.

Human balance control is a complex feedback system that must be adaptable and robust in an infinitely varying external environment. It is probable that there are many concurrent control loops occurring in the central nervous system that achieve stability for a variety of postural perturbations. Though many engineering models of human balance control have been tested, no models of how these controllers might operate within the nervous system have yet been developed. We have focused on building a model of a proprioceptive feedback loop with simulated neurons. The proprioceptive referenced portion of human balance control has been successfully modeled by a PD controller with a time delay and output torque positive feedback. For this model, angular position is measured at the ankle and corrective torque is applied about the joint to maintain a vertical orientation. In this paper, we construct a neural network that performs addition, subtraction, multiplication, differentiation and signal filtering to demonstrate that a simulated biological neural system based off of the engineering control model is capable of matching human test subject dynamics.

An algorithm for solving the large-scale Traveling Salesman Problem is presented. Research into past work in the area of Hopfield neural network use in solving the Traveling Salesman Problem has yielded design ideas that have been incorporated into this work. The algorithm consists of an unsupervised learning algorithm and a recursive Hopfield neural network. The unsupervised learning algorithm was used to decompose the problem into clusters. The recursive Hopfield neural network was applied to the centroids of the clusters, then to the cities in each cluster, in order to find an optimal path. An improvement in both computation speed and solution accuracy is shown by the proposed algorithm over the straight use of the Hopfield neural network.

Abstract not available

A new Bayesian framework for training and selecting the complexity of artificial neural networks (ANNs) is developed in this thesis, based on Markov chain Monte Carlo (MCMC) techniques. The primary motivation of the research presented is the incorporation of uncertainty into ANNs used for water resources modelling, with emphasis placed on obtaining accurate results, while maintaining simplicity of implementation, which is considered to be of utmost importance for adoption of the framework by practitioners in this field. By applying the Bayesian framework to a number of synthetic and real-world case studies and by comparison with a state-of-the-art ANN development approach, it is shown throughout this thesis how the Bayesian approach can be used to address the three most significant issues facing the wider acceptance of ANNs in this field; namely generalisability, interpretability and uncertainty. The state-of-the-art approach is devised through reviewing and, where necessary, improving current best practice deterministic ANN development methods, leading to the recommended use of the global SCE-UA optimisation algorithm, which has not been used before for ANN training, and the development of a modified connection weight approach for extracting knowledge from trained ANNs. The real-world case studies used in this research, which involve salinity forecasting in the River Murray at Murray Bridge, South Australia, and the forecasting of cyanobacteria (Anabaena spp.) in the River Murray at Morgan, South Australia, are used to demonstrate the practical value of the Bayesian framework, particularly when extrapolation is required and when the available data are of poor quality. These issues lead to poor model performance when deterministic ANN development methods are applied, yet as the generated predictions are deterministic, there is no direct way of assessing their quality. Application of the proposed Bayesian framework leads to better average performance of the ANN models developed, since a minimal ANN structure is selected and a more generalised input-output mapping is obtained. More importantly, prediction limits are provided which quantify the uncertainty in the predictions and enable management and design decisions to be made based on a known level of confidence. / http://proxy.library.adelaide.edu.au/login?url= http://library.adelaide.edu.au/cgi-bin/Pwebrecon.cgi?BBID=1235735 / Thesis (Ph.D.) --, University of Adelaide, School of Civil and Environmental Engineering, 2006

Bibliography: p. 199-224. / xviii, 224 p. : ill. ; 30 cm. / Title page, contents and abstract only. The complete thesis in print form is available from the University Library. / This thesis reviews models of the simple cell receptive field profile (RFP) and its variation over the simple cell population, and uses artifical neural networks to investigate the multi-dimensional signal processing role of the RFP in the formation of a cortical representation of the visual image. / Thesis (Ph.D.)--University of Adelaide, Dept. of Electrical and Electronic Engineering, 1994?

"April 2004" / Bibliography: p. 285-299. / xxviii, 299p : ill., map ; 30 cm. / Title page, contents and abstract only. The complete thesis in print form is available from the University Library. / Artificial neural networks (ANNs), trained to make short term forecasts of algal blooms in lakes and rivers, are potentially useful decision making tools for the operational management of eutrophication. This thesis addresses the question of whether a standardised, gemeric ANN model representation can be developed to achieve this goal. It is argued that four requirements need to be addressed: i) compatibility of models with existing water quality monitoring regimes, ii) stability and repeatability of training outcomes, iii) realistic and meaningful estimates of model performance, and iv) explanation of predictions. / Thesis (Ph.D.)--University of Adelaide, School of Earth and Environmental Sciences, Discipline of Environmental Biology, 2004

Modelling of stream macroinvertebrate communities has been widely accepted as an interesting and powerful tool to support water quality assessment and management. Stream Decision Support Framework (SDSF) offers an alternative approach to the current statistical models as Australian River Assessment Scheme (AusRivAs) for the derivation of scientific basis to support management applications regarding fresh water systems. Implementation of Artificial Neural Networks (ANNs) offers a possibility to overcome constraints of the statistical methods in dealing with high nonlinearity of stream data. This thesis includes several case studies illustrating application of Self Organising Map (SOM) and Multilayer Perceptron (MLP) neural networks to various tasks involving analysis, assessment and prediction of stream macroinvertebrates in three Australian states. The data for this study have been provided by the Queensland Department of Natural Resources (NR&M), EPA Victoria and the Department of Land and Water Conservation, New South Wales (NSW). SDSF approach utilises predictive models for both 'referential' and 'dirty-water' approaches. Applicability and high accuracy of ANN models for the purpose of prediction both occurrence of individual taxa and taxonomic richness of stream macroinvertebrates have been demonstrated using data from Victoria and NSW. A comprehensive analysis of salinity sensitivity of stream macroinvertebrate has been demonstrated using both types of ANNs plus statistical methods, and pressure specific Salinity Index was suggested as a measurement of changes within macroinvertebrate communities in response to the secondary salinisation. Scenario analysis of the combined effect of increasing salinity and nutrient load demonstrated predictability and ecological meaningfulness of the Salinity Index. Application of SOM has been demonstrated using the data from Queensland and Victoria in order to analyse natural variability of macroinvertebrate communities between reference sites. SOM component planes provided a valuable insight into the relationships between abiotic variables (as water quality and geoclimatic factors) and distribution of taxa and trophic structure of macroinvertebrate communities. Potential of SOM as data exploration tool has been also demonstrated for the analysis of the output of scenario simulation in order to understand the difference in response to salinisation in different sites. Flexibility and potential of SDSF have been illustrated by using the combination of SOM and MLP, and combination of ANNs with statistical methods. Application of both SOM and Canonical Correspondence Analysis allowed the extraction of additional information and provided convenient visualisation of the relationships between water quality factors and the structure of macroinvertebrate communities. In general, SDSF provides convenient, flexible and accurate approach for the analysis, assessment and prediction of stream biota. In addition to the freedom from the limitations inherent to the traditional statistical methods it allows many more options than currently used modelling frameworks, namely: highly accurate predictions using both 'referential' and 'dirty-water' approaches, sensitivity analysis, scenario analysis and pattern exploration using SOM. / Thesis (Ph.D.)--School of Earth & Environmental Sciences, 2005.