Thesis (Ph. D.)--University of Hong Kong, 2005.

Thesis (M.S.)--West Virginia University, 2006. / Title from document title page. Document formatted into pages; contains viii, 68 p. : ill. (some col.). Includes abstract. Includes bibliographical references (p. 50-51).

Thesis (M.S.)--Virginia Polytechnic Institute and State University, 1994. / Vita. Abstract. Includes bibliographical references (leaves 155-159). Also available via the Internet.

The issue of robust training is tackled for fixed multilayer feedforward architectures. Several researchers have proved the theoretical capabilities of Multilayer Feedforward networks but in practice the robust convergence of standard methods like standard backpropagation, conjugate gradient descent and Quasi-Newton methods may be poor for various problems. It is suggested that the common assumptions about the overall surface shape break down when many individual component surfaces are combined and robustness suffers accordingly. A new method to train Multilayer Feedforward networks is presented in which no particular shape is assumed for the surface and where an attempt is made to optimally combine the individual components of a solution for the overall solution. The method is based on computing Tangent Hyperplanes to the non-linear solution manifolds. At the core of the method is a mechanism to minimise the sum of squared errors and as such its use is not limited to Neural Networks. The set of tests performed for Neural Networks show that the method is very robust regarding convergence of training and has a powerful ability to find good directions in weight space. Generalisation is also a very important issue in Neural Networks and elsewhere. Neural Networks are expected to provide sensible outputs for unseen inputs. A framework for hyperplane based classifiers is presented for improving average generalisation. The framework attempts to establish a trained boundary so that there is an optimal overall spacing from the boundary to training points closest to this boundary. The framework is shown to provide results consistent with the theoretical expectations.

Neural networks can be analysed from two points of view: training and generalisation. The training is characterised by a trade-off between the 'goodness' of the training algorithm itself (speed, reliability, guaranteed convergence) and the 'goodness' of the architecture (the difficulty of the problems the network can potentially solve). Good training algorithms are available for simple architectures which cannot solve complicated problems. More complex architectures, which have been shown to be able to solve potentially any problem do not have in general simple and fast algorithms with guaranteed convergence and high reliability. A good training technique should be simple, fast and reliable, and yet also be applicable to produce a network able to solve complicated problems. The thesis presents Constraint Based Decomposition (CBD) as a technique which satisfies the above requirements well. CBD is shown to build a network able to solve complicated problems in a simple, fast and reliable manner. Furthermore, the user is given a better control over the generalisation properties of the trained network with respect to the control offered by other techniques. The generalisation issue is addressed, as well. An analysis of the meaning of the term "good generalisation" is presented and a framework for assessing generalisation is given: the generalisation can be assessed only with respect to a known or desired underlying function. The known properties of the underlying function can be embedded into the network thus ensuring a better generalisation for the given problem. This is the fundamental idea of the complex backpropagation network. This network can associate signals through associating some of their parameters using complex weights. It is shown that such a network can yield better generalisation results than a standard backpropagation network associating instantaneous values.

Please read the abstract in the section 00front of this document / Dissertation (MSc (Computer Science))--University of Pretoria, 2007. / Computer Science / unrestricted

Robotic simulators for use in Evolutionary Robotics (ER) have certain challenges associated with the complexity of their construction and the accuracy of predictions made by these simulators. Such robotic simulators are often based on physics models, which have been shown to produce accurate results. However, the construction of physics-based simulators can be complex and time-consuming. Alternative simulation schemes construct robotic simulators from empirically-collected data. Such empirical simulators, however, also have associated challenges, such as that some of these simulators do not generalize well on the data from which they are constructed, as these models employ simple interpolation on said data. As a result of the identified challenges in existing robotic simulators for use in ER, this project investigates the potential use of Artificial Neural Networks, henceforth simply referred to as Neural Networks (NNs), as alternative robotic simulators. In contrast to physics models, NN-based simulators can be constructed without needing an explicit mathematical model of the system being modeled, which can simplify simulator development. Furthermore, the generalization capabilities of NNs suggest that NNs could generalize well on data from which these simulators are constructed. These generalization abilities of NNs, along with NNs’ noise tolerance, suggest that NNs could be well-suited to application in robotics simulation. Investigating whether NNs can be effectively used as robotic simulators in ER is thus the endeavour of this work. Since not much research has been done in employing NNs as robotic simulators, many aspects of the experimental framework on which this dissertation reports needed to be carefully decided upon. Two robot morphologies were selected on which the NN simulators created in this work were based, namely a differentially steered robot and an inverted pendulum robot. Motion tracking and robotic sensor logging were used to acquire data from which the NN simulators were constructed. Furthermore, custom code was written for almost all aspects of the study, namely data acquisition for NN training, the actual NN training process, the evolution of robotic controllers using the created NN simulators, as well as the onboard robotic implementations of evolved controllers. Experimental tests performed in order to determine ideal topologies for each of the NN simulators developed in this study indicated that different NN topologies can lead to large differences in training accuracy. After performing these tests, the training accuracy of the created simulators was analyzed. This analysis showed that the NN simulators generally trained well and could generalize well on data not presented during simulator construction. In order to validate the feasibility of the created NN simulators in the ER process, these simulators were subsequently used to evolve controllers in simulation, similar to controllers developed in related studies. Encouraging results were obtained, with the newly-evolved controllers allowing real-world experimental robots to exhibit obstacle avoidance and light-approaching behaviour with a reasonable degree of success. The created NN simulators furthermore allowed for the successful evolution of a complex inverted pendulum stabilization controller in simulation. It was thus clearly established that NN-based robotic simulators can be successfully employed as alternative simulation schemes in the ER process.

No description available.

The thesis is written in chapter form. Chapter 1 describes some of the history of neural networks and its place in the field of artificial intelligence. It indicates the biological basis from which neural network approximation are made. Chapter 2 describes the properties of neural networks and their uses. It introduces the concepts of training and learning. Chapters 3, 4, 5 and 6 show the perceptron and adaline in feedforward and recurrent networks particular reference is made to regression substitution by "group method data handling. Networks are chosen that explain the application of neural networks in classification, association, optimization and self organization. Chapter 7 addresses the subject of practical inputs to neural networks. Chapter 8 reviews some interesting recent developments. Chapter 9 reviews some ideas on the future technology for neural networks. Chapter 10 gives a listing of some neural network types and their uses. Appendix A gives some of the ideas used in portfolio selection for the Johannesburg Stock Exchange. / Computing / M. Sc. (Operations Research)

published_or_final_version / Physiology / Doctoral / Doctor of Philosophy