Back to search

University of Patras

Νέες μέθοδοι εκπαίδευσης τεχνητών νευρωνικών δικτύων, βελτιστοποίησης και εφαρμογές

Abstract

dc:description

This thesis investigates Optimization methods for Artificial Neural Network training and their applications. In the first chapters we discuss the basic neural network definitions, well known network architectures and training methods, as well as the theoretical background that supports the development of new efficient and effective training algorithms. In Chapter 3 a new generalized theoretical result is presented that underpins the development of first-order globally convergent batch training algorithms which employ local learning rates. This result allows us to equip the algorithms of this class with a strategy for adapting the direction of search to a descent one. In this way, a decrease of the batch-error measure at each training iteration is ensured, and convergence of the sequence of weight iterates to a local minimizer of the batch error function is obtained from remote initial weights. The effectiveness of the theoretical result is illustrated in application examples by comparing two well-known training algorithms with local learning rates to their globally convergent modifications. In Chapter 4, a mathematical framework for the convergence analysis of the well-known QuickProp method is described. Furthermore, we propose a modification of this method that exhibits improved convergence speed and stability, while at the same time, alleviates the use of heuristic learning parameters. Simulations are conducted to compare and evaluate the performance of the new modified QuickProp algorithm with various popular training algorithms. The results of the experiments indicate that the increased convergence rate achieved by the proposed algorithm, by no means affects its generalization capability and stability. In Chapter 5, evolutionary neural network training algorithms are presented. These algorithms are applied to train neural networks with weight values confined to a narrow band of integers. We also train the network using threshold activation functions. Furthermore, parallel evolutionary algorithms for integer weight neural network training are introduced. These algorithms have been designed keeping in mind that the resulting integer weights require less bits to be stored and the digital arithmetic operations between them are easier to be implemented in hardware. Another advantage of the proposed evolutionary strategies is that they are capable of continuing the training process previously computed pieces of information regarding the learning rate adaptation procedure. The proposed alogorithm has been implemented and tested in various problems with large data sets and networks. Additionally, we propose a hybrid Evolutionary algorithm for on-line training, capable of training and retraining a neural network when the task is nonstationary. Finally, in Chapter 8 we present deterministic nonmonotone learning strategies, i.e. deterministic training algorithms in which error function values are allowed to increase at some iterations. To this end, we argue that the current error function value must satisfy a nonmonotone criterion with respect to the maximum error function value of the M previous epochs, and we propose a subprocedure to dynamically compute M. the nonmonotone strategy can be incorporated in any batch training algorithm and provides fast, stable and reliable learning. Extensive experimental results in different classes of problems show that this approach improves the convergence speed and success percentage of first-order training algorithms, and alleviates the need for fine-tuning problem-depended heuristic parameters.

Degree

thesis:*
Grantor dc:publisher
University of Patras
Year dc:date
2002

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Πλαγιανάκος, Βασίλειος
  • Plagianakos, Vassilis

Subjects

dc:subject × 21

Rights

Language dc:language
gre

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/14281

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Πλαγιανάκος, Βασίλειος; Plagianakos, Vassilis. Νέες μέθοδοι εκπαίδευσης τεχνητών νευρωνικών δικτύων, βελτιστοποίησης και εφαρμογές. University of Patras, 2002. http://hdl.handle.net/10442/hedi/14281