Return to search

Optimization in an Error Backpropagation Neural Network Environment with a Performance Test on a Pattern Classification Problem

Various techniques of optimizing the multiple class cross-entropy error function
to train single hidden layer neural network classifiers with softmax output transfer
functions are investigated on a real-world multispectral pixel-by-pixel classification
problem that is of fundamental importance in remote sensing. These techniques
include epoch-based and batch versions of backpropagation of gradient descent,
PR-conjugate gradient and BFGS quasi-Newton errors. The method of choice
depends upon the nature of the learning task and whether one wants to optimize
learning for speed or generalization performance. It was found that, comparatively
considered, gradient descent error backpropagation provided the best and most stable
out-of-sample performance results across batch and epoch-based modes of operation.
If the goal is to maximize learning speed and a sacrifice in generalisation is acceptable,
then PR-conjugate gradient error backpropagation tends to be superior. If the
training set is very large, stochastic epoch-based versions of local optimizers should
be chosen utilizing a larger rather than a smaller epoch size to avoid inacceptable
instabilities in the generalization results. (authors' abstract) / Series: Discussion Papers of the Institute for Economic Geography and GIScience

Identiferoai:union.ndltd.org:VIENNA/oai:epub.wu-wien.ac.at:4150
Date03 1900
CreatorsFischer, Manfred M., Staufer-Steinnocher, Petra
PublisherWU Vienna University of Economics and Business
Source SetsWirtschaftsuniversität Wien
LanguageEnglish
Detected LanguageEnglish
TypePaper, NonPeerReviewed
Formatapplication/pdf
Relationhttp://epub.wu.ac.at/4150/

Page generated in 0.0019 seconds