Back to search

Democritus University of Thrace (DUTH)

Training and performance improvement techniques for artificial neural networks

Abstract

dc:description

Artificial Neural Networks (ANN) have shown that ANN can provide viable solutions to a variety of problems. The emergence of deep learning and deeper and more complex architecture has increased their outreach and their effectiveness into multiple areas and application fields. This has become possible with the emergence of super-performing computing devices with great parallelization capabilities, i.e. graphics cards. Since these devices are extremely power consuming, a open problem in neural network research is the optimization of the neural network architecture for each task. However, their complexity and the lack of efficient interpretation of ANN architectures (commonly considered black box techniques), has adverse effects on the optimization of each ANN architecture. The commonly used solution for this solution is the manual determination of the network architecture through trial and error. In this thesis, the main aim is to address this problem by exploring a solution by three distinct perspectives that can, however, work cooperatively. The first investigated aspect is using a combination of smaller in series connected neural networks that achieve the same if not better performance of larger and more complex neural networks. The data are handled, based on evaluation of a probabilistic function that attempts to identify a portion of the unknown data that are most probably classified correctly and those that need to be reclassified from the beginning. The correctly identified data can be used during a second training process in order to boost the overall classification performance. Another investigated aspect is finding the optimal topology of a neural network, based on the given dataset automatically and during training. The neural network is viewed from an information-theoretic point-of-view and information theoretic criteria were established in order to estimate the optimal size of each layer of the network. The developed solution provided better utilization of the given resources, thus minimizing the computational complexity of the training process. Finally, a novel activation function was proposed with learn-able parameters, based on the ReLU family of functions, that achieved better training efficiency in most of the tested datasets and neural network topologies. The learn-able parameter of the network appeared to act as a regularizing factor for the network. This thesis offers some important contributions towards the automatic topology optimization of neural networks, as well as a form of semi-supervised training strategy for classification. Both proposed solutions appeared to boost classification performance or offer better resource utilization for the optimized network. Finally, the self-regularizing activation function with a trainable parameter offer an increase in classification performance, compared to the fixed activation function, with a minimal computational cost.

Degree

thesis:*
Grantor dc:publisher
Democritus University of Thrace (DUTH)
Year dc:date
2022

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Maniatopoulos, Andreas-Antonios
  • Μανιατόπουλος, Ανδρέας-Αντώνιος

Subjects

dc:subject × 12

Rights

Language dc:language
eng

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/52963

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Maniatopoulos, Andreas-Antonios; Μανιατόπουλος, Ανδρέας-Αντώνιος. Training and performance improvement techniques for artificial neural networks. Democritus University of Thrace (DUTH), 2022. http://hdl.handle.net/10442/hedi/52963