Back to search

Πανεπιστήμιο Μακεδονίας

Large scale data mining via parallel and distributed neural networks and machine learning schemes

Abstract

dc:description

The thesis focuses on four important factors: parallel learning, distributed learning, hierarchical learning and local learning. Parallelizable algorithms for regression with Extreme Learning Machines (ELM) are investigated in chapters 3 and 4. We speedup the computations for model construction and model selection phase by employing Cholesky, SVD, QR and Eigen decompositions. In chapter 4 we propose the Parallel Enhanced and Convex Incremental Extreme Learning Machine (ECI-ELM) which combines two popular existing incremental versions of ELM, namely the Enhanced I-ELM with the Convex I-ELM, and thus it outperforms them in accuracy, and reveals parallel speedups very close to ideal. Parallel Probabilistic Neural Network (PNN) classifiers are investigated in chapter 5 in a Ring pipeline parallel scheme, where we map the proposed leave-one-out kernel averaged gradient descent algorithm together with Subtractive Clustering and Expectation Maximization. The PNN model is created automatically. Parallel Radial Basis Function (RBF) Neural Networks are studied in chapter 6 by using the Ring pipeline, where we map the proposed leave-one-out Kernel Averaged Gradient descent Subtractive Clustering (KG-SC) for automatically selecting appropriate RBF centers and then the training continues with a mini-batch gradient descent for refining the network parameters. A distributed asynchronous and privacy-preserving Regularization Network committee machine composed of many Regularization networks with no local data exchange among the classifiers is studied in chapter 7. We propose a simple strategy that maps all peers in an asymmetric mutual validation matrix. Experimental results show that the proposed RN committee outperforms majority voting, weighted average and stacked generalization. For distributed asynchronous and privacy-preserving neural network ensemble selection in chapter 8 we proposed the Confidence Ratio Affinity Propagation algorithm, which constructs a two-by-two mapping of all the classifier pairs by using simple one directional point-to-point communication messages. The proposed model showed promising results as compared with other pruning methods, such as reduce-error pruning, Kappa pruning, orientation pruning and JAM’s diversity pruning. Distributed class specialized learning in chapter 9 deals with a Probabilistic Neural Network (PNN) committee machine, where the pattern neurons are composed of locally trained class-specialized regularization networks. Then an asynchronous distributed computing P2P cycle is executed to construct a mutual validation matrix. From this matrix, based on regularized non-negative weighting, it is possible to perform weight based ensemble selection of best members for every class. A new Hierarchical Markovian Radial Basis Function neural network (HiMarkovRBFNN) is presented in chapter 10. The hierarchical structure of this network is composed of nested RBF Neural Networks with arbitrary levels of hierarchy and allows data access in a recursive fashion that enables recursive operations. Comparisons with Committee Machines and Cascaded Machines reveal that the proposed HiMarkovRBFNN serves well as a hierarchical combiner. For Local Learning algorithms that use a different model for a different testing point, we consider Regularization Networks in chapter 11. We explore four different cases for improving the training accuracy and speed by exploiting the interplay between locally optimized and globally optimized parameters.

Degree

thesis:*
Grantor dc:publisher
Πανεπιστήμιο Μακεδονίας
Year dc:date
2016

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Kokkinos, Yiannis
  • Κόκκινος, Γιάννης

Subjects

dc:subject × 30

Rights

Language dc:language
eng

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/44004

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Kokkinos, Yiannis; Κόκκινος, Γιάννης. Large scale data mining via parallel and distributed neural networks and machine learning schemes. Πανεπιστήμιο Μακεδονίας, 2016. http://hdl.handle.net/10442/hedi/44004