Πανεπιστήμιο Μακεδονίας
Large scale data mining via parallel and distributed neural networks and machine learning schemes
Abstract
dc:descriptionThe thesis focuses on four important factors: parallel learning, distributed learning, hierarchical learning and local learning. Parallelizable algorithms for regression with Extreme Learning Machines (ELM) are investigated in chapters 3 and 4. We speedup the computations for model construction and model selection phase by employing Cholesky, SVD, QR and Eigen decompositions. In chapter 4 we propose the Parallel Enhanced and Convex Incremental Extreme Learning Machine (ECI-ELM) which combines two popular existing incremental versions of ELM, namely the Enhanced I-ELM with the Convex I-ELM, and thus it outperforms them in accuracy, and reveals parallel speedups very close to ideal. Parallel Probabilistic Neural Network (PNN) classifiers are investigated in chapter 5 in a Ring pipeline parallel scheme, where we map the proposed leave-one-out kernel averaged gradient descent algorithm together with Subtractive Clustering and Expectation Maximization. The PNN model is created automatically. Parallel Radial Basis Function (RBF) Neural Networks are studied in chapter 6 by using the Ring pipeline, where we map the proposed leave-one-out Kernel Averaged Gradient descent Subtractive Clustering (KG-SC) for automatically selecting appropriate RBF centers and then the training continues with a mini-batch gradient descent for refining the network parameters. A distributed asynchronous and privacy-preserving Regularization Network committee machine composed of many Regularization networks with no local data exchange among the classifiers is studied in chapter 7. We propose a simple strategy that maps all peers in an asymmetric mutual validation matrix. Experimental results show that the proposed RN committee outperforms majority voting, weighted average and stacked generalization. For distributed asynchronous and privacy-preserving neural network ensemble selection in chapter 8 we proposed the Confidence Ratio Affinity Propagation algorithm, which constructs a two-by-two mapping of all the classifier pairs by using simple one directional point-to-point communication messages. The proposed model showed promising results as compared with other pruning methods, such as reduce-error pruning, Kappa pruning, orientation pruning and JAM’s diversity pruning. Distributed class specialized learning in chapter 9 deals with a Probabilistic Neural Network (PNN) committee machine, where the pattern neurons are composed of locally trained class-specialized regularization networks. Then an asynchronous distributed computing P2P cycle is executed to construct a mutual validation matrix. From this matrix, based on regularized non-negative weighting, it is possible to perform weight based ensemble selection of best members for every class. A new Hierarchical Markovian Radial Basis Function neural network (HiMarkovRBFNN) is presented in chapter 10. The hierarchical structure of this network is composed of nested RBF Neural Networks with arbitrary levels of hierarchy and allows data access in a recursive fashion that enables recursive operations. Comparisons with Committee Machines and Cascaded Machines reveal that the proposed HiMarkovRBFNN serves well as a hierarchical combiner. For Local Learning algorithms that use a different model for a different testing point, we consider Regularization Networks in chapter 11. We explore four different cases for improving the training accuracy and speed by exploiting the interplay between locally optimized and globally optimized parameters.
Degree
thesis:*- Grantor dc:publisher
- Πανεπιστήμιο Μακεδονίας
- Year dc:date
- 2016
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Kokkinos, Yiannis
- Κόκκινος, Γιάννης
Subjects
dc:subject × 30- Τεχνητά νευρωνικά δίκτυα
- Υπολογισμοί μεγάλη κλίμακας
- Εξόρυξη γνώσης
- Κατηγοριοποίηση
- Παλινδρόμηση
- Συσταδοποίηση
- Παράλληλη μάθηση
- Κατανεμημένη μάθηση
- Σύνολα και επιτροπές νευρωνικών δικτύων
- Ιεραρχικά νευρωνικά δίκτυα
- Τοπική μάθηση
- Artificial neural networks
- Large scale computing
- Data mining
- Classification
- Regression
- Clustering
- Parallel learning
- Distributed learning
- Neural network ensembles
- Local learning systems
- Hierarchical markovian neural networks
- Φυσικές Επιστήμες
- Επιστήμη Ηλεκτρονικών Υπολογιστών και Πληροφορική
- Επιστήμες Μηχανικού και Τεχνολογία
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Natural Sciences
- Computer and Information Sciences
- Engineering and Technology
- Electrical Engineering, Electronic Engineering, Information Engineering
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/44004