University of Patras
Structuring and machine learning methodologies for knowledge mining and applications on big data
Abstract
dc:descriptionEducational institutions and medium-sized companies often possess significant underutilized computing power in their desktop hardware, while lacking the budget for dedicated high-performance computing (HPC) or expensive public cloud services. To address this gap, we designed and implemented CommC, a system that transforms existing commodity hardware into a cost-effective private computing cluster. The system leverages a hybrid architecture combining the Proxmox hypervisor for virtual machines, a layered Kubernetes cluster for orchestrating containerized applications, and lightweight Linux Containers for performance-critical tasks. The primary contribution of this work is an economic and architectural model for sustainable, on-premises computing. Our analysis demonstrates that the CommC cluster reduces computational costs by a factor of 6x to 8x compared to running equivalent workloads on major public cloud providers. To support this infrastructure, we developed a novel suite of management tools, including CommC Node Toolkit and CommC K8s Deployer for automating the installation procedure, CommC Provisioner for intelligent resource scheduling, SparkQueue for simplified job submission and SparkReact for interactive model creation. The platform's versatility and performance were validated through the successful deployment of real-world scientific applications, including the DiscMycoVir bioinformatics pipeline, running experiments for research projects in the field of NLP and the development of SparkBN, a scalable Bayesian Network classifier. The CommC project provides a proven blueprint for repurposing existing hardware into a powerful, secure, and economically transformative computing environment, freeing institutions from the recurring costs and vendor lock-in of commercial cloud platforms.
Degree
thesis:*- Grantor dc:publisher
- University of Patras
- Year dc:date
- 2025
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Bompotas, Agorakis
- Μπομπότας, Αγοράκης
Subjects
dc:subject × 20- Υπολογιστικό νέφος
- Αποδοτικότητα κόστους
- Μεγάλα δεδομένα
- Apache spark
- Μηχανική μάθηση
- Επεξεργασία φυσικής γλώσσας
- Βιοπληροφορική
- Cloud computing
- Commodity hardware
- Cost-effectiveness
- Big data
- Machine learning
- Natural language processing
- Bioinformatics
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Επιστήμες Μηχανικού και Τεχνολογία
- Υπολογιστές, Υλικό (hardware) και Αρχιτεκτονική
- Electrical Engineering, Electronic Engineering, Information Engineering
- Engineering and Technology
- Computer science, Hardware and Architecture
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/59912