University of Peloponesse
Εξελιγμένα μοντέλα transformers, αλγόριθμοι μηχανικής μάθησης και τεχνικές συνόλου για ερμηνεύσιμη ανάλυση κατηγοριοποίησης κειμένων και συναισθήματος σε πολυγλωσσικά περιβάλλοντα
Abstract
dc:descriptionThis thesis deals with the thorough study, application, and evaluation of modern natural language processing methods (Natural Language Processing – NLP), focusing in particular on Transformer models, deep learning techniques, hybrid schemes with classical classifiers, and explainability methods. The study is structured into nine independent but interrelated scientific papers, covering a wide range of research approaches and applications: from text categorization using the Zero-Shot method to sentiment analysis, classification of Greek-language data in real environments, and comparative evaluation of ensemble models. The experimental applications were mainly based on English-language datasets - such as Twitter US Airline Sentiment and the BBC Text Dataset - while three studies were applied to Greek data from the PSN Helpdesk, with the aim of categorizing requests and queries. The selection of different language datasets allowed for the evaluation of model generalization and the detection of limitations in less supported languages, such as Greek. One of the main findings of the thesis concerns the comparative performance of pre-trained Transformers models, such as DeBERTa, BERT, RoBERTa, DistilBERT, XLM-R, and GPT2, either in zero-shot scenarios or after fine-tuning, in combination with classical classifiers such as XGBoost or through aggregation mechanisms such as Majority Voting and Soft Voting. In cases such as multilingual classification of Greek queries, architectures combining Transformer models with TCN (Temporal Convolutional Networks) and additional attention layers were proposed, achieving significant improvement in all evaluation metrics. Furthermore, it is clearly shown that Majority Voting schemes consistently outperform Soft Voting in datasets with complexities such as irony, uncertainty, or category imbalance. The paper "From Transformers to Voting Ensembles for Interpretable Sentiment Classification" demonstrates, with strong statistical evidence (Friedman and Nemenyi tests), that the performance of many classifiers does not show statistically significant differences, which reinforces the need for evaluation based on the actual content and structure of each dataset and not just metric terms.11 Particular emphasis is placed on evaluating models on the same dataset, which allows for rigorous and methodologically consistent comparisons. An impressive finding of the study concerns the superiority of the Bi-LSTM model with LIME explainability in terms of accuracy (93.5%), F1-score (93.4%), and ROC AUC (98.2%), which exceeds the performance of advanced Transformer-based ensembles, such as GPT2 + T5 + Pythia (Accuracy: 91.7%). This finding reinforces the objections in the literature regarding the "self-evident" superiority of pre-trained models and highlights the need for context-sensitive architecture selection, depending on the nature of the data, the complexity of the problem, and transparency requirements. The contribution of the thesis is both theoretical and practical: it combines different experimental scenarios, offers evidence-based guidelines for the use and evaluation of NLP models in applications with high accuracy and interpretability requirements, and indicates the advantages and limitations of each approach.Of particular research interest is the observation that model performance may be influenced by the length of instances (text length), a factor that has not been sufficiently studied in the international literature. Future work may focus on systematically investigating the relationship between text length and model performance, in conjunction with parameters such as language complexity, category clarity, and decision interpretability. Overall, this thesis contributes significantly to the empirical documentation of the effectiveness of modern NLP methods, offering clear guidelines for model optimization, selecting architectures, and enhancing reliability and transparency in natural language understanding applications.
Degree
thesis:*- Grantor dc:publisher
- University of Peloponesse
- Year dc:date
- 2026
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Kyritsis, Konstantinos
- Κυρίτσης, Κωνσταντίνος
Subjects
dc:subject × 38- Επεξεργασία φυσικής γλώσσας
- Ανάλυση συναισθήματος
- Κατηγοριοποίηση κειμένου
- Μετασχηματιστές
- Μέθοδοι Συνδυαστικής Μάθησης
- Επεξηγήσιμη τεχνητή νοημοσύνη
- Πολυγλωσσικά Embeddings
- Βαθιά μάθηση
- Μάθηση Zero-Shot
- Εποπτευόμενη Προσαρμογή (Fine-Tuning)
- Βαθμονόμηση
- Ερμηνευσιμότητα Μοντέλων
- Πολυγλωσσικά Δεδομένα
- Πολυγλωσσικά Μοντέλα
- Ταξινόμηση κειμένου
- Μεγάλα γλωσσικά μοντέλα
- Natural language processing
- Sentiment analysis
- Text classification
- Distribution transformers
- Ensemble learning
- Explainable artificial intelligence
- Multilingual Embeddings
- Deep learning
- Zero-shot learning
- Supervised Fine-Tuning
- Calibration
- Model Interpretability
- Multilingual Data
- Multilingual Models
- Text categorization
- Large language models
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Επιστήμες Μηχανικού και Τεχνολογία
- Υπολογιστές, Υλικό (hardware) και Αρχιτεκτονική
- Electrical Engineering, Electronic Engineering, Information Engineering
- Engineering and Technology
- Computer science, Hardware and Architecture
Rights
- Language dc:language
- gre
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/61751