Back to search

University of Peloponesse

Εξελιγμένα μοντέλα transformers, αλγόριθμοι μηχανικής μάθησης και τεχνικές συνόλου για ερμηνεύσιμη ανάλυση κατηγοριοποίησης κειμένων και συναισθήματος σε πολυγλωσσικά περιβάλλοντα

Abstract

dc:description

This thesis deals with the thorough study, application, and evaluation of modern natural language processing methods (Natural Language Processing – NLP), focusing in particular on Transformer models, deep learning techniques, hybrid schemes with classical classifiers, and explainability methods. The study is structured into nine independent but interrelated scientific papers, covering a wide range of research approaches and applications: from text categorization using the Zero-Shot method to sentiment analysis, classification of Greek-language data in real environments, and comparative evaluation of ensemble models. The experimental applications were mainly based on English-language datasets - such as Twitter US Airline Sentiment and the BBC Text Dataset - while three studies were applied to Greek data from the PSN Helpdesk, with the aim of categorizing requests and queries. The selection of different language datasets allowed for the evaluation of model generalization and the detection of limitations in less supported languages, such as Greek. One of the main findings of the thesis concerns the comparative performance of pre-trained Transformers models, such as DeBERTa, BERT, RoBERTa, DistilBERT, XLM-R, and GPT2, either in zero-shot scenarios or after fine-tuning, in combination with classical classifiers such as XGBoost or through aggregation mechanisms such as Majority Voting and Soft Voting. In cases such as multilingual classification of Greek queries, architectures combining Transformer models with TCN (Temporal Convolutional Networks) and additional attention layers were proposed, achieving significant improvement in all evaluation metrics. Furthermore, it is clearly shown that Majority Voting schemes consistently outperform Soft Voting in datasets with complexities such as irony, uncertainty, or category imbalance. The paper "From Transformers to Voting Ensembles for Interpretable Sentiment Classification" demonstrates, with strong statistical evidence (Friedman and Nemenyi tests), that the performance of many classifiers does not show statistically significant differences, which reinforces the need for evaluation based on the actual content and structure of each dataset and not just metric terms.11 Particular emphasis is placed on evaluating models on the same dataset, which allows for rigorous and methodologically consistent comparisons. An impressive finding of the study concerns the superiority of the Bi-LSTM model with LIME explainability in terms of accuracy (93.5%), F1-score (93.4%), and ROC AUC (98.2%), which exceeds the performance of advanced Transformer-based ensembles, such as GPT2 + T5 + Pythia (Accuracy: 91.7%). This finding reinforces the objections in the literature regarding the "self-evident" superiority of pre-trained models and highlights the need for context-sensitive architecture selection, depending on the nature of the data, the complexity of the problem, and transparency requirements. The contribution of the thesis is both theoretical and practical: it combines different experimental scenarios, offers evidence-based guidelines for the use and evaluation of NLP models in applications with high accuracy and interpretability requirements, and indicates the advantages and limitations of each approach.Of particular research interest is the observation that model performance may be influenced by the length of instances (text length), a factor that has not been sufficiently studied in the international literature. Future work may focus on systematically investigating the relationship between text length and model performance, in conjunction with parameters such as language complexity, category clarity, and decision interpretability. Overall, this thesis contributes significantly to the empirical documentation of the effectiveness of modern NLP methods, offering clear guidelines for model optimization, selecting architectures, and enhancing reliability and transparency in natural language understanding applications.

Degree

thesis:*
Grantor dc:publisher
University of Peloponesse
Year dc:date
2026

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Kyritsis, Konstantinos
  • Κυρίτσης, Κωνσταντίνος

Subjects

dc:subject × 38

Rights

Language dc:language
gre

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/61751

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Kyritsis, Konstantinos; Κυρίτσης, Κωνσταντίνος. Εξελιγμένα μοντέλα transformers, αλγόριθμοι μηχανικής μάθησης και τεχνικές συνόλου για ερμηνεύσιμη ανάλυση κατηγοριοποίησης κειμένων και συναισθήματος σε πολυγλωσσικά περιβάλλοντα. University of Peloponesse, 2026. http://hdl.handle.net/10442/hedi/61751