Institutional Repository of Vilnius University
Mašininio mokymo algoritmų taikymas klasifikuojant IT pagalbos užklausas /
Abstract
dc:descriptionThis bachelor's thesis presents a comparative analysis aimed at partially automating the classification of IT support tickets submitted in mostly the Lithuanian language. The primary objective of the research is to evaluate and compare the efectiveness of different technological paradigms within the IT sector while accounting for the morphological complexity of Lithuanian language and the challenged posed by data imbalance. Three models are examined in the study: the probabilistic Multinomial Naive Bayes classifier, the geometric Linear Support Vector Machine (LinearSVC) and the deep learning-based multilingual BERT (mBERT) transformer model. The analysis of preprocessing effects revealed that lemmatization is essential for simple statistical models like Naive Bayes to reduce feature space sparsity. An anomaly was also identified where lemmatization in Lithuanian improved performance even when using the mBERT transformer model. The LinearSVC architecture on the other hand excelled by leveraging the diversity of word endings as additional contextual information, achieving a weighted F1 score of 0.854 with minimal preprocessing. The most advanced model used in the study, mBERT, demonstrated the capacity to understand complex semantic structures and queries in English. However, the research uncovered an overconfidence problem where the model produced incorrect predictions with extremely high confidence scores. It was also found that the examined architectures struggle to identify small categories due to imbalanced datasets. The conclusions provide practical recommendations for model selection in production environments, highlighting the efficiency and stability of LinearSVC alongside mBERT's contextual understanding. Furthermore, calibration tools are suggested that were not tested during the course of this study.
Degree
thesis:*- Grantor dc:publisher
- Institutional Repository of Vilnius University
- Year dc:date
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Mažvila, Mantas,
Rights
dc:rights- Statement dc:rights
-
- info:eu-repo/semantics/openAccess
- Language dc:language
- lit
Identifiers
dc:identifier.*- Repository record dc:identifier
- https://repository.vu.lt/VU:ELABAETD277691017&prefLang=en_US