Institutional Repository of Vilnius University
Automatinis natūralios kalbos apdorojimas /
Abstract
dc:descriptionOne of the areas of natural language processing is text classification. This task includes text preprocessing, feature extraction, selection, and evaluation of classifiers’ results. It is known that selection of features is an important step, because it mainly affects the results. In the Bachelor thesis author analyses text classification process, history and research made on this topic. The main goal of the thesis is automatic Lithuanian literary texts classification by author. The dataset for this task contains texts written by seven different authors, each text contains on average 2200 words. Two programs have been developed for the best solution– one for data preprocessing, including steps such as lemmatization, tokenization, and another – for text classification, combining different features, such as average word and sentence lengths, function words, character and word n-grams, vocabulary richness, parts of speech, with feature selection methods and classifiers. Feature extraction for the described task includes Bag of Words and TF-IDF methods. As classifiers, Multinomial Naïve Bayes, Support Vector machine and Logistic regression models were used. The results have shown that the most suitable features for the given dataset are word unigrams combined with function words. While using Support vector machine classifier and TF-IDF feature selection together with these features, the highest of 0.978% accuracy was achieved.
Degree
thesis:*- Grantor dc:publisher
- Institutional Repository of Vilnius University
- Year dc:date
- 2021
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Maraulytė, Deimena,
- Contributors dc:contributor
-
- Dičiūnas, Valdas
Rights
dc:rights- Statement dc:rights
-
- info:eu-repo/semantics/openAccess
- Language dc:language
- lit
Identifiers
dc:identifier.*- Repository record dc:identifier
- https://repository.vu.lt/VU:ELABAETD107116992&prefLang=en_US