Institutional Repository of Vilnius University
Anglų kalbos naujienų straipsnių klasterizavimas taikant šiuolaikinius teksto įterpinių modelius /
Abstract
dc:descriptionThere has been an increase of articles published on the internet. Publishers are facing trouble processing all the incoming information to decide what are the best topics to write about. Large–scale web content can be systematically clustered, filtered, and analyzed to extract core themes, helping editorial teams decision–making process. The goal of this study is to conduct a wide analysis of article clustering pipeline components. This involves: text data preprocessing, text embeddings, dimension reduction techniques and clustering algorithms evaluating them with geometric and semantic cluster metrics. Study consists of five experimental stages performed on 51433 English language news articles. In total 1662 configurations were evaluated. The optimal configuration for clustering English news articles was achieved with Google gemini-embedding-001 model with text lemmatization, UMAP dimensionality reduction and HDBSCAN clustering algorithm, a coherence cv = 0.8215 was achieved. LLM based embedding methods outperformed classical statistical methods. Testing dimensionality reduction algorithms it was noticed, that nonlinear methods (UMAP, PaCMAP) produced better results than linear ones. During the study it was observed, that geometric and semantic coherence metrics do not show any significant correlation.
Degree
thesis:*- Grantor dc:publisher
- Institutional Repository of Vilnius University
- Year dc:date
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Vailenka, Laimonas,
Rights
dc:rights- Statement dc:rights
-
- info:eu-repo/semantics/openAccess
- Language dc:language
- lit
Identifiers
dc:identifier.*- Repository record dc:identifier
- https://repository.vu.lt/VU:ELABAETD308335149&prefLang=en_US