Institutional Repository of Vilnius University
Lietuviško teksto dalinimo sakiniais algoritmo analizė ir mokymas /
Abstract
dc:descriptionThe main goal of the thesis is to create a tool for dividing Lithuanian text into sentences, using the algorithm of the NLTK library and Lithuanian corpora. The following tasks were being solved: 1. To analyze the literature related to splitting the text into sentences. 2. Analyze the NLTK library algorithms used for text splitting. 3. Collect Lithuanian corpora intended for the training of the algorithm for splitting the text into sentences. 4. Train the algorithm. 5. Evaluate the test results. In the literature analysis, the concept of text segmentation, various methods and techniques of text division are explored. The NLTK library, a widely used natural language processing toolkit, is analyzed to understand existing text segmentation algorithms. This analysis helps to determine the strengths and weaknesses of the available tools. To ensure the availability of suitable data for the training of the algorithm, a collection of Lithuanian language texts is undertaken. This task involves data acquisition, cleaning and pre-processing to create a diverse and representative corpus. The quality and relevance of the data collected are critical to the success of the training process. Using the collected text, an algorithm for splitting the Lithuanian text into sentences is trained. This task also involves iteratively improving the performance of the model through experimentation and refinement. Finally, the developed algorithm is tested and evaluated. The evaluation process involves measuring key metrics such as accuracy and computational efficiency. The evaluation results provide insights into the efficiency and reliability of the algorithm, highlighting its strengths and potential areas for improvement. Keywords: natural language processing, unsupervised machine learning, text segmentation.
Degree
thesis:*- Grantor dc:publisher
- Institutional Repository of Vilnius University
- Year dc:date
- 2023
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Česnauskis, Justas,
Rights
dc:rights- Statement dc:rights
-
- info:eu-repo/semantics/openAccess
- Language dc:language
- lit
Identifiers
dc:identifier.*- Repository record dc:identifier
- https://repository.vu.lt/VU:ELABAETD192842241&prefLang=en_US