Back to search

Institutional Repository of Vilnius University

Teksto žinių perkėlimas natūralios kalbos apdorojimo uždaviniams /

Abstract

dc:description

This thesis analyses transformer architecture deep neural networks for natural language processing tasks and text knowledge transfer for Lithuanian language using these networks. During the experiment two transformer architecture models were created and trained using knowledge transfer methods. In order to test the performance of created models, they were fine-tuned to recognize authors of Lithuanian texts. The development of natural language processing is briefly reviewed then the literature about transformer deep neural networks is analysed. Thesis describes the features of the original transformer architecture: encoder, decoder, attention mechanism, input encodings. Then BERT architecture is described, similarities and differences with original transformer architecture. Knowledge transfer methods are also described. Experiments start by choosing the model architecture. The data needed for knowledge transfer is described. Data was collected from Directorate-General for Translation of the European Commission translation memory. Models were trained using knowledge transfer method. Created models were fine-tuned on author recognition task and their result were compared to a model trained from scratch. Also it’s shown how created models can be applied in semantic search. This work demonstrates that Lithuanian language model can be created by transferring knowledge from pretrained English language model without large corpus of data.

Degree

thesis:*
Grantor dc:publisher
Institutional Repository of Vilnius University
Year dc:date
2020

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Grotuzas, Domantas,
Contributors dc:contributor
  • Petkevičius, Linas

Rights

dc:rights
Statement dc:rights
  • info:eu-repo/semantics/openAccess
Language dc:language
lit

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
Vilnius University
Base URL
epublications.vu.lt/oai
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Grotuzas, Domantas,. Teksto žinių perkėlimas natūralios kalbos apdorojimo uždaviniams /. Institutional Repository of Vilnius University, 2020. https://repository.vu.lt/VU:ELABAETD81706922&prefLang=en_US