Institutional Repository of Vilnius University
Naratyvų generavimas iš paveikslėlių naudojantis transformerių neuroniniais tinklais /
Abstract
dc:descriptionThis thesis analyzes transformer neural networks and their application to narrative generation from images. In this thesis' context, a narrative is a descriptive caption of an image that consists of one or more sentences. The thesis explains improvements in both computer vision and natural language processing during the last few years. It presents transformer neural networks and two of their variations - BART transformer and the newly proposed vision transformer which performs better in image classification than convolutional neural networks. The thesis proposes a new model for narrative generation with an architecture that is only made of transformer parts. The model is a transformer which uses a vision transformer as its encoder and BART's decoder as a decoder. The model is trained on the Google Open Images localized narrative dataset, finetuning both the encoder and the decoder to solve this task. This results in a model that can generate a relatively accurate narrative from an image. This thesis proves that it is possible to create a model for narrative creation which is not dependent on the outputs of a convolutional neural network like current state-of-the-art image captioning models.
Degree
thesis:*- Grantor dc:publisher
- Institutional Repository of Vilnius University
- Year dc:date
- 2021
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Rutkauskas, Karolis,
- Contributors dc:contributor
-
- Petkevičius, Linas
Rights
dc:rights- Statement dc:rights
-
- info:eu-repo/semantics/openAccess
- Language dc:language
- lit
Identifiers
dc:identifier.*- Repository record dc:identifier
- https://repository.vu.lt/VU:ELABAETD107115676&prefLang=en_US