Back to search

Institutional Repository of Vilnius University

Didžiųjų kalbos modelių šališkumo ir etiškumo testavimas lietuvių kalba /

Abstract

dc:description

This bachelor's thesis aims to adapt large language model testing for ethics and bias methodology to the Lithuanian language. After conducting a literature review, the concepts of ethics and bias were defined, and the aspect of social bias was selected for the study. Legal aspects related to ethics and social bias in artificial intelligence systems were also reviewed. Afterwards, in this thesis, metrics and benchmarks for evaluating large language model bias and ethics are analyzed with the goal of selecting the most suitable methodology for this research. Based on the described methodologies for investigating social bias, the “StereoSet” benchmark was chosen. A methodology for evaluating ethics and bias in Lithuanian was developed, including the preparation of the dataset and evaluation metrics. LLM-as-a-judge was used to test bias and ethics. During the experiment both open-source and closed-source large language models were tested. Based on the results of the experiment conducted during this thesis, a Lithuanian dataset adapted for testing ethics and bias was prepared. The “DeepEval” framework metrics for Bias, Fairness, and Coherence, along with the “StereoSet” benchmark metrics, were checked and adapted for evaluation in Lithuanian. The results of the experiment confirmed the initial hypothesis of the thesis - that the methodology developed by foreign researchers is suitable for testing large language models in Lithuanian. However, further improvements are needed in future research.

Degree

thesis:*
Grantor dc:publisher
Institutional Repository of Vilnius University
Year dc:date
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Sirvidas, Rokas,

Rights

dc:rights
Statement dc:rights
  • info:eu-repo/semantics/openAccess
Language dc:language
lit

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
Vilnius University
Base URL
epublications.vu.lt/oai
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Sirvidas, Rokas,. Didžiųjų kalbos modelių šališkumo ir etiškumo testavimas lietuvių kalba /. Institutional Repository of Vilnius University, 2025. https://repository.vu.lt/VU:ELABAETD239032286&prefLang=en_US