Institutional Repository of Vilnius University
Gramatinės giminės signalo šalinimas kontekstualizuotuose žodžių įterpiniuose /
Abstract
dc:descriptionGrammatical gender, inherent in many languages, often leads to biases in natural language processing models. It has been shown in static word embeddings that grammatical gender causes word embeddings with the same gender to be closer to each other than to word embeddings with different genders. Grammatical gender also interferes with the semantic gender information captured in static word embeddings, referred to as the grammatical gender signal in this study. The aim of this research is to investigate the effects of disentangling the grammatical gender signal in contextualized word embeddings. This study utilizes a method proposed by M. Kaneko and D. Bollegala for this purpose, the method is further referred to as KB. By applying this method, the study evaluates the extent to which grammatical gender signals can be removed from word embeddings in the French and German language models, CamemBERT and GottBERT. The effectiveness of bias removal is assessed using SEAT and LogProb tests, comparing the semantic quality and bias levels before and after the intervention. The findings indicate that while the KB method successfully reduces gender bias in the French model, its effectiveness varies in the German model, sometimes even intensifying the bias. Despite the reduced semantic quality post-intervention, the adapted models still perform well in paraphrase and natural language inference tasks.
Degree
thesis:*- Grantor dc:publisher
- Institutional Repository of Vilnius University
- Year dc:date
- 2024
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Baikauskaitė, Viktorija,
Rights
dc:rights- Statement dc:rights
-
- info:eu-repo/semantics/openAccess
- Language dc:language
- lit
Identifiers
dc:identifier.*- Repository record dc:identifier
- https://repository.vu.lt/VU:ELABAETD210644921&prefLang=en_US