Institutional Repository of Vilnius University
Konteksto triukšmo įtaka atsakymų kokybei teisinių dokumentų RAG sistemose /
Abstract
dc:descriptionRetrieval-Augmented Generation (RAG) enables large language models to generate responses grounded in external document sources, however, the performance of such systems strongly depends on the quality of the provided context. One of the main challenges in RAG systems is context noise - irrelevant or semantically misleading document fragments that may negatively affect response quality. This issue is particularly important in the legal domain, where semantically similar fragments may carry substantially different legal meanings. The aim of this thesis is to investigate the impact of context noise quantity and type on answer quality in legal-document RAG systems. To achieve this, a controlled experimental RAG environment was developed in which automatic document retrieval was intentionally omitted in order to isolate the effect of context noise from retrieval errors. The experiments were conducted using the CUAD dataset and three types of noise were constructed: random, topical, and adversarial noise. Context quality was controlled by varying the Signal-to-Noise Ratio (SNR). Responses were generated using the gpt-4o-mini model and evaluated with the RAGAS evaluation framework. The experimental results demonstrated that decreasing the SNR reduces both response quality and context precision, while adversarial noise produced the strongest negative impact in most scenarios. The study also revealed that different types of legal questions exhibit varying sensitivity to context noise: short and specific questions were more sensitive to semantically similar fragments than broader provisions requiring wider contextual understanding. The results indicate that the effectiveness of RAG systems depends not only on the quantity of retrieved information, but also on its semantic characteristics. These findings may contribute to the development of more reliable RAG systems for legal document analysis and more effective context construction strategies.
Degree
thesis:*- Grantor dc:publisher
- Institutional Repository of Vilnius University
- Year dc:date
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Katinaitė, Patricija,
Rights
dc:rights- Statement dc:rights
-
- info:eu-repo/semantics/openAccess
- Language dc:language
- lit
Identifiers
dc:identifier.*- Repository record dc:identifier
- https://repository.vu.lt/VU:ELABAETD308337289&prefLang=en_US