Back to search

Institutional Repository of Vilnius University

Paieška papildyto generavimo metodo taikymo poveikis didžiųjų kalbos modelių generuojamų integracinių testų kokybei /

Abstract

dc:description

This thesis examines how the Retrieval‑Augmented Generation method affects the quality of integration tests generated by large language models. Integration tests verify how different system components work together as a whole, so the model needs context that allows it to understand their dependencies. RAG can supply it by retrieving relevant code directly from the project. Whether and to what extent this actually improves integration test quality is what this work investigates. The experiment was conducted using a point‑of‑sale system. The study analysed 16 methods from three different services, with tests generated under three conditions: no additional context (baseline), semantic search in a vector database with a reranker (naive RAG), and structure‑driven retrieval from a code dependency graph (graph‑based RAG). Each condition was repeated five times with two models — Claude Sonnet 4.5 and DeepSeek V4 Pro. A total of 480 tests were generated and evaluated. Syntax correctness was near‑perfect across all conditions, but more notable differences emerged in compilation results. Graph‑based RAG achieved a 30.0% compilation rate — more than twice the baseline (12.5%) and better than naive RAG (21.3%). Compilation failures were mostly caused by missing type context rather than the model ignoring provided information. Execution success remained low across all conditions (7.5–8.8%), as models frequently failed to set up the initial database state correctly. Comparing both models across all conditions, the retrieval strategy had a greater effect on test quality than the choice of model.

Degree

thesis:*
Grantor dc:publisher
Institutional Repository of Vilnius University
Year dc:date
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Karalius, Rytis,

Rights

dc:rights
Statement dc:rights
  • info:eu-repo/semantics/openAccess
Language dc:language
lit

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
Vilnius University
Base URL
epublications.vu.lt/oai
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Karalius, Rytis,. Paieška papildyto generavimo metodo taikymo poveikis didžiųjų kalbos modelių generuojamų integracinių testų kokybei /. Institutional Repository of Vilnius University, 2026. https://repository.vu.lt/VU:ELABAETD308337213&prefLang=en_US