Institutional Repository of Vilnius University
Daugiaagentinių debatų įtaka lokalių didelių kalbos modelių (LLM) kodo generavimo efektyvumui skirtingo sudėtingumo užduotyse /
Abstract
dc:descriptionLarge language models are increasingly used for automatic code generation, but the closed commercial models that achieve the best results require paid services and send user code to third parties. Open-weight 7-9 billion parameter models can be run locally, yet on harder tasks they often produce incorrect, incomplete or poorly structured code. This work studies whether multi-agent debate, where several models iteratively propose, critique and revise each other's solutions, can narrow this gap. A modular multi-agent debate system tailored to 7-9 billion parameter models was designed and implemented. The whole system spans about 27 thousand lines (Python code, a web interface template, HPC scripts and JSON task descriptions), and its core code is split into six modular packages. The system uses a four-phase debate cycle: initial proposal, critique, revision and voting. To address the weaknesses of smaller models, several mechanisms were implemented: an anti-regression policy, adaptive temperature, critique history and chunked generation of multi-file tasks. A set of 90 unique tasks across four difficulty levels was created, including 40 multi-file extreme-level tasks designed specifically for this work. The experiments were run on the VU MIF high-performance computing cluster. The main stage included eleven SLURM jobs and produced 960 execution records, and an additional adaptive temperature A/B ablation added another 240 records. In total 15 SLURM jobs and 1200 execution records were used. Three factors were compared: proposer pool composition (7B and 9B class), judge model type, and the difference between debate and single-model modes. The results showed that multi-agent debate stably outperformed the single-model baseline by 27.3 percentage points: across 120 paired comparisons debate won 108, tied 12 and lost none. On tasks where the initial solution completely failed, debate reached at least a partially working solution in 90.1% of cases. The hypothesis that a larger 9B model pool would outperform the 7B pool was not confirmed - on the hardest tasks the more diverse 7B pool even surpassed the more homogeneous 9B one. Adding a judge agent did not improve the debate result by itself, and a code-specialised judge even worsened the strongest pool. The most important factor behind improvement turned out to be not the amount of criticism, but its conversion into concrete code fixes.
Degree
thesis:*- Grantor dc:publisher
- Institutional Repository of Vilnius University
- Year dc:date
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Magalinski, Lukaš Patrik,
Rights
dc:rights- Statement dc:rights
-
- info:eu-repo/semantics/openAccess
- Language dc:language
- lit
Identifiers
dc:identifier.*- Repository record dc:identifier
- https://repository.vu.lt/VU:ELABAETD308337536&prefLang=en_US