Back to search

Institutional Repository of Vilnius University

Po mokymo atliekamos kvantizacijos poveikis didelių kalbos modelių kodo generavimo kokybei ir efektyvumui /

Abstract

dc:description

This thesis examines the impact of post-training quantization (PTQ) on the quality and efficiency of code generation in large language models. The study analyses the Qwen/Qwen2.5Coder-7B-Instruct model and four of its GGUF-format variants: a control F16 model, a publicly available Q4_K_M model, and self-prepared Q4_K_M, Q3_K_M, and Q2_K quantized variants. The quality of the models was evaluated using the HumanEval benchmark and the EvalPlus library, generating 10 responses for each of the 164 tasks and computing the pass@1, pass@5, and pass@10 metrics. The model size on disk, peak GPU memory usage, and average generation time were also recorded. The study found that quantization caused a gradual decrease in pass@1 quality, from 82.20 % for the F16 model to 75.18 % for the Q2_K variant; however, the differences between models decreased significantly under the pass@10 metric. The best balance between quality and efficiency was achieved by the self-prepared Q4_K_M variant with imatrix calibration, which was 3.3 times smaller on disk, almost 6 times faster, and achieved a pass@10 score of 89.24 %. Contrary to the initial assumption, no clear quantization threshold beyond which code generation quality drops sharply was observed in this study – the degradation is gradual, and therefore the choice of quantization level is a matter of compromise between quality and resource consumption.

Degree

thesis:*
Grantor dc:publisher
Institutional Repository of Vilnius University
Year dc:date
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Kuolas, Normantas,

Rights

dc:rights
Statement dc:rights
  • info:eu-repo/semantics/openAccess
Language dc:language
lit

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
Vilnius University
Base URL
epublications.vu.lt/oai
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Kuolas, Normantas,. Po mokymo atliekamos kvantizacijos poveikis didelių kalbos modelių kodo generavimo kokybei ir efektyvumui /. Institutional Repository of Vilnius University, 2026. https://repository.vu.lt/VU:ELABAETD308337437&prefLang=en_US