Back to search

University of British Columbia

Merging task-specific adapters in code LLMs for automated program repair

Abstract

dc:description

Large Language Models (LLMs) have shown good performance in several software development related tasks such as program repair, documentation, code refactoring, debugging, and testing. Adapters are specialized, small modules designed for parameter efficient fine-tuning of LLMs for specific tasks, domains, or applications without requiring extensive retraining of the entire model. These adapters offer a more efficient way to customize LLMs for particular needs, leveraging the pre existing capabilities of the large model. Merging LLMs and adapters has shown promising results for various natural language domains and tasks, enabling the use of the learned models and adapters without additional training for a new task. This research proposes continual merging and empirically studies the capabilities of merged adapters in Code LLMs, specially for the Automated Program Repair (APR) task. The goal is to gain insights into whether and how merging task-specific adapters can affect the performance of APR. In our framework, MergeRepair, we merge multiple task-specific adapters using three different merging methods and evaluate the performance of the merged adapter for the APR task. Particularly, we employ two main merging scenarios for all three techniques, (i) merging using equal-weight averaging applied on parameters of different adapters, where all adapters are of equal importance; and (ii) our proposed approach, continual merging, in which we sequentially merge the task-specific adapters and the order and weight of merged adapters matter. By exploratory study of merging techniques, we investigate the improvement and generalizability of merged adapters for APR. Through continual merging, we explore the capability of merged adapters and the effect of task order, as it occurs in real-world software projects. The results show that we can gain performance improvement on APR benchmark by merging APR adapter with other similar task adapters. Additionally, we can observe the effectiveness of the continual merging approach if the order of the merged tasks is selected carefully.

Degree

thesis:*
Name thesis:degree_name
Master of Science - MSc
Level thesis:degree_level
master's
Discipline thesis:degree_discipline
Computer Science
Grantor dc:publisher
University of British Columbia
Year dc:date
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Dehghan, Meghdad
Contributors dc:contributor
  • Fard, Fatemeh Hendijani

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-NoDerivatives 4.0 International
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/91356

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Dehghan, Meghdad. Merging task-specific adapters in code LLMs for automated program repair. master's thesis, University of British Columbia, 2025. http://hdl.handle.net/2429/91356