Back to search

Schulich School of Engineering

Automated Test Case Generation Using Transformers and Domain Adaptation

Abstract

dc:description.abstract

Software testing is an important part of the software development cycle. It helps prevent future potential defects in the system and reduce maintenance cost. However, it is often neglected by developers like other tasks such as documentation, code review, etc. Automated unit test generation is the process of generating unit tests for a method or a class for a given project. It helps developers to detect some of the potential defects automatically. Existing approaches usually use heuristic methods such as random data generation (fuzzing) or genetic algorithms (search-based testing) to generate tests, which are based on optimizing a cost function such as code coverage. Some of the main downsides of these methods are (a) the synthetic nature of the tests that are generated based on predefined templates and (b) targeting surrogate measures such as coverage, which might not always detect existing defects. In this study, we leverage novel machine translation models, i.e., transformers to generate unit tests for a given program. Specifically, we use CodeT5, i.e., a state-of-the-art large code model, and fine-tune it on the test generation downstream task. We use the Methods2test dataset for fine-tuning CodeT5. We also use Defects4j for project-level domain adaptation and evaluation. The main contribution of this study is proposing a fully automated testing framework that leverages developer-written tests and available code models to generate compilable unit tests. Results show that using domain adaptation, we can increase line coverage of the model-generated unit tests by 49.9% and 54% in terms of mean and median (compared to the model without domain adaptation). We can also use our framework as a complementary solution alongside common search-based methods to increase the overall coverage with mean and median of 25.3% and 6.3%. It can also increase the mutation score of search-based methods with mean and median of 8.6% and 1%.

Degree

thesis:*
Name thesis:degree_name
Master of Science (MSc)
Discipline thesis:degree_discipline
Engineering – Electrical & Computer
Grantor dc:publisher.institution
Schulich School of Engineering
Year dc:date.issued
2023

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Hashtroudi, Sepehr Pourabolfath
Advisor dc:contributor.advisor
  • Hemmati Hadi
Committee members dc:contributor.committeemember
  • Ioannou Yani
  • Karimipour Hadis

Subjects

dc:subject × 4

Rights

dc:rights
Statement dc:rights
  • University of Calgary graduate students retain copyright ownership and moral rights for their thesis. You may use this material in any way that is permitted by the Copyright Act or through licensing that has been assigned to the document. For uses that are not allowable under copyright legislation or licensing, you are required to seek permission.
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Citation
Hashtroudi, S. P. (2023). Automated test case generation using transformers and domain adaptation (Master's thesis, University of Calgary, Calgary, Canada). Retrieved from https://prism.ucalgary.ca.

Chain of custody

source
Harvested from
University of Calgary
Base URL
ucalgary.scholaris.ca/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Hashtroudi, Sepehr Pourabolfath. Automated Test Case Generation Using Transformers and Domain Adaptation. Schulich School of Engineering, 2023. http://hdl.handle.net/1880/116140