Back to search

Carleton University

Bidirectional Generation of Behavior-Driven Development Artifacts Using Fine-Tuning and Retrieval-Augmented Generation

Abstract

dc:description.abstract

This thesis investigates bidirectional generation between semi-structured records and structured Behavior-Driven Development (BDD) scenarios using Large Language Models (LLMs). It compares fine-tuning (CodeT5+ and Qwen2.5-Coder) with Retrieval-Augmented Generation (RAG) (Meta-Llama-3.1-8B, DeepSeek-Coder) across a curated dataset of 2,100 aligned pairs. Results demonstrate that generation quality is heavily influenced by task direction and context management. Fine-tuned models significantly outperformed RAG-based methods, achieving F1 scores up to 0.9549 and Exact Match scores of 0.8119. RAG struggled to maintain the strict structural fidelity required for Gherkin-based BDD artifacts, failing to achieve exact matches. These findings suggest that fine-tuning remains the most reliable strategy for generating technically accurate BDD artifacts, as it better handles the rigid structural constraints of automated testing specifications compared to in-context prompting alone.

Degree

thesis:*
Name thesis:degree_name
Master of Applied Science (M.App.Sc.)
Level thesis:degree_level
Master's
Discipline thesis:degree_discipline
Engineering, Electrical and Computer
Grantor dc:publisher
Carleton University
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Biniek, Artin

Subjects

dc:subject × 4

Rights

dc:rights
Statement dc:rights
  • Copyright © 2026 the author(s). Theses may be used for non-commercial research, educational, or related academic purposes only. Such uses include personal study, distribution to students, research and scholarship. Theses may only be shared by linking to the Carleton University Institutional Repository and no part may be copied without proper attribution to the author; no part may be used for commercial purposes directly or indirectly via a for-profit platform; no adaptation or derivative works are permitted without consent from the copyright owner.
Language dc:language.iso
en

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
Carleton University
Base URL
carleton.scholaris.ca/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Biniek, Artin. Bidirectional Generation of Behavior-Driven Development Artifacts Using Fine-Tuning and Retrieval-Augmented Generation. Master's thesis, Carleton University, 2026. https://hdl.handle.net/20.500.14718/45261