Back to search

University of British Columbia

Hey, ChatGPT, look at my work : using conversational AI in requirements engineering education

Abstract

dc:description

The emergence of conversational AI tools in late 2022 practically changed the face of software engineering and software engineering education. Contemplating the question of how to best prepare and evaluate students in this new reality, we experimented with systematically introducing a conversational AI tool, ChatGPT, into the 2023 offering of an upper-level undergraduate project-based course on Software Engineering. In this course, 20 groups of four students each had to design and implement a project of their choice, with an Android-based mobile client and a Node.js-based cloud server. This thesis discusses our goals, approach, and lessons learned from introducing ChatGPT into the first phase of the project development: scoping the work and defining the project requirements. Our experience shows that students can achieve comparable results using a variety of ChatGPT interaction modes and the success of each mode largely depends on students’ preferences, learning styles, and the invested effort. Yet, in any of the modes, with moderate effort, students can produce artifacts of a mid-range quality level of around 80%. Moving above this range requires substantial investment, which can be spent on brainstorming, crafting high-quality prompts, or critically assessing ChatGPT’s output. We also observe low prompting proficiency of the students: students can improve their prompting strategies by providing a more adequate description of their course and project setup, examples, and expected output format for their requests. Interestingly, students can often be “swayed” by ChatGPT’s projected confidence, even when their original ideas are, in fact, more appropriate than the proposed refinements. In a follow up study, we further evaluate the latest version of ChatGPT on the original students’ prompts, and discover that while ChatGPT can produce artifacts that are better in some attributes, the results are still mediocre and would need substantial effort to get 95%+ scoring artifacts. We hope that our experience and lessons learned will help spark further discussions on how to best embed AI tools into the software engineering curriculum and work practices.

Degree

thesis:*
Name thesis:degree_name
Master of Applied Science - MASc
Level thesis:degree_level
master's
Discipline thesis:degree_discipline
Electrical and Computer Engineering
Grantor dc:publisher
University of British Columbia
Year dc:date
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Tegegn, Michael
Contributors dc:contributor
  • Rubin, Julia

Rights

dc:rights
Statement dc:rights
  • Attribution-NoDerivatives 4.0 International
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/93147

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Tegegn, Michael. Hey, ChatGPT, look at my work : using conversational AI in requirements engineering education. master's thesis, University of British Columbia, 2025. http://hdl.handle.net/2429/93147