Carleton University
Multi-Label Ambiguity Detection in Software Requirements using Language Models
Abstract
dc:description.abstractSoftware requirements capture stakeholder needs; however, they are typically written in natural language, which often contains ambiguity. Identifying ambiguity helps development teams prevent misunderstandings, address defects earlier, and reduce project costs. While recent studies have focused on detecting only one or two types of ambiguity, this research investigates whether language models can detect multiple types, including structural ambiguity, scope ambiguity, anaphoric ambiguity, and vagueness. We collected 1,266 real-world requirements from multi-domain public datasets and labeled them using previous methods. Two language models, DistilBERT and RoBERTa, were fine-tuned on the labeled dataset and evaluated on 255 manually annotated requirements. RoBERTa achieved superior performance, obtaining a macro-averaged F1-score of 75.61%, whereas DistilBERT scored 73.15%. Back-translation data augmentation improved RoBERTa's F1-score to 78.84%. Compared to existing methods, our multi-label model achieved approximately 12% and 7% higher F1-scores for scope and structural ambiguity detection, respectively, with slightly lower performance for anaphoric ambiguity and vagueness detection.
Degree
thesis:*- Name thesis:degree_name
- Master of Applied Science (M.App.Sc.)
- Level thesis:degree_level
- Master's
- Discipline thesis:degree_discipline
- Data Science, Analytics, and Artificial Intelligence
- Grantor dc:publisher
- Carleton University
- Year dc:date.issued
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Oghabi, Paniz
Subjects
dc:subject × 2Rights
dc:rights- Statement dc:rights
-
- Copyright © 2025 the author(s). Theses may be used for non-commercial research, educational, or related academic purposes only. Such uses include personal study, distribution to students, research and scholarship. Theses may only be shared by linking to the Carleton University Institutional Repository and no part may be copied without proper attribution to the author; no part may be used for commercial purposes directly or indirectly via a for-profit platform; no adaptation or derivative works are permitted without consent from the copyright owner.
- Language dc:language.iso
- en