Back to search

McMaster University

Putting Software Testing Terminology to the Test

Abstract

dc:description.abstract

Despite the prevalence and importance of software testing, it lacks a standardized and consistent taxonomy, instead relying on a large body of literature with many flaws—even within individual documents! This hinders precise communication, contributing to misunderstandings when researching, planning, and performing testing. In this thesis, we explore the current state of software testing terminology by: 1. identifying established standards and prominent testing resources, 2. capturing relevant testing terms from these sources, along with their definitions and relationships (both explicit and implicit), and 3. constructing visualizations to analyze these data. This process uncovered 567 test approaches and four in-scope methods for deriving test approaches, such as those related to 75 software qualities. We also manually record flaws as they arise and build tools to detect more flaws automatically, analyze all of our recorded flaw data, and visualize the relations between test approaches. This revealed 344 flaws, including 13 terms used as synonyms to two (or more) disjoint test approaches and 17 pairs of test approaches that may either be synonyms or have a parent-child relationship. We also found notable confusion surrounding functional testing, operational acceptance testing, recovery testing, and scalability testing. Our findings make clear the urgent need for improved testing terminology so that the discussion, analysis, and implementation of various test approaches can be more coherent.

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Crawford, Samuel

Subjects

dc:subject × 6

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial 4.0 International
Language dc:language.iso
en

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
McMaster University
Base URL
macsphere.mcmaster.ca/server/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Crawford, Samuel. Putting Software Testing Terminology to the Test. 2026. https://hdl.handle.net/11375/32815