Back to search

University of Southampton

Operationalising intelligibility within the CEFR-CV framework: A mixed methods approach combining Adaptive Comparative Judgment and Think Aloud Protocol

Abstract

dc:description.abstract

The constructs of intelligibility (listeners’ actual understanding) and comprehensibility (listeners’ perceived ease of understanding) are central in L2 pronunciation research, signalling a shift from emphasising nativeness to recognising English as a global Lingua Franca. Despite advances, precise definitions of these constructs remain elusive. The 2020 Companion Volume to the CEFR (CEFR-CV) redefined intelligibility by merging it with comprehensibility, but effective tools for assessing it are still lacking, which complicates its understanding. This study evaluates the efficacy of Adaptive Comparative Judgment (ACJ) in measuring CEFR-CV intelligibility. ACJ involves judges who compare speech samples in pairs and select those that best align with a holistic criterion. This process is enriched by integrating Think Aloud Protocols (TAPs), which reveal the linguistic features influencing assessments. These insights have guided the development of comprehensive CEFR-CV intelligibility constructs. To establish these constructs across diverse contexts, judges evaluated recordings from both controlled and uncontrolled tasks. Twelve Mandarin-speaking judges assessed 30 Mandarin L1 English L2 learners in the controlled task, while 12 mixed-L1 judges evaluated 30 mixed-L1 learners in the uncontrolled task. The integration of quantitative ACJ data with qualitative TAP analyses provided strong evidence of the effectiveness of ACJ. Analysis of the TAPs resulted in 1,526 and 917 codings from each task respectively, establishing the foundation for the CEFR-CV intelligibility constructs. The findings reveal that CEFR-CV intelligibility not only encompasses traditional pronunciation features but also broadens to include wider speaking assessment criteria. Constructs for controlled tasks include segmentals, suprasegmentals, fluency, grammar, discourse, and acceptability, while those for uncontrolled tasks additionally consider accentedness and lexis. These constructs provide a solid framework for L2 pronunciation assessment. This study not only opens new avenues for employing ACJ in L2 pronunciation assessment but also elucidates the intelligibility construct as defined by the CEFR-CV, offering significant implications for language teaching and testing practice in ELF contexts.

Degree

thesis:*
Name dc:type.qualificationname
Ph.D.
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
University of Southampton
Year dc:date.issued
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Wang, Jingwen
Advisors dc:contributor.advisor
  • Zheng, Ying
  • Dominguez, Laura

Rights

Language dc:language
English, eng

Chain of custody

source
Harvested from
University of Southampton
Base URL
eprints.soton.ac.uk/cgi/oai2
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Wang, Jingwen. Operationalising intelligibility within the CEFR-CV framework: A mixed methods approach combining Adaptive Comparative Judgment and Think Aloud Protocol. doctoral thesis, University of Southampton, 2025.