National and Kapodistrian University of Athens
Exploring listening comprehension difficulty in language proficiency testing: the case of the greek state certificate examination
Abstract
dc:descriptionThis research is concerned with the assessment of listening comprehension, a rather neglected area in the field of foreign language testing. In particular, it addresses the issue of listening comprehension test difficulty and seeks to provide empirical evidence in terms of the specific factors i.e., item-related, text-related, linguistic and non-linguistic, contributing to task and text complexity. As an outcome, a Task Difficulty Typology is proposed involving detailed descriptors of item and text difficulty. The study is conducted in the context of the Greek national language proficiency examination suite, a multilingual exam suite, which is known by the Greek acronym KPG (ΚΠΓ). The data (i.e., test items and audio texts) investigated for the present study is retrieved from the listening comprehension test papers of the B2 level English exam. In particular two main item types are examined, namely a) choice test items (i.e., multiple choice and true/false/not stated) and b) completion test items (i.e., fill-in or short responses). Two general directions of research have been followed: one investigating the test method effect on item response difficulty and the other investigating task and text complexity from the listener’s perspective. As regards the former, by drawing on post-administration test data showing item difficulty, the study offers a systematic description of item-related and item-by-text related variables associated with ‘problematic’ listening test items demonstrating high levels of difficulty. In relation to the second direction of research, i.e., the investigation of the listener’s perspective, quantitative as well as qualitative data on task and text difficulty is considered as elicited from actual test takers and learners of English. More specifically, KPG candidates’ perceptions of listening comprehension text and task difficulty have been investigated through the use of (post-administration) feedback questionnaires especially designed for the B2 level English exams. The findings of the questionnaire data revealed the combined effect that specific text linguistic or paralinguistic features (e.g., information density, lexical difficulty, rate of speech,) and topic familiarity or interest can have on perceived test difficulty as well as their possible correlations with actual test scores.The qualitative data of the study has been collected through implementation of the verbal report method, in the form of semi-structured interviews and self-perception open ended questionnaires. To this end, two different groups of learners have been used, namely a controlled group of undergraduate students of the Faculty of English Language and Literature, University of Athens, and a random group of actual B2 learners of English. Learners’ verbal reports provided explanations in terms of the difficulties encountered. In this sense, they gave useful insights into the cognitive and processing difficulties entailed as a result of specific item variables and text-related factors having a joint effect on understanding. In the end, it was through triangulation, namely by associating the findings drawn from analysis of test data with the listener’s perspective, that the Task Difficulty Typology was developed.The findings of the study reflect the complex interaction of task design factors (i.e., item and text difficulty variables), cognitive attributes and processing abilities in listening comprehension test difficulty. In doing so, they reflect the complexity of listening comprehension test difficulty. By providing detailed descriptors of item-related and text-related difficulty factors, the Task Difficulty Typology could serve as an inventory for item writers and test developers to produce future listening comprehension tests by taking into consideration a range of variables that are associated with task or text difficulty, ensuring in this way overall test validity and reliability. Consequently, it will prove of practical use to language proficiency examination suites in which a listening comprehension test paper is involved. Besides language proficiency testing, the findings of this research may also benefit FL pedagogy serving as a source of information for language teachers who make a great effort to enhance their students’ listening comprehension abilities but are not fully aware of the factors that influence their performance. In particular, the Typology could serve as a tool for test strategy instruction in the EFL classroom as a means of overcoming item- or text-related difficulties.
Degree
thesis:*- Grantor dc:publisher
- National and Kapodistrian University of Athens
- Year dc:date
- 2014
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Apostolou, Elisabeth
- Αποστόλου, Ελισάβετ
Subjects
dc:subject × 18- Κατανόηση προφορικού λόγου
- Παράγοντες δυσκολίας
- Ερωτήματα
- Ηχητικά κείμενα
- Γλωσσικά χαρακτηριστικά
- Παρα-γλωσσικά χαρακτηριστικά
- Νοητικές διεργασίες
- Listening comprehension
- Difficulty variables
- Test items
- Oral texts
- linguistic features
- Paralinguistic characteristics
- Cognitive processes
- Ανθρωπιστικές Επιστήμες και Τέχνες
- Γλώσσα και Λογοτεχνία
- Humanities and the Arts
- Languages and Literature
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/38839