The study aims to investigate the fundamental validity issues that can affect the DR Congo English state examination, a national exit test administered to high school final year students for certification. The study aspires to generate an understanding of the potential issues that affect the construct validity of a test within the epistemological stance that supports a strong relationship between test construct and test context. The study draws its theoretical underpinning from three theories: the validity theory that provides a theoretical ground necessary for understanding the quality of tests needed for assessing students’ reading abilities / the construction-integration theory that provides an understanding of how texts used in reading assessments are processed and understood by the examinees / and the strategic competence theory that explains how examinees deploy strategies to complete test tasks, and the extent to which these strategies tap into the reading construct. Furthermore, the study proposes a reading model that signposts the social context of testing / therefore, conceptualizing reading as both a cognitive and a social process. As research design, the study adopts an exploratory design using both qualitative and quantitative data. Besides, the study uses protocol analysis and content analysis methodologies. While the former provides an understanding of the cognitive processes that mediate the reading construct and test performance so as to explore the different strategies examinees use to answer the English state examination (henceforth termed ESE) test questions, the latter examines the content of the different ESE papers so as to identify the different textual and item features that potentially impact on examinees’ performance on the ESE tasks. As instruments, the study uses a concurrent strategies questionnaire administered to 496 student-participants, a contextual questionnaire administered to 26 student-participants, a contextual questionnaire administered to 27 teacher-articipants, and eight tests administered to 496 student-participants. The findings indicate that, the ESE appears to be less appropriate to the ESE context as the majority of ESE test items target careful reading than expeditious reading / on the one hand, and reading at global level than to reading at local level / on the other hand. The findings also indicate that the ESE tasks hardly take account of the text structure and the underlined cognitive demands appropriate to the text types. Besides, the ESE fails to include other critical aspects of the reading construct. Finally, the findings also indicate that the ESE constructors may not be capable to construct an ESE with five functioning distractors as expected. Moreover, the inclusion of the implicit option 6 overlaps with the conceptual meaning of this option. The entire process of the present study has generated some insights that can advance our understanding of the construct validity of reading tests. These insights are: (a) the concept of validity is an evolving and context-dependent concept, (b) reading construct cannot be examined outside the actual context of reading activity, (c) elimination of distractors can sometimes be a construct-relevant strategy, (d) construct underrepresentation is a context-dependent concept, and (e) a reading test cannot be valid in all contexts. The suggested proposal for the improvement of the ESE requires the Congolese government through its Department of Education to (a) always conduct validation studies to justify the use of the ESE, (b) always consider the actual context of reading activity while developing the ESE, (c) revisit the meanings and interpretations of the ESE scores, (d) ensure the appropriateness of tasks to be included in the ESE, (e) ensure the construct representativeness of the ESE tasks, (f) revisit the number of questions to be included in the ESE, (g) avoid bias in the ESE texts in order to ensure fairness, (h) diversify the genres of ESE texts, (i) ensure the coherence of ESE texts through the use of transitions and cohesive devices, (j) ensure that the order of test questions is in alignment with the order of text information, (k) revisit the structure and length of the texts to be included in the ESE, (l) revisit the number of alternatives to be included in the ESE, and (m) reconsider the use of the implicit alternative 6.

<p>The test context is of paramount importance in language testing as it provides an understanding of the kind of tasks to be included in the test, how these tasks are executed by the test takers and how they can be efficiently administered. The objective of this study was to investigate the extent to which the context of the DR Congo English state examination (ESE) is valid and to come out with some useful suggestions that are likely to improve its validity. Two basic theories, the modern validity theory and the schema theory, informed this study. Weir‟s (2005) socio-cognitive framework was used to build the validity argument for the evaluation of the English state examination.</p> <p>A mixed method was used where the research design consisted of the combination of both qualitative and quantitative data during the collection and analysis stages. The content document analysis method was used to examine the content of the different state examination papers so as to identify the main features of the test, and the statistic (descriptive) method was used to quantify observations identified in the state examination papers and to evaluate the context validity of the ESE. Three techniques were used to collect the research data: the questionnaire, the<br /> test, and the interview.</p> <p>Three main findings of this study were reported: (1) the conditions under which the ESE tasks are performed and the relevance of these tasks to the test domain and characteristics are still far to contribute to the quality of evaluation of high school finalist students / (2) the extent to which the ESE includes tasks that take into consideration the nature of information in the text as well as the knowledge required for completing the task is globally good / (3) the conditions under which the test takes place are poor and these conditions affect the validity of test scores. The study recommends the test developers to approximate test tasks to those students have been exposed to in classroom situations and those they are likely to encounter in real life. It also recommends all the people involved in the administration of the test to adhere to high ethical standards.</p>

<p>The test context is of paramount importance in language testing as it provides an understanding of the kind of tasks to be included in the test, how these tasks are executed by the test takers and how they can be efficiently administered. The objective of this study was to investigate the extent to which the context of the DR Congo English state examination (ESE) is valid and to come out with some useful suggestions that are likely to improve its validity. Two basic theories, the modern validity theory and the schema theory, informed this study. Weir‟s (2005) socio-cognitive framework was used to build the validity argument for the evaluation of the English state examination.</p> <p>A mixed method was used where the research design consisted of the combination of both qualitative and quantitative data during the collection and analysis stages. The content document analysis method was used to examine the content of the different state examination papers so as to identify the main features of the test, and the statistic (descriptive) method was used to quantify observations identified in the state examination papers and to evaluate the context validity of the ESE. Three techniques were used to collect the research data: the questionnaire, the<br /> test, and the interview.</p> <p>Three main findings of this study were reported: (1) the conditions under which the ESE tasks are performed and the relevance of these tasks to the test domain and characteristics are still far to contribute to the quality of evaluation of high school finalist students / (2) the extent to which the ESE includes tasks that take into consideration the nature of information in the text as well as the knowledge required for completing the task is globally good / (3) the conditions under which the test takes place are poor and these conditions affect the validity of test scores. The study recommends the test developers to approximate test tasks to those students have been exposed to in classroom situations and those they are likely to encounter in real life. It also recommends all the people involved in the administration of the test to adhere to high ethical standards.</p>

Magister Educationis - MEd / The test context is of paramount importance in language testing as it provides an understanding of the kind of tasks to be included in the test, how these tasks are executed by the test takers and how they can be efficiently administered. The objective of this study was to investigate the extent to which the context of the DR Congo English state examination (ESE) is valid and to come out with some useful suggestions that are likely to improve its validity. Two basic theories, the modern validity theory and the schema theory, informed this study. Weir's (2005) socio-cognitive framework was used to build the validity argument for the evaluation of the English state examination. A mixed method was used where the research design consisted of the combination of both qualitative and quantitative data during the collection and analysis stages. The content document analysis method was used to examine the content of the different state examination papers so as to identify the main features of the test, and the statistic (descriptive) method was used to quantify observations identified in the state examination papers and to evaluate the context validity of the ESE. Three techniques were used to collect the research data: the questionnaire, the test, and the interview. Three main findings of this study were reported: (1) the conditions under which the ESE tasks are performed and the relevance of these tasks to the test domain and characteristics are still far to contribute to the quality of evaluation of high school finalist students; (2) the extent to which the ESE includes tasks that take into consideration the nature of information in the text as well as the knowledge required for completing the task is globally good; (3) the conditions under which the test takes place are poor and these conditions affect the validity of test scores. The study recommends the test developers to approximate test tasks to those students have been exposed to in classroom situations and those they are likely to encounter in real life. It also recommends all the people involved in the administration of the test to adhere to high ethical standards. / South Africa

Doctor Educationis / The study aims to investigate the fundamental validity issues that can affect the DR Congo English state examination, a national exit test administered to high school final year students for certification. The study aspires to generate an understanding of the potential issues that affect the construct validity of a test within the epistemological stance that supports a strong relationship between test construct and test context. The study draws its theoretical underpinning from three theories: the validity theory that provides a theoretical ground necessary for understanding the quality of tests needed for assessing students’ reading abilities; the construction-integration theory that provides an understanding of how texts used in reading assessments are processed and understood by the examinees; and the strategic competence theory that explains how examinees deploy strategies to complete test tasks, and the extent to which these strategies tap into the reading construct. Furthermore, the study proposes a reading model that signposts the social context of testing; therefore, conceptualizing reading as both a cognitive and a social process. As research design, the study adopts an exploratory design using both qualitative and quantitative data. Besides, the study uses protocol analysis and content analysis methodologies. While the former provides an understanding of the cognitive processes that mediate the reading construct and test performance so as to explore the different strategies examinees use to answer the English state examination (henceforth termed ESE) test questions, the latter examines the content of the different ESE papers so as to identify the different textual and item features that potentially impact on examinees’ performance on the ESE tasks. As instruments, the study uses a concurrent strategies questionnaire administered to 496 student-participants, a contextual questionnaire administered to 26 student-participants, a contextual questionnaire administered to 27 teacher-participants, and eight tests administered to 496 student-participants. The findings indicate that, the ESE appears to be less appropriate to the ESE context as the majority of ESE test items target careful reading than expeditious reading; on the one hand, and reading at global level than to reading at local level; on the other hand. The findings also indicate that the ESE tasks hardly take account of the text structure and the underlined cognitive demands appropriate to the text types. Besides, the ESE fails to include other critical aspects of the reading construct. Finally, the findings also indicate that the ESE constructors may not be capable to construct an ESE with five functioning distractors as expected. Moreover, the inclusion of the implicit option 6 overlaps with the conceptual meaning of this option. The entire process of the present study has generated some insights that can advance our understanding of the construct validity of reading tests. These insights are: (a) the concept of validity is an evolving and context-dependent concept, (b) reading construct cannot be examined outside the actual context of reading activity, (c) elimination of distractors can sometimes be a construct-relevant strategy, (d) construct underrepresentation is a context-dependent concept, and (e) a reading test cannot be valid in all contexts. The suggested proposal for the improvement of the ESE requires the Congolese government through its Department of Education to (a) always conduct validation studies to justify the use of the ESE, (b) always consider the actual context of reading activity while developing the ESE, (c) revisit the meanings and interpretations of the ESE scores, (d) ensure the appropriateness of tasks to be included in the ESE, (e) ensure the construct representativeness of the ESE tasks, (f) revisit the number of questions to be included in the ESE, (g) avoid bias in the ESE texts in order to ensure fairness, (h) diversify the genres of ESE texts, (i) ensure the coherence of ESE texts through the use of transitions and cohesive devices, (j) ensure that the order of test questions is in alignment with the order of text information, (k) revisit the structure and length of the texts to be included in the ESE, (l) revisit the number of alternatives to be included in the ESE, and (m) reconsider the use of the implicit alternative 6.