Large-scale assessments are increasingly being used for more than one purpose, such as admissions, placement, and diagnostic decision-making, with each additional use requiring validation regardless of previous studies investigating other purposes. Despite this increased multiplicity of test use, there is limited validation research on adding diagnostic purposes—with the intention of directly benefiting teaching and learning—to existing large-scale assessments designed for high-stakes decision-making. A challenge with validating diagnostic purposes is to adequately balance investigations into the score interpretations and the intended beneficial consequences for teachers and students. The Assessment Use Argument (AUA) makes explicit these internal and consequential validity questions through a two-stage validation argument (Bachman & Palmer, 2010). This research adopted the AUA to examine the appropriateness of the Canadian Academic English Language (CAEL) Assessment for diagnostic purposes, by forming a validity argument that asked, to what extent did the CAEL essay meet the new diagnostic scoring challenges from the rater perspective, and a utilization argument centered on teachers' and students’ uses of the diagnostic information obtained from the assessment. This study employed three research phases at an English for Academic Purposes (EAP) program in one Canadian university. Data collection strategies included interview and verbal protocol data from two raters (Phase 1), interview and classroom observation data from one EAP course instructor (Phase 2), and interview and open-ended survey data from 47 English Language Learners (Phase 3). A multifaceted perception of CAEL for diagnostic purposes was observed: raters noted the greatest diagnostic potential at higher score levels, and teacher and student perceptions were largely influenced by previous diagnostic assessment experiences. This research emphasized the necessity of including multiple perspectives across contexts to form a deeper realization of the inferences and decisions made from diagnostic results. / Thesis (Ph.D, Education) -- Queen's University, 2013-04-29 09:40:22.649

This study aims to investigate Korean teachers’ beliefs and their practice with respect to classroom-based English language assessment; thus it examines the teachers’ current working principles of assessment and their practices. This study also sets out to uncover, and to gain an in-depth understanding of further issues which emerged from the dissonance between the teachers’ beliefs and their practice. Following a discussion of the English teaching and assessment context, the first part of the study examines mainstream theories of language testing or assessment; it then considers how closely classroom-based assessment in Korean primary schools conforms to these theoretical principles. The second part of the study presents a small-scale research project. Four stages in teachers’ classroom-based assessment were examined; planning, implementation, monitoring, and recording and dissemination. A questionnaire was developed reflecting these stages; its findings were analyzed statically and qualitatively. Further qualitative data was also collected and analyzed through interviews with volunteer participants. This is based on an analysis of teachers’ firsthand experience and their opinions of the assessment of English as a foreign language. The results of the study revealed that generally the teachers hold and exercise their own firm beliefs regarding classroom-based assessment, and have a good knowledge of assessment or testing principles; thus they carried out their assessment using appropriate procedures taking into account the context of English teaching and assessment in which they operate. However, there were a number of issues which emerged from their assessment beliefs and their practice. It became clear that they did not put some of their principles into practice; a number of important factors, which are normally outside the teachers’ control, were found to be responsible for this, these include: overcrowded classrooms, heavy teaching loads, the central bureaucracy of the education system which controls primary education, and a shortage of funding for foreign language teaching. Teachers were also affected by the rather complex relationship with other teachers, head teachers, and even the parents of the students. However, it is evident that the teachers are constantly developing their skills and knowledge regarding assessment in order to address any possible challenges or tasks given to them. In addition, certain areas needing further investigation were identified. Based on the literature review and the findings of the research, tentative implications and recommendations for the development of classroom-based language assessment are discussed.

The purpose of this study was to explore ESL teachers’ perceptions and use of classroom-based reading assessments. The research questions underpinning this study were: 1) What types of classroom-based reading assessments are used in ESL classrooms and how are they used? 2) What are ESL teachers’ perceptions regarding the function and effectiveness of classroom-based reading assessments? 3) What and how do external factors influence ESL teachers’ use of classroom-based reading assessments? 4) What and how do internal factors influence ESL teachers’ use of classroom-based reading assessments? The participants of this study were six middle school ESL teachers and seven elementary school ESL teachers. Data consisted of interviews with the participating ESL teachers, classroom observations, and assessment materials. The finding of this study indicated that there were three kinds of classroombased reading assessments commonly used by ESL teachers in the classrooms: tests, observation, and using writing to assess reading. These classroom-based reading assessments served ESL teachers in two ways: helping teachers make decisions about individual students and helping teachers make decisions about instruction. In addition, classroom-based reading assessments were viewed as effective instructional instruments. ESL teachers highly valued classroom-based reading assessments, considered them accurate and valuable, and thought these assessments could provide great help to the daily teaching of reading. Students, statewide mandated standardized tests, and districts were three major forces that influenced this assessment process. Four conclusions can be drawn from this study. First, classroom-based reading assessments played a central role in ESL teachers’ teaching and assessing of reading. Second, ESL teachers highly valued classroom-based reading assessments, considering them valuable, accurate, and efficient. Compared to statewide mandated standardized testing, ESL teachers preferred classroom-based reading assessments. Third, ESL teachers’ use of classroom-based reading assessments was largely under the control of districts or school authorities and there were many disagreements on the ways of assessing reading of ESL students between teachers and the districts or schools. Finally, statewide mandated standardized testing had distorted ESL teachers’ use of classroom-based reading assessments in practice.