This study was designed to investigate the usefulness of the administration of a Spanish translation of the Peabody Picture Vocabulary Test, Form A, in the language assessment of bilingual children.

Thesis (Ed.D.)--Boston University / PLEASE NOTE: Boston University Libraries did not receive an Authorization To Manage form for this thesis or dissertation. It is therefore not openly accessible, though it may be available by request. If you are the author or principal advisor of this work and would like to request open access for it, please contact us at open-help@bu.edu. Thank you. / This research explored four techniques of language assessment of counselee behavior to see if linguistic patterning occurred during the counseling process and whether or not such patterning could be assessed. Linguistic patterns do occur in the counseling process and can be assessed in a functional manner through the analyses of taped sessions. Among the ten undergraduate counselees who participated in the 4 to 10 sessions analyzed for the study, it was possible to differentiate those "most improved" and those "least improved." Linguistic analyses supported by judges ratings, showed the "most improved" counselee had balanced frequencies of past, present and future tense usage; and shifted in their association-analogy categories toward a more frequent use of analogy and a less frequent use of association. The wide range of reliability coefficients reported previously partially accounts for the finding that the eight syntactical relationships and the three personal construct categories did not appear related to improvement in counseling as evaluated by the researcher and judges. [TRUNCATED] / 2031-01-01

This thesis investigates the effectiveness of English language assessment in the Foundation Programme (FP) and its predictive validity for academic achievement in the First Year (FY) at two Colleges of Applied Sciences (CAS) in Oman. The objectives of this study are threefold: (1) Identify how well the FP assessment has met its stated and unstated objectives and evaluate its intended and unintended outcomes using impact evaluation approaches. (2) Study the predictive validity of FP assessment and analyse the linguistic needs of FY academic courses and assessment. (3) Investigate how FP assessment and its impact are perceived by students and teachers. The research design was influenced by Messick‟s (1989; 1994; 1996) unitary concept of validity, by Norris (2006; 2008; 2009) views on validity evaluation and by Owen‟s (2007) ideas on impact evaluation. The study was conducted in two phases using five different methods: questionnaires, focus groups, interviews, document analysis and a correlational study. In the first phase, 184 students completed a questionnaire and 106 of these participated in 12 focus groups, whilst 27 teachers completed a different questionnaire and 19 of these were interviewed. The aim of this phase was to explore the perceptions of the students and teachers on the FP assessment instruments in terms of their validity and reliability, structure, and political and social impact. The findings indicated a general positive perception of the instruments, though more so for the Academic English Skills course (AES) than the General English Skills course (GES). There were also calls for increasing the quantity and quality of the assessment instruments. The political impact of the English language FP assessment was strongly felt by the participants. In the second phase, 176 students completed a questionnaire and 83 of them participated in 15 focus groups; 29 teachers completed a different questionnaire and of these 23 teachers were interviewed. The main focus was on students and teachers‟ perceptions of FP assessment, and how language accuracy should be considered in marking academic written courses. One finding was that most students in FY tended to face difficulties not only in English but also in what could be called „study skills‟; some of these were attributed to the leniency of FP assessment exit criteria. Throughout the two phases, 118 documents on FP assessment at CAS were thematically analysed. The objective was to understand the official procedures prescribed for writing and using assessment instruments in FP and compare them against actual test papers and classroom practices. The findings revealed the use of norm-referenced assessment instead of criterion referenced, incompatibility between what was assessed and what was taught, inconsistency in using assessment criteria and in the unhelpful verbatim replication of national assessment standards. The predictive validity studies generally found a low overall correlation between students‟ scores in English language assessment instruments and their scores in academic courses. The findings of this study are in line with most but not all previous studies. The strength of predictive validity was dependent on a number of variables especially the students‟ specializations, and their self-evaluations of their own English language levels. Some recommendations are offered for the reform of entry requirements of the Omani higher education.

No description available.

The option to bring-your-own-device (BYOD) to educational settings is becoming more prevalent as mobile technologies are more accessible than ever, yet little research has been done to examine the effect of those devices on language assessment. In this study, participants (n=175) were divided by stratified random sampling into four groups. Using a Latin square design to control for ordering, two forms of a multiple-choice reading and listening exam were administered over two days. On each day, participants took one test on a BYOD mobile device and one on a computer. A repeated measures ANOVA was used to determine the effect that device type had on score. During the administration of the test, the BYOD condition revealed a number of difficulties that would caution against full-scale adoption for high stakes testing, but the test scores on the computer and BYOD mobile version of the exam were not significantly different in either skill area.

Large-scale assessments are increasingly being used for more than one purpose, such as admissions, placement, and diagnostic decision-making, with each additional use requiring validation regardless of previous studies investigating other purposes. Despite this increased multiplicity of test use, there is limited validation research on adding diagnostic purposes—with the intention of directly benefiting teaching and learning—to existing large-scale assessments designed for high-stakes decision-making. A challenge with validating diagnostic purposes is to adequately balance investigations into the score interpretations and the intended beneficial consequences for teachers and students. The Assessment Use Argument (AUA) makes explicit these internal and consequential validity questions through a two-stage validation argument (Bachman & Palmer, 2010). This research adopted the AUA to examine the appropriateness of the Canadian Academic English Language (CAEL) Assessment for diagnostic purposes, by forming a validity argument that asked, to what extent did the CAEL essay meet the new diagnostic scoring challenges from the rater perspective, and a utilization argument centered on teachers' and students’ uses of the diagnostic information obtained from the assessment. This study employed three research phases at an English for Academic Purposes (EAP) program in one Canadian university. Data collection strategies included interview and verbal protocol data from two raters (Phase 1), interview and classroom observation data from one EAP course instructor (Phase 2), and interview and open-ended survey data from 47 English Language Learners (Phase 3). A multifaceted perception of CAEL for diagnostic purposes was observed: raters noted the greatest diagnostic potential at higher score levels, and teacher and student perceptions were largely influenced by previous diagnostic assessment experiences. This research emphasized the necessity of including multiple perspectives across contexts to form a deeper realization of the inferences and decisions made from diagnostic results. / Thesis (Ph.D, Education) -- Queen's University, 2013-04-29 09:40:22.649

This research explores the final assessment of language competence of future foreign language (FL) teachers (university graduates) in the Russian educational context. Foreign Language teacher training has always been an essential part of Russian education and its importance increased in the 1990s. Later however, with significant educational reforms at primary and secondary school level, teacher training became an area of least attention and interest from the Ministry of Education of Russia and local education authorities. This research is based on the belief that no school reforms are possible without investing in teachers and, therefore, in initial and in-service teacher education, with assessment being one of its key dimensions. The study aims to describe optimal methods of assessing language competence of novice teachers of English as a FL in Russia. For this purpose, the following objectives have been achieved: - a description of current notions of FL teacher language competence, based on analyses of previous theoretical and empirical research; - design of exam evaluation tools – 3 questionnaires and an interview framework, and their use in data collection from various stakeholders in a Russian state pedagogical university; - identification of strengths and weaknesses of the current Final language assessment; - description of possible alternative options for the Final Language Examination and discussion of their impact on different stakeholders. The research follows a mixed-methods design with both qualitative and quantitative data collected and discussed. The study involves various stakeholders at different levels and from different backgrounds – university students, Final Exam takers; Exam designers and administrators, and also teachers of English who provided their valuable vision of the current Final Language Examination and its possible alternatives. The data obtained through surveys and interviews allows for tentative conclusions on the current Language Examination’s appropriacy and relevance, and provides ground for a multi-faceted analysis of the Exam’s strong points and weaknesses, and for development of alternative assessment tasks. The research concludes by viewing possible changes in the Exam as likely and less likely to happen in the near future, based on analysis of the Russian higher education context.

The purpose of my dissertation was to explore whether a virtual world could be a suitable platform for second language assessment. It specifically looked at the validity evidence of a virtual-world group oral test compared to that of a face-to-face group oral test by means of test-retest reliability, concurrent validity, face validity, and discourse features in the two testing modes. A total of 64 ESL students who were enrolled in a large language institution in the southwestern part of the United States participated in the study. Thirty students served as a control group and took two sets of face-to-face and virtual-world tests. Thirty-four students served as an experimental group and took two sets of face-to-face and virtual-world tests after receiving familiarity training in the virtual-world setting. Data were drawn from the students’ group oral test scores, a survey asking for their perceptions on the two testing modes, interviews, and speech samples from their group oral tests. The findings showed that students produced similar scores when tested again in the virtual world, confirming the test-retest validity. The results also revealed that the students’ group oral test scores in the virtual world were comparable to their face-to-face group oral scores, providing concurrent validity of the virtual-world testing mode. It is noteworthy that students produced comparable scores in the virtual world only after they experienced the virtual world. Students’ perceptions on the virtual-world testing mode were promising in terms of anxiety, ease of the testing mode, and ease of turn-taking. Qualitative analyses of interviews showed that virtual worlds have some benefits, including interesting, relaxing testing conditions as well as a feeling of co-presence in the virtual world. Conversation analysis of the students’ interactions revealed four interactional features unique to the two testing modes: 1) different strategies to start the conversation in each testing mode, 2) more scaffolding in face-to-face, 3) more successful instances of showing agreement and disagreement in face-to-face mode, and 4) different turn-taking patterns in each testing mode. / text

Washback is a concept commonly used in applied linguistics to refer to the influence of testing on teaching and learning. The purpose of this study was to investigate the washback effect of a new system of English language speaking assessment in Sri Lanka. The new assessment was introduced with the intention of promoting the teaching and learning of English speaking skills in schools as part of a Presidential educational initiative called the English as a Life Skill Programme. The study examined the washback effect of the introduction of speaking assessments at both National and school levels from the perspectives of participants at three levels of the education system: the decision making level, intervening level (teacher trainers and in-service advisors), and implementing level (teachers and students). For this purpose, a mixed methods research approach was employed. Semi-structured interviews were conducted with participants at the decision making level and intervening level to examine whether there were any important gaps in translating policy intentions to the implementing level participants (teachers and students). A questionnaire survey was conducted with teachers and students to investigate their perceptions of the assessment change and its effects on teaching and learning speaking in the classroom. Classroom observations were conducted to gain insights into actual classroom practices in relation to teaching and learning speaking, along with follow-up interviews to seek teachers’ accounts of their classroom practices. The study found that the assessment change did influence teachers’ and students’ perceptions of teaching and learning speaking in the classroom, as well as teachers’ instructional practices. Therefore, some of the policymakers’ intended aims were achieved. However, the intensity and direction of washback were shown to be influenced by several mediating factors such as teachers’ training and contextual factors such as the availability of classroom resources. The findings of this study suggest that assessment reforms can be used to promote change both in what is taught in the classroom and how it is taught, but to different degrees. The study indicated that washback does occur in this context, but it operates in a complex manner associated with many other variables besides the assessment itself. The findings of this study have implications for the improvement of future assessment policies in Sri Lanka, highlighting the importance of timely implementation of reforms and of monitoring them. The findings suggest that it is especially important to listen to key stakeholders’ (teachers’ and students’) voices in the initial planning and feasibility study phases of reform.

The overarching purpose of both dissertation studies is to contribute to the extant literature base on language assessment in the context of poverty and African American English (AAE) dialect. Language assessment with culturally and linguistically diverse populations, in particular children who speak AAE, has been a longstanding challenge for professionals in the field of speech-language pathology despite the preponderance of scholarly attention this topic has received. The purpose of the first study is to conduct a systematic review of the literature to synthesize the existing literature on AAE from the past approximately three and a half decades, to identify aspects of language and assessment approaches that have been most informative for identifying language impairment in this population. The purpose of the second study is to examine the grammaticality judgments of school-age, AAE-speaking children as a function of their nonmainstream dialect density and language ability. Data for this study came from 273 African American children from low-income backgrounds who were participants in a larger project focused on language and literacy outcomes for children reared in urban areas. the relationship between language ability and dialect density was explored using correlational analysis and the contribution of language ability and dialect density on grammaticality judgments was analyzed using multiple regression. Finally, a multivariate analysis of variance was conducted to investigate the impact of dialect density and language ability on various items that differed in grammatical constructions. Results from both studies are discussed relative to the existing oral language profiles of AAE speakers and the impact of linguistic variation on assessment. Together, these papers contribute to the extant literature by supporting the development of a more comprehensive profile of AAE and increasing the field's understanding of language assessment and language impairment in child AAE speakers.