The purpose of my dissertation was to explore whether a virtual world could be a suitable platform for second language assessment. It specifically looked at the validity evidence of a virtual-world group oral test compared to that of a face-to-face group oral test by means of test-retest reliability, concurrent validity, face validity, and discourse features in the two testing modes. A total of 64 ESL students who were enrolled in a large language institution in the southwestern part of the United States participated in the study. Thirty students served as a control group and took two sets of face-to-face and virtual-world tests. Thirty-four students served as an experimental group and took two sets of face-to-face and virtual-world tests after receiving familiarity training in the virtual-world setting. Data were drawn from the students’ group oral test scores, a survey asking for their perceptions on the two testing modes, interviews, and speech samples from their group oral tests. The findings showed that students produced similar scores when tested again in the virtual world, confirming the test-retest validity. The results also revealed that the students’ group oral test scores in the virtual world were comparable to their face-to-face group oral scores, providing concurrent validity of the virtual-world testing mode. It is noteworthy that students produced comparable scores in the virtual world only after they experienced the virtual world. Students’ perceptions on the virtual-world testing mode were promising in terms of anxiety, ease of the testing mode, and ease of turn-taking. Qualitative analyses of interviews showed that virtual worlds have some benefits, including interesting, relaxing testing conditions as well as a feeling of co-presence in the virtual world. Conversation analysis of the students’ interactions revealed four interactional features unique to the two testing modes: 1) different strategies to start the conversation in each testing mode, 2) more scaffolding in face-to-face, 3) more successful instances of showing agreement and disagreement in face-to-face mode, and 4) different turn-taking patterns in each testing mode. / text

No description available.

Testing, at times, can be a complicated matter. It takes a lot of time, precision and trial and error to adequately create a valid and reliable test. When creating a test, we should be aware of the impact that it is going to have on our teaching and whether it will be positive or negative. The goal of this project was to create four good intermediate Spanish diagnostic pre-tests that could be taken at the Brigham Young University Humanities Computer Testing Laboratory. The purpose of these pre-tests was to ascertain the grammatical strengths and weaknesses of our Fall Semester 2012 Spanish 205 students' in regards to specific grammatical principles covered throughout the course. Then, the results could be used to determine how to best plan class time and promote good instructional decisions. This report covers the necessary steps it took to develop and validate said pre-tests and concludes with reviewing the results of the validity and reliability process and gives recommendations for future application of the pre-tests designed and implemented.

En esta tesis se analizan aspectos relacionados con exámenes de segundas lenguas de alto impacto (high-stakes). A diferencia de estudios anteriores, se trata de un trabajo de investigación comparativo que toma como referencia los contrastes y las similitudes que existen entre la PAU española y el sistema inglés de exámenes A-levels en relación a distintos aspectos. Por un lado, se presenta el tema desde una perspectiva histórica de la evolución de los exámenes de acceso a la universidad en cada país. Por otro, se hace un análisis en profundidad de aspectos clave de la evaluación como la validez del constructo, las características de las actividades o ítems y la fiabilidad. El contraste entre dos realidades evaluadoras diferentes nos permite ilustrar los desaciertos del sistema español mediante una evaluación de los éxitos relativos al procedimiento de evaluación pre-universitario inglés. Además, al comparar la naturaleza de la prueba en España con un equivalente europeo, se pretende comprender la manera en la que la PAU puede ser mejorada en el futuro. Finalmente, en esta tesis presentamos una propuesta de cambios fundamentales y viables para la prueba de inglés en la PAU actual. / López Navas, MD. (2012). COMPARATIVE STUDY OF TWO FOREIGN LANGUAGE EXAMINATIONS FOR UNIVERSITY ENTRY IN ENGLAND AND SPAIN [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/18055

In an increasingly interconnected world where English has become the lingua franca of business, culture, entertainment, and academia, learners of English as a second language (L2) have been steadily growing. This has contributed to an increasing demand for automatic spoken language assessment systems for formal settings and practice situations in Computer-Assisted Language Learning. One common misunderstanding about automated assessment is the assumption that machines should replicate the human process of assessment. Instead, computers are programmed to identify, extract, and quantify features in learners' productions, which are subsequently combined and weighted in a multidimensional space to predict a proficiency level or grade. In this regard, transferring human assessment knowledge and skills into an automatic system is a challenging task since this operation should take into account the complexity and the specificities of the proficiency construct. This PhD thesis presents research conducted on methods and techniques for the automatic assessment and feedback of L2 spoken English, mainly focusing on the application of deep learning approaches. In addition to overall proficiency grades, the main forms of feedback explored in this thesis are feedback on grammatical accuracy and assessment related to particular aspects of proficiency (e.g., grammar, pronunciation, rhythm, fluency, etc.). The first study explores the use of written data and the impact of features extracted through grammatical error detection on proficiency assessment, while the second illustrates a pipeline which starts from disfluency detection and removal, passes through grammatical error correction, and ends with proficiency assessment. Grammar, as well as rhythm, pronunciation, and lexical and semantic aspects, is also considered in the third study, which investigates whether it is possible to use systems targeting specific facets of proficiency analytically when only holistic scores are available. Finally, in the last two studies, we investigate the use of self-supervised learning speech representations for both holistic and analytic proficiency assessment. While aiming at enhancing the performance of state-of-the-art automatic systems, the present work pays particular attention to the validity and interpretability of assessment both holistically and analytically and intends to pave the way to a more profound and insightful knowledge and understanding of automatic systems for speaking assessment and feedback.