<p>Despite an emphasis on oral communication in most foreign language classrooms, the resource-intensive nature (i.e. time and manpower) of speaking tests hinder regular oral assessments. A possible solution is the development of a (semi-) automated scoring system. When it is used in conjunction with human raters, the consistency of computers can complement human raters’ comprehensive judgments and increase efficiency in scoring (e.g., Enright & Quinlan, 2010). In search of objective and quantifiable variables that are strongly correlated with overall oral proficiency, a number of studies have reported that some utterance fluency variables (e.g., speech rate and mean length of run) might be strong predictors for L2 learners’ speaking ability (e.g., Ginther et al., 2010; Hirotani et al., 2017). However, these findings are difficult to generalize due to small sample sizes, narrow ranges of proficiency levels, and/or a lack of data from languages other than English. The current study analyzed spontaneous speech samples collected from 170 Japanese learners at a wide range of proficiency levels determined by a well-established speaking test, the American Council on the Teaching of Foreign Languages’ (ACTFL) Oral Proficiency Interview (OPI). Prior to analysis, 48 <i>Complexity, Accuracy, Fluency</i> (CAF) measures (with a focus on fluency variables) were calculated from the speech samples. First, the study examined the relationships among the CAF measures and learner oral proficiency assessed by the ACTFL OPI. Then, using an empirically-based approach, a feasibility of using a composite measure to predict L2 oral proficiency was investigated. The results revealed that <i>Speech Speed</i> and <i>Complexity</i> variables demonstrated strong correlation to the OPI levels, and moderately strong correlations were found for the variables in the following categories: <i>Speech Quantity, Pause</i>, <i>Pause Location</i> (i.e., Silent pause ratio within AS-unit), <i>Dysfluency</i> (i.e., Repeat ratio), and <i>Accuracy.</i> Then, a series of multiple regression analyses revealed that a combination of five CAF measures (i.e., Effective articulation rate, Silent pause ratio, Repeat ratio, Syntactic complexity, and Error-free AS-unit ratio) can predict 72.3% of the variance of the OPI levels. This regression model includes variables that correspond to Skehan’s (2009) proposed three categories of fluency (speed, breakdown, and repair) and variables that represent CAF, supporting the literature (e.g., Larsen-Freeman, 1978, Skehan, 1996).</p>

Oral proficiency plays a crucial part in mastering the English language. It is, because of that, heavily debated among professors and teachers. Among teachers, uncertainty prevails regarding how oral proficiency should be taught and assessed. This qualitative study aims to raise awareness of how teachers define, understand, and experience oral proficiency in the upper secondary EFL classroom. Furthermore, this study seeks to analyze teachers’ conflicting perceptions regarding the assessment process along with what problems they encounter and how they solve these problems. Four EFL teachers from three schools in the south of Sweden lay the foundation for the study. Semi-structured interviews were used as the method of data collection in order to gain insights from the teachers’ reflections. The data of the interviews were analyzed with the help of the grounded theory approach. Adopting this theory, the results of the study show that the four EFL teachers had general difficulties understanding the terms assessment and oral proficiency. While one teacher viewed oral proficiency as a two-fold challenge, another stated that in order to get a good grade, good discussion abilities are vital. When assessing students’ oral proficiency, the teachers focused on different features such as pronunciation, fluency, variation, interaction, content, and vocabulary.

Orientador: Douglas Altamiro Consolo / Resumo: Neste trabalho investiga-se a abordagem de avaliação da proficiência oral do professor de línguas estrangeiras empregado por um exame brasileiro voltado a essa finalidade específica, o EPPLE – Exame de Proficiência para Professores de Línguas Estrangeiras. O estudo tem por objetivo propor uma abordagem que atenda a necessidades levantadas pela literatura da área acerca da importância da democratização e transparência do processo avaliativo e do emprego de processos decisórios que garantam os princípios de validade sem causar interferências na praticidade das avaliações. Assim, a abordagem proposta subordina-se à elaboração de uma escala de mensuração e à reestruturação da escala de proficiência do exame, agregando-se a opinião de usuários com diferentes graus de especialidade sobre a avaliação de proficiência. Trata-se, portanto, de um estudo de caso de natureza híbrida e cunho exploratório-descritivo que se utiliza de dados secundários e primários para auxiliar a investigação. Os dados secundários consistem em estudos e documentos sobre a caracterização da proficiência linguística para finalidades gerais e para finalidades de ensino, bem como sobre a avaliação dessas proficiências; em dados de fala do teste oral do EPPLE referentes à participação de formandos em Letras de uma universidade pública paulista e de duas universidades públicas mineiras, na qualidade de candidatos ao exame, nos anos de 2015 e 2017; e nos respectivos conceitos atribuídos aos candidatos, seguindo-se ... (Resumo completo, clicar acesso eletrônico abaixo) / Abstract: This research study investigates the assessment model used by a teaching-oriented language examination, the EPPLE – Proficiency Examination for Foreign Language Teachers, in the assessment of foreign language teachers’ oral proficiency. We aim at evaluating the model used by the oral test and proposing a new one that meets current concerns raised by the literature regarding the importance of democratizing the assessment process, guaranteeing transparency and using decision models that assure validity principles without causing interferences in the test’s practicality. Therefore, the model we propose relies on the reorganization of the oral proficiency scale used in the exam through the inclusion of its stakeholders’ opinions. The investigation conducted consists on a quali-quantitative and exploratory-descriptive case study that uses primary and secondary data. Secondary data consist of research studies and documents on the concept of general and teaching purpose language proficiency; speaking data from candidates to the oral test of the EPPLE examination; as well as the respective rates assigned to such candidates according to the proficiency scale used in the exam. The candidates to the EPPLE examination that participate in this study are Letters course undergraduate students from Brazilian public universities, one located in the state of São Paulo and two in the state of Minas Gerais, in the years of 2015 and 2017. Primary data, on the other hand, consist of the answers ob... (Complete abstract click electronic access below) / Doutor

Testing oral proficiency is an important, but often neglected part of the foreign language classroom. Currently accepted methods in testing oral proficiency are timely and expensive. Some work has been done to test and implement new assessment methods, but have focused primarily on English or Spanish (Graham et al. 2008). In this thesis, I demonstrate that the processes established for English and Spanish elicited imitation (EI) testing are relevant to French EI testing. First, I document the development, implementation and evaluation of an EI test to assess French oral proficiency. I also detail the incorporation of the use of automatic speech recognition to score French EI items. Last, I substantiate with statistical analyses that carefully engineered, automatically scored French EI items correlate to a high degree with French OPI scores.

No description available.

The influx of international students attending American universities has put the U.S. at the forefront of international education, generating over $14.5 billion yearly. International students often face adjustment issues impacting their level of satisfaction with their experience. These involve oral language proficiency, as well as feelings of anxiety and isolation resulting from language deficiencies, which increase their feelings of marginality. Engaged learning, which is dependent on comprehension, is affected by the anxiety students bring to class. It is believed that coursecasts may alleviate anxiety, thus improving student engagement in learning. A pre-experimental research design investigates whether coursecasting mitigates feelings of marginalization and isolation, deficiencies in language comprehension and proficiency, and anxiety in international students. Research in the value of coursecasting has been anecdotal, and concrete evidence of its educational value is needed before its wide-spread adoption in educational settings (Elliot, King, & Scutter, 2009). Participants completed the Cultural Adjustment Difficulties Checklist (CADC) by Sodowsky and Lai (1997), the Foreign Language Classroom Anxiety Scale (FLCAS) by Horwitz, Horwitz, and Cope (1986), the Self-Reported Fluency of English Scale (SRFES) by Yeh and Inose (2003), and the Test of English as a Foreign Language (TOEFL), Listening portion. A pretest-posttest approach was chosen to evaluate the influence of podcasting within a naturalistic setting. The implications of the study were categorized into three areas. First, it provided insight into the barriers that negatively impact international student learning due to anxiety. Second, it identified methods of instruction, and issues that may have hindered international students from achieving at their highest level. Finally, it provided an opportunity to explore the educational effectiveness of a technology created for entertainment. Quantitative measures yielded no significance, so the null hypotheses were rejected. It was thought that these results were due to limited exposure to the coursecasting treatment. Their significance was further discussed by providing conclusions and implications. Recommendations for future research suggest replicating the study with a control group or administering the coursecasting treatment for a longer period of time. Suggestions for educators and educational institutions include requesting faculty and staff to identify specific ways to help this vulnerable population with their acculturation instead of waiting for them to ask for help. Although no significant impact could be established between available coursecasts and international students' perceptions of marginality and isolation, anxiety, and English language proficiency and comprehension, the findings regarding the analysis of the dependent variables seem to present interesting implications for education.

Orientador: Douglas Altamiro Consolo / Banca: Ana Mariza Benedetti / Banca: Maria Helena Vieira-Abrahão / Banca: Deise Prina Dutra / Banca: Hélcio de Pádua Lanzoni / Resumo: Este trabalho, de natureza híbrida, insere-se na linha de investigações sobre avaliação de proficiência e estuda a validade dos descritores de gramática na escala do Teste de Proficiência Oral em Língua Inglesa (TEPOLI), especificamente elaborado para professores de inglês como língua estrangeira no Brasil. Justifica-se socialmente pelo resultado de diversas pesquisas em Linguística Aplicada que atestam o baixo nível de proficiência (principalmente oral) de formandos em Letras e professores em serviço no ensino regular, o que tem sido apontado na literatura como uma das razões para o insucesso no ensinoaprendizagem nesses contextos. São revisados na literatura pressupostos teóricos sobre avaliação de proficiência oral e sobre investigação da gramática em produções orais. As perguntas que norteiam esta pesquisa são: (1) como se caracteriza a gramática ao longo das faixas de proficiência no desempenho dos participantes em relação a (a) características da língua falada, (b) precisão e (c) complexidade?; e (2) que relações podem ser estabelecidas entre o desempenho gramatical dos participantes no teste oral e no seminário?. Participam da pesquisa oito alunas formandas de um curso de Letras de uma universidade pública paulista, cujas notas no TEPOLI se distribuem ao longo da escala. Os dados foram coletados por meio de questionários, entrevistas semiestruturadas e gravações de seminários realizados durante o último semestre da graduação e de aplicações do TEPOLI. O foco das análises é dado para a precisão e a complexidade gramatical das estruturas morfossintáticas empregadas na produção oral das participantes durante os eventos de avaliação. Os resultados apontam que há diferenças quantitativas e qualitativas entre as faixas de proficiência para precisão e para complexidade, e que o teste... (Resumo completo, clicar acesso eletrônico abaixo) / Abstract: This research study, of a hybrid nature, is included in the language testing body of research. It aims at investigating the validity of the grammar descriptors of the Test of Oral Language Proficiency in English (TEPOLI), specifically designed for teachers of English as a foreign language in Brazil. The work is socially justified by the results of several studies in Applied Linguistics which attest the low proficiency level (particularly in oral skills) attained by students graduating from teacher education undergraduating courses and in-service teachers working in regular schools, something that has been accounted as one of the reasons for unsuccessful teaching and learning in such settings. It reviews the theoretical principles of oral language testing and grammar in oral production. The research questions that guide this study are: (1) how is grammar characterized along the proficiency bands in the participants' performance in terms of (a) the characteristics of spoken language, (b) accuracy and (c) complexity?; and (2) what interrelations can be established between the participants' grammatical performance in the oral test and in a seminar? Eight students graduating from a teacher education course in a public university in the state of Sao Paulo take part in this study. Data were collected by means of questionnaires, semistructured interviews, recordings of the seminars given in the last term of the students at university, and recordings of the administration of the TEPOLI. The focus of analysis is on grammatical accuracy and complexity of the morphosyntactic structures employed in the participants' oral production at the assessment events. Results point to quantitative and qualitative differences between the proficiency bands for accuracy and complexity, and also indicate that the test has predictive power over... (Complete abstract click electronic access below) / Doutor

Prior studies suggest that raters' familiarity with test-takers' first language (L1) can be a potential source of bias in rating speaking tests. However, there is still no consensus between researchers on how and to what extent that familiarity affects the scores. This study investigates raters' performance and focuses on not only how raters' second language (L2) proficiency level interacts with examinees' L1, but also if raters' teaching experience has any effect on the scores. Speaking samples of 58 ESL learners with L1s of Spanish (n = 30) and three Asian languages (Korean, n = 12; Chinese, n = 8; and Japanese, n = 8) of different levels of proficiency were rated by 16 trained raters with varying levels of Spanish proficiency (Novice to Advanced) and different degrees of teaching experience (between one and over 10 semesters). The ratings were analyzed using Many-Facet Rasch Measurement (MFRM). The results suggest that extensive rater training can be quite effective: there was no significant effect of either raters' familiarity with examinees' L1, or raters' teaching experience on the scores. However, even after training, the raters still exhibited different degrees of leniency/severity. Therefore, the main conclusion of this study is that even trained raters may consistently rate differently. The recommendation is to (a) have further rater training and calibration; and/or (b) use MFRM with fair average to compensate for the variance.

<p>When speaking a language, whether it is our first or second language, grammatical mistakes will be made. The aim of this essay is to look into what kinds of mistakes some Swedish learners of English make when speaking English and to analyze why these mistakes are made. The essay also aims at looking into what grammar teaching can look like in Sweden and how some teachers look upon their students’ oral proficiency.</p><p>The method used for this study was a qualitative one, namely interviews. Twelve students, eight in grade seven and four in grade nine, and two teachers were interviewed. During the interviews with the students a dictaphone was used. When interviewing the teachers notes were taken, and these have been the foundation of the analysis.</p><p>The results showed that many of the mistakes made by the students seemed to originate in transfer from their first language. Preposition mistakes, for instance, were made in 20% of the cases and they mainly originated in interference with their first language.</p><p>Verbs turned out to be the area where most mistakes were made, followed by prepositions and pronouns. 50% of the mistakes made by students in grade nine were verb mistakes, whereas the students in grade seven made verb mistakes in 33% of the cases.</p><p>This study further shows that the teachers had a good grasp of what their students know, and do not know, but there were some mistakes the learners made which the teachers did not mention. Finally, the study showed that spoken language is in focus within the classroom. Students are allowed to make mistakes, even though the interviewed teachers find grammar important.</p>

Made available in DSpace on 2014-06-11T19:32:45Z (GMT). No. of bitstreams: 0 Previous issue date: 2009-06-05Bitstream added on 2014-06-13T19:22:19Z : No. of bitstreams: 1 almeida_vb_dr_sjrp.pdf: 2222844 bytes, checksum: 34a0b9d0faaf0a86e6e1279e94cd3b12 (MD5) / Este trabalho, de natureza híbrida, insere-se na linha de investigações sobre avaliação de proficiência e estuda a validade dos descritores de gramática na escala do Teste de Proficiência Oral em Língua Inglesa (TEPOLI), especificamente elaborado para professores de inglês como língua estrangeira no Brasil. Justifica-se socialmente pelo resultado de diversas pesquisas em Linguística Aplicada que atestam o baixo nível de proficiência (principalmente oral) de formandos em Letras e professores em serviço no ensino regular, o que tem sido apontado na literatura como uma das razões para o insucesso no ensinoaprendizagem nesses contextos. São revisados na literatura pressupostos teóricos sobre avaliação de proficiência oral e sobre investigação da gramática em produções orais. As perguntas que norteiam esta pesquisa são: (1) como se caracteriza a gramática ao longo das faixas de proficiência no desempenho dos participantes em relação a (a) características da língua falada, (b) precisão e (c) complexidade?; e (2) que relações podem ser estabelecidas entre o desempenho gramatical dos participantes no teste oral e no seminário?. Participam da pesquisa oito alunas formandas de um curso de Letras de uma universidade pública paulista, cujas notas no TEPOLI se distribuem ao longo da escala. Os dados foram coletados por meio de questionários, entrevistas semiestruturadas e gravações de seminários realizados durante o último semestre da graduação e de aplicações do TEPOLI. O foco das análises é dado para a precisão e a complexidade gramatical das estruturas morfossintáticas empregadas na produção oral das participantes durante os eventos de avaliação. Os resultados apontam que há diferenças quantitativas e qualitativas entre as faixas de proficiência para precisão e para complexidade, e que o teste... / This research study, of a hybrid nature, is included in the language testing body of research. It aims at investigating the validity of the grammar descriptors of the Test of Oral Language Proficiency in English (TEPOLI), specifically designed for teachers of English as a foreign language in Brazil. The work is socially justified by the results of several studies in Applied Linguistics which attest the low proficiency level (particularly in oral skills) attained by students graduating from teacher education undergraduating courses and in-service teachers working in regular schools, something that has been accounted as one of the reasons for unsuccessful teaching and learning in such settings. It reviews the theoretical principles of oral language testing and grammar in oral production. The research questions that guide this study are: (1) how is grammar characterized along the proficiency bands in the participants’ performance in terms of (a) the characteristics of spoken language, (b) accuracy and (c) complexity?; and (2) what interrelations can be established between the participants’ grammatical performance in the oral test and in a seminar? Eight students graduating from a teacher education course in a public university in the state of Sao Paulo take part in this study. Data were collected by means of questionnaires, semistructured interviews, recordings of the seminars given in the last term of the students at university, and recordings of the administration of the TEPOLI. The focus of analysis is on grammatical accuracy and complexity of the morphosyntactic structures employed in the participants’ oral production at the assessment events. Results point to quantitative and qualitative differences between the proficiency bands for accuracy and complexity, and also indicate that the test has predictive power over... (Complete abstract click electronic access below)