Orientador: Matilde Virginia Ricardi Scaramucci / Tese (doutorado) - Universidade Estadual de Campinas, Instituto de Estudos da Linguagem / Made available in DSpace on 2018-08-08T02:44:45Z (GMT). No. of bitstreams: 1 Retorta_MiriamSester_D.pdf: 14612163 bytes, checksum: f7fefea3ecb78986aa1456e541123b87 (MD5) Previous issue date: 2007 / Resumo: Este trabalho visa investigar se a prova de língua inglesa do vestibular da Universidade Federal do Paraná causa efeitos retroativos no ensino dessa língua, em escolas públicas e privadas de nível médio e cursos pré-vestibulares, e, em caso positivo, determinar quais são esses efeitos. Para atingir tais objetivos, foi desenvolvida uma pesquisa qualitativa interpretativista de cunho etnográfico, na qual puderam ser ouvidas diversas vozes da comunidade escolar: escolas públicas (urbanas e rurais) e particulares, bem como cursos pré-vestibulares (particulares e gratuitos). Na busca de multiperspectivas sobre o fenômeno, procurou-se escolher os cenários que levassem em conta a grande desigualdade social do país, e incluíssem, como participantes, a maioria dos stakeholders (pessoas ligadas ao fenômeno direta ou indiretamente) para, ao final, triangularem-se os dados coletados. Além de entrevistas dos participantes, aulas de professores também foram observadas. Os resultados deste estudo mostram que o efeito retroativo da prova de inglês do vestibular da UFPR não ocorre nas escolas públicas. O que direciona o ensino desses cenários são os livros didáticos que cada escola adota. No entanto, pudemos perceber a existência do efeito retroativo nos outros cenários ¿ escolas particulares e cursos pré-vestibulares ¿, ora exercendo efeitos positivos (motivação dos diretores e professores para buscar informações sobre o exame; motivação dos alunos para estudar mais a disciplina e passar no exame; direcionamento de ementas e construção de objetivos claros; incentivo ao ensino da leitura), ora negativos (estreitamento do currículo; ansiedade dos sujeitos dos cenários). Verificou-se também que o efeito retroativo ocorreu aí em intensidades diferentes, pois interferiram no fenômeno fatores como: informações que os professores tinham sobre o exame; concepções de língua, leitura e avaliação desses professores; nível de conhecimento de língua inglesa desses profissionais; cobrança dos diretores e do grupo social (os pais e alunos); chances de aprovação, dentre outros. O que direcionou o ensino desses cenários foi o programa oficial do exame de vestibular da UFPR. Este estudo oferece contribuições teóricas ao proporcionar melhor entendimento do conceito efeito retroativo; contribuições metodológicas, por seu desenho investigativo inovador e abrangente; e, finalmente, contribuições práticas, enquanto um conjunto de subsídios para o ensino e avaliação de língua inglesa no ensino médio / Abstract: The objective of this thesis is to investigate if the English test of the University Entrance Examination of UFPR sets off the washback effect in the teaching/learning of the language in public and private high schools as well as cramming courses and, if so, what effects were they. In order to meet these objectives a qualitative interpretativist investigation was conducted in which various voices of the school community were heard such as the participants of public schools (urban and rural), the private schools and the cramming courses (private and free ones). Since there was an intention of having a multiperspective of the phenomenon, the scenarios were chosen because of the great social inequalities of this country and, therefore, many stakeholders (participants who were directly or indirectly involved in the phenomenon) were selected to be interviewed. The data was triangulated, analyzed and discussed. Besides the interview, class observations were included. The results of this study show that there was no washback effect of the English test of the University Entrance Examination of UFPR in public schools. What helped set the teaching goals of the discipline were the contents suggested in the didactic books adopted in each school. In the other scenarios, the washback effect was observed. The positive effects were the motivation of the directors and teachers to search for information about the test; motivation of the students to study harder to pass the test; the test was used to set clear teaching objectives and reading began to be taught. The negative effects were: anxiety of the participants of some scenarios and curriculum narrowing. Different intensities of the washback effect were observed because there were other factors which interfered in the phenomenon such as information teachers had about the test, their concept of language, reading and evaluation, their knowledge of the English language, the directors demanding for the teachers¿ best, the social group (parents and students) demanding for the teachers¿ best and the chances the students had in being approved, among others. The official program of the test was used to guide the teaching/learning of these scenarios. This study offers a theoretical contribution when it helps us understand a bit more about the washback effect; methodological contribution due to the research design which is innovative and broad and, finally, a practical contribution because it intends to offer a set of information which can give support to the teaching and evaluation of the English discipline in high schools / Doutorado / Lingua Estrangeira / Doutor em Linguística Aplicada

Les travaux réalisés dans cette thèse proposent une aide à la décision pour améliorer la qualité de l’apprentissage. L’apprentissage scolaire englobe deux dimension; une dimension humaine et une dimension pédagogique. La dimension humaine inclut l’apprenant et l’enseignant. La dimension pédagogique, représentée par le programme fixé par l’établissement éducatif, correspond au savoir. Ce dernier va se transformer en connaissance chez l’apprenant. Les deux notions de connaissance et savoir sont donc tout à fait différentes. La distance entre les deux représente la distance entre ce que l’enseignant présente (le savoir) et ce que l’apprenant acquière (la connaissance). La qualité de l’apprentissage concerne les apprenants qui vont à l’école pour acquérir le savoir. En fait, apprendre consiste à intérioriser le savoir. Cette intériorisation demande des efforts pour un changement intellectuel persévérant et exige une continuité basée sur les expériences antérieures. L’acquisition du savoir et sa transformation en connaissance par l’apprenant sont influencées par plusieurs facteurs qui interviennent positivement ou négativement sur la quantité et la qualité de cette connaissance. Il peut résider chez l’apprenant une confusion entre les deux notions qui peut l’amener à valoriser ou ignorer sa connaissance. Le processus de construction des connaissances par le savoir diffusé exige une constante mise en œuvre de procédures d’évaluation. Le processus d’évaluation apprécie alors la structure de la connaissance pour prendre des décisions destinées à la faire évoluer. Cependant, lors d’une évaluation, la confusion entre connaissance et savoir peut amener l’apprenant à valoriser le score, négligeant ainsi le regard qu’il pourrait porter sur les processus de transformation des connaissances au profit d’une restitution la plus fidèle possible du savoir. Cette confusion peut être mise en évidence pourvu que l’évaluation intègre une dimension processuelle. Dès lors, l’évaluation peut être mieux associée à des actions d’amélioration et de transformation des connaissances. L’évaluation peut alors être abordée dans une logique d’aide à la décision. Dans ce travail nous montrerons donc qu’une situation d’apprentissage s’apparente à une situation d’aide à la décision. / The research realized in this thesis proposes a decision support to improve the quality of learning. The learning includes two dimensions; human dimension and pedagogic one. The human dimension includes the learner and the teacher. The pedagogic dimension represented in curriculum set by the educational establishment; it is the know. The learner is going to transform the know into knowledge. Thus the know and the knowledge are two notions completely different. The distance between both is the distance between what the teacher presents (the know) and what the learner acquires (the knowledge). The quality of the learning concerns the learners who go to the school to acquire the know. In fact, learning consists in interiorizing the know. This internalization requires the efforts for persistent intellectual change and demands continuity based on past experiences. The acquisition of knowledge and its transformation into knowledge by the learner is influenced by several factors that affect positively or negatively on the quantity and quality of this knowledge. The confusion between the know and the knowledge guide the learner to value or to ignore his knowledge. The knowledge construction process by the diffused know requires an constant evaluation process. The process of evaluation then appreciates the structure of knowledge to make decisions intended to make it evolve. However, during an evaluation, the confusion between knowledge and knowledge can bring learner to value the score so neglecting the importance which he must give for the transformation knowledge process in favor of the highest possible fidelity of knowledge. This confusion can be detected provided that the evaluation includes a processual dimension. Therefore, the evaluation may be better associated with improvement actions and transformation of knowledge. Then the evaluation can be addressed in a logical decision support. Therefore In this research we demonstrate that the learning situation is a decision aiding situation.

Language assessments are increasingly being leveraged for the purposes of educational reform and accountability, yet relatively little is known about how teachers respond to their implementation or what impact they have on the learning outcomes of students. Drawing on theories of educational innovation, this mixed methods study explores the interplay between testing, teaching, and learning in an urban Midwestern K-12 world languages program. Specifically, it investigates the washback effect of the American Council on the Teaching of Foreign Languages (ACTFL) Assessment of Performance toward Proficiency in Languages (AAPPL), and the potential for its influence on teachers’ practices to complement strategic efforts to introduce and manage curricular innovation at the local level. The study was divided into three distinct but complementary phases. In Phase 1, teachers (n=47) responded to an online survey addressing their background, instructional practices, and the potential influence of the AAPPL therein. In Phase 2, semi-structured interviews were conducted with a subset of teachers (n=12) in order to more richly describe their instructional and assessment-related practices. Finally, in Phase 3, AAPPL scores were obtained for all students (n=846) who took the assessment. Quantitative analyses predominantly involved the use of descriptive statistics to depict teachers’ practices and students’ performance on the AAPPL, as well as bivariate correlations to measure the relationship between the two. Qualitative analyses of interview data were aimed at identifying categories and themes leading to a rich description of teachers’ behaviors and activities in response to the AAPPL. In the final stages of analysis, both qualitative and quantitative data sources and analyses were synthesized in an attempt to develop a more nuanced understanding of the interplay between assessment, instruction, and learning. Findings suggested a moderately positive washback effect of the AAPPL, which was manifested through individual and collective efforts to refine curricula and instruction with the goal of further enhancing students’ language performance and proficiency development. Key findings included teachers’ increased use of ACTFL’s core documents to guide their planning, and an adjustment to the amount of instructional time devoted to the four modalities (listening, reading, writing, and speaking). This evidence for positive washback of the AAPPL was observed among teachers across languages and levels, and extended beyond those immediately implicated in its administration. Furthermore, the strength and direction of the influence was mediated by the close link between testing and teaching, individual differences among teachers, and the collaborative approach to managing the innovation. Evidence connecting teachers’ practices with student outcomes on the AAPPL was diffuse, with bivariate correlations indicating a potential relationship between the use of target language and explicit grammar instruction and students’ composite scores. This investigation has contributed to a greater understanding of how external language performance assessments may serve as a catalyst for refining and improving language teaching in K-12 contexts. Those seeking to promote the adoption of similar innovative practices should ensure that the assessment is aligned with the pedagogical goals of the curriculum and that their approach to introducing and managing change is responsive to the local context. While more research is needed in order to determine the potential influence of innovation on the learning outcomes of students (Green, 2013), world language programs should be encouraged to implement standards-based external assessments as a means by which to promote teachers’ adoption of proficiency-oriented instructional practices.

The aim of this thesis is to ascertain whether and to what extent English language lessons at Czech secondary schools are affected by the so-called washback effect related to the final leaving examination ("maturita"), and to attempt to define how this potential negative impact may be minimized. The washback effect phenomenon occurs when a (language) course is concluded by a final examination, and describes how such an examination influences the actual curriculum, the instruction as such, and all the participants. Applied linguistics offers a number of both theoretical and empirical studies conducted worldwide which focus on the particular aspects of the washback effect occurring during language examinations. This thesis strives to follow up on the findings of these empirical studies and apply them to the current centralised Czech form of the English "maturita" examination. The thesis was initially inspired by students' observations during English lessons at a Czech grammar school, which were a part of pedagogical education at the Faculty of Arts of the Charles University. After establishing the focus of the paper, preliminary research in the form of semi- structured interviews with four students in the final grades at Prague grammar schools was conducted. While this pre-research study allowed only...

Les travaux réalisés dans cette thèse proposent une aide à la décision pour améliorer la qualité de l'apprentissage. L'apprentissage scolaire englobe deux dimension; une dimension humaine et une dimension pédagogique. La dimension humaine inclut l'apprenant et l'enseignant. La dimension pédagogique, représentée par le programme fixé par l'établissement éducatif, correspond au savoir. Ce dernier va se transformer en connaissance chez l'apprenant. Les deux notions de connaissance et savoir sont donc tout à fait différentes. La distance entre les deux représente la distance entre ce que l'enseignant présente (le savoir) et ce que l'apprenant acquière (la connaissance). La qualité de l'apprentissage concerne les apprenants qui vont à l'école pour acquérir le savoir. En fait, apprendre consiste à intérioriser le savoir. Cette intériorisation demande des efforts pour un changement intellectuel persévérant et exige une continuité basée sur les expériences antérieures. L'acquisition du savoir et sa transformation en connaissance par l'apprenant sont influencées par plusieurs facteurs qui interviennent positivement ou négativement sur la quantité et la qualité de cette connaissance. Il peut résider chez l'apprenant une confusion entre les deux notions qui peut l'amener à valoriser ou ignorer sa connaissance. Le processus de construction des connaissances par le savoir diffusé exige une constante mise en œuvre de procédures d'évaluation. Le processus d'évaluation apprécie alors la structure de la connaissance pour prendre des décisions destinées à la faire évoluer. Cependant, lors d'une évaluation, la confusion entre connaissance et savoir peut amener l'apprenant à valoriser le score, négligeant ainsi le regard qu'il pourrait porter sur les processus de transformation des connaissances au profit d'une restitution la plus fidèle possible du savoir. Cette confusion peut être mise en évidence pourvu que l'évaluation intègre une dimension processuelle. Dès lors, l'évaluation peut être mieux associée à des actions d'amélioration et de transformation des connaissances. L'évaluation peut alors être abordée dans une logique d'aide à la décision. Dans ce travail nous montrerons donc qu'une situation d'apprentissage s'apparente à une situation d'aide à la décision.

När lärares undervisning blir påverkad av ett nationellt prov talar man om washbackeffekter av provet. Det är vad den här licentiatuppsatsen handlar om. Genom att studera svensklärares uppfattningar om det nationella läsförståelseprovet i kursen Svenska 1 på gymnasiet belyser studien vad som kan hända när nationell policy bestående av ett nationellt prov möter den lokala kontexten i form av enskilda lärares undervisning. I studien har arton gymnasielärare i svenska fått komma till tals genom intervjuer. Resultaten visar att det nationella läsförståelseprovet i Svenska 1 inte i någon nämnvärd utsträckning påverkar lärarnas undervisning. Detta beror dels på att lärarna uppfattar att undervisningen redan behandlar det som provet testar, dels på att lärarna inte anser att resultatet på provet är användbart vid betygsättningen. Lärarna uppfattar också att deras tolkningar av vad läsförståelse är skiljer sig från provkonstruktörens. Delvis kan detta bero på att lärarna saknar ett metaspråk för att tala om läsförståelse och bedömning av elevers läskompetens. Maria Larsson är verksam gymnasielärare i svenska och engelska vid Lugnetgymnasiet i Falun och har bedrivit forskarstudier på halvtid med start höstterminen 2013. Hon har ingått i forskarskolan Skolnära som är ett samarbete mellan Högskolan Dalarna och Karlstads universitet. / This study investigates the washback effects on teaching practices caused by the national test in the course Swedish 1 which was introduced in upper secondary school in 2011. The test consists of three parts; oral presentation, writing proficiency and reading comprehension. The focus of the study is on the washback effects of the reading comprehension test which is an entirely new type of test for upper secondary school in Sweden. The study draws on qualitative data from interviews with 18 teachers of Swedish in upper secondary school. The interviews were carried out in four focus groups and also in a series of four interviews with five individual teachers. The theoretical framework consists of Stephen Ball’s theory of policy enactment. In addition, theory about reading comprehension and an adaptation of Ivanič’s concept discourses of writing transferred into the context of reading are used in order to analyse data. The findings indicate that even though there seems to be substantial washback from the oral and the writing tests, the reading comprehension test does not seem to have affected reading instruction to any large extent. One reason seems to be that reading instruction already covers the contents of the test to a great extent. The teachers in the study also perceive that there is a discrepancy in the alignment between the results of the test and the assessment critera of the course, making it problematic to communicate the results to pupils and to relate them to the final grade. As the result of the test is not interpreted as being useful to the teachers they admit to paying little attention to the reading comprehension test. Furthermore, the teachers tend not to consider the test as an important indicator of pupils’ reading competence as they seem to oppose the notion that interpreting fiction is something that can be measured and objectively assessed according to preset standards. When talking about reading instruction the teachers seem to value most the type of reading which deals with basic values, gives new perspectives on what it means to be human and which encourages pupils to want to read more literature, a type of reading the test cannot offer. Findings also indicate that even though the teachers have adequate knowledge about the teaching of reading comprehension they seem to lack a metalanguage to talk about the different components it is made up of which affects how they perceive the national reading comprehension test.

När lärares undervisning blir påverkad av ett nationellt prov talar man om washbackeffekter av provet. Det är vad den här licentiatuppsatsen handlar om. Genom att studera svensklärares uppfattningar om det nationella läsförståelseprovet i kursen Svenska 1 på gymnasiet belyser studien vad som kan hända när nationell policy bestående av ett nationellt prov möter den lokala kontexten i form av enskilda lärares undervisning. I studien har arton gymnasielärare i svenska fått komma till tals genom intervjuer. Resultaten visar att det nationella läsförståelseprovet i Svenska 1 inte i någon nämnvärd utsträckning påverkar lärarnas undervisning. Detta beror dels på att lärarna uppfattar att undervisningen redan behandlar det som provet testar, dels på att lärarna inte anser att resultatet på provet är användbart vid betygsättningen. Lärarna uppfattar också att deras tolkningar av vad läsförståelse är skiljer sig från provkonstruktörens. Delvis kan detta bero på att lärarna saknar ett metaspråk för att tala om läsförståelse och bedömning av elevers läskompetens. Maria Larsson är verksam gymnasielärare i svenska och engelska vid Lugnetgymnasiet i Falun och har bedrivit forskarstudier på halvtid med start höstterminen 2013. Hon har ingått i forskarskolan Skolnära som är ett samarbete mellan Högskolan Dalarna och Karlstads universitet. / This study investigates the washback effects on teaching practices caused by the national test in the course Swedish 1 which was introduced in upper secondary school in 2011. The test consists of three parts; oral presentation, writing proficiency and reading comprehension. The focus of the study is on the washback effects of the reading comprehension test which is an entirely new type of test for upper secondary school in Sweden. The study draws on qualitative data from interviews with 18 teachers of Swedish in upper secondary school. The interviews were carried out in four focus groups and also in a series of four interviews with five individual teachers. The theoretical framework consists of Stephen Ball’s theory of policy enactment. In addition, theory about reading comprehension and an adaptation of Ivanič’s concept discourses of writing transferred into the context of reading are used in order to analyse data. The findings indicate that even though there seems to be substantial washback from the oral and the writing tests, the reading comprehension test does not seem to have affected reading instruction to any large extent. One reason seems to be that reading instruction already covers the contents of the test to a great extent. The teachers in the study also perceive that there is a discrepancy in the alignment between the results of the test and the assessment critera of the course, making it problematic to communicate the results to pupils and to relate them to the final grade. As the result of the test is not interpreted as being useful to the teachers they admit to paying little attention to the reading comprehension test. Furthermore, the teachers tend not to consider the test as an important indicator of pupils’ reading competence as they seem to oppose the notion that interpreting fiction is something that can be measured and objectively assessed according to preset standards. When talking about reading instruction the teachers seem to value most the type of reading which deals with basic values, gives new perspectives on what it means to be human and which encourages pupils to want to read more literature, a type of reading the test cannot offer. Findings also indicate that even though the teachers have adequate knowledge about the teaching of reading comprehension they seem to lack a metalanguage to talk about the different components it is made up of which affects how they perceive the national reading comprehension test.

The theoretical part of this diploma thesis provides a broader academic context for the research presented in the theoretical part and is concerned with the specifics of the writing assessment. It proposes reasons why writing should be assessed, describes methods of assessment and scoring, and also examines writing tests. At the end of the theoretical part, the presumed controlling idea, called the washback effect, which significantly influenced the course of the research in the practical part of the thesis, is presented. The practical part is divided into two parts. The first part focuses on the types of scoring ESL grammar school teachers employ for assessing grammar school pupils. The aim was to discover how many of them use analytical and/or multiple-trait scoring. It was revealed that analytical scoring the most used means of assessment. Holistic scoring was next, followed by multiple-trait and lastly primary-trait scoring. Other scoring procedures are rarely employed. The second part is a case study of assessment criteria of selected teachers which compares these criteria against those used in Cambridge English: Preliminary examination and the Czech state secondary school-leaving exam. The author of the thesis tried to identify criteria or descriptors which appear across these systems and could,...