The College English Test (CET) in China is the largest language test in the world. The number of CET test-takers has steadily increased from 100,000 for its first administration in 1987 to 13 million in 2006. CET scores are used to draw inferences about the test-takers’ English as a foreign language proficiency as well as their specific skills in listening, speaking, reading, and writing. To justify the inferences drawn from test scores, evidence from a variety of sources should be constantly collected (Cronbach & Meehl, 1955; Messick, 1992; Chapelle, 1998; Bachman, 2000; Weir, 2005). Despite the large-scale and high-stakes nature of the CET and the importance of test validation, studies on the quality of the CET are scarce. This study aims to examine the construct validity of the reading comprehension section of the CET by modeling the internal relationships between test-takers’ scores on the CET reading section and their underlying reading abilities. Six components have been chosen as observed variables of the latent variable of reading ability, namely, word recognition efficiency, working memory, semantic knowledge, syntactic knowledge, discourse knowledge, and metacognitive reading skills. A pseudowords identification task programmed by the DMDX computer software, a revised version of Daneman & Carpenter’s (1980) sentence reading span working memory test, Meara & Milton’s (2002) Yes/No vocabulary tests, the syntactic test used in Shiotsu & Weir’s (2007) study, Abeywickrama’s (2007) discourse knowledge test, and a revised version of Phakiti’s (2008) strategy use questionnaire were utilized to measure these six observed variables. A total of 181 Chinese undergraduates participated in the study. With a baseline confirmatory factor model of reading ability and the CET scores, a structural model was analyzed. The results indicated that the path from reading ability to test performance was .75 and the squared regression coefficient of test performance was .56, which implied that participants’ test performance was strongly underlined by their actual reading ability. Therefore, the scores on the CET reading section are largely justifiable for use in drawing inferences about participants’ reading ability. Implications for validation research and reading instruction were also explored. / text

The purposes of this project were to introduce (1) the four main college level English tests used in the People's Republic of China to expatriate English language educators, researchers, curriculum developers, and program designers; and (2) the college English language teaching situation in China to expatriates and give them information on where and how to apply for teaching positions there. The project produced two products to fulfill these goals. First, a paper titled an introduction to major university English tests in China: Their nature, development and importance is now ready to submit for publication. It is a paper that introduces all four of the main college level English tests in China to the outside world. It explains these high-stakes English tests to scholars outside of China. Second, a website, http://www.tesolinchina.blog.com, was created to fulfill the second goal of the project. It is the first website that provides those who are interested in teaching English in China with a complete list of major universities in each geographical region of China as well as other necessary information, and links to these universities.

This study examined motivation, anxiety, global awareness, and linguistic confidence, and their relation to language test performance within the context of Chinese university students taking the College English Test Band 4(CET-4) in China. Using a mixed methods approach, through survey and interview inquiries, this study explored whether and how the selected psychological factors contributed to students’ CET performance. Results from exploratory factor analysis revealed that Chinese university students displayed three types of instrumental motivation (i.e., mark orientation, further-education orientation, and job orientation), two types of anxiety (i.e., language anxiety and test anxiety), and two types of confidence (i.e., linguistic confidence and test confidence). The results of confirmatory factor analysis led to a modified socio-educational model of motivation with some context-specific concepts (i.e., new instrumental orientations, global awareness, and linguistic confidence) that more accurately represented the characteristics of the Chinese university students. The results of structural equation modelling confirmed that attitude toward the learning situation and integrative orientation were two strong indicators of motivation, which in turn influenced language achievement and confidence. The negative impact of anxiety on language achievement was confirmed. Certain group differences were found in comparing male students with female students, high achievers with low achievers, students from the Arts programs with those from the Science programs, and students who started to learn English before Grade 7 with those iii who did so after Grade 7. The interview findings indicated stronger instrumental orientations than integrative orientations. External influences, including influences from society, teachers, and peers, were also identified. Students expressed their mixed feelings toward the CET-4, indicating that this test had both positive and negative influences in promoting their English learning. Testing well-developed motivation and anxiety models in the Chinese context enriched and expanded our knowledge in theory development in English language education in China. The implications of this study point to the importance of understanding language test-takers’ characteristics in their macro and micro learning contexts, as well as the importance of establishing the relevance of English language learning to language teaching, and testing in English as Foreign Language contexts. / Thesis (Ph.D, Education) -- Queen's University, 2009-12-30 22:08:41.138

This dissertation follows an oral language assessment tool from initial design and implementation to validity analysis. The specialized variables of this study are the population: international teaching assistants and the purpose: spoken assessment as a hiring prerequisite. However, the process can easily be applied to other populations and assessment goals.While evaluating the TBEST (Task-Based English Speaking Test) and TAST (TOEFL Academic Speaking Test), I search for a preponderance of evidence for assessment validity that indicate the most appropriate tool for evaluating potential ITAs. The specific evidences of assessment validity that are examined are:1. Evidence of Domain (Content) Validity: Which test, the TBEST or the TAST most closely measures the actual skills needed to be an ITA?2. Evidence of Predictive Criterion Validity: Which test, the TBEST or the TAST, is more valid in predicting ITA teaching success based on end of semester student evaluation (TCEs)?Following the analyses of these points of evidence, the results of a follow-up survey of ITA impressions about the ITA training and evaluating process are reviewed. Reviewing the results of this survey places the language assessment and hiring process recommendations within its larger context, directing attention toward suggestions for improvement of ITA training and evaluating procedures.Over the course of 18 months, 335 ITAs were assessed using the TBEST. 193 ITAs took the TAST prior to taking the TBEST, and those scores are used for correlation analysis. 119 ITAs participated in a follow up survey about their ITA experience.Analysis of domain validity shows that the TBEST is better suited for assessing ITAs than the TAST due to specialized assessment content not present on the more generic TAST. The TBEST is marginally better at predicting teaching success, though the results were statistically insignificant and recommendations are made for a follow-up study. Post-hoc analysis of the discriminative utility of both tests show that the TBEST results show more useful shades of distinction between candidates while the TAST results place the majority of students in a `fair' category which requires secondary interviews to assess teaching ability.

本研究旨在探討四技二專英文科測驗與高職英文課程綱要之互動，分別從以下三方面來探討: 一、四技二專英文試題的發展歷程。二、高職英文課程綱要在不同階段的特色。三、四技二專英文科試題與高職英文課程綱要之互動。 本研究採試題定性分析法，以分析81年至97年四技二專英文科試題及76、87、95年教育部所頒布之高職英文課程綱要，同時參考相關之測驗、教學研究論文，探討四技二專英文科測驗與高職英文課程綱要之互動。主要發現如下： 第一， 81-87年四技二專英文科測驗的重點在於僅測驗學生單項的語言知識，且偏重學生的記憶力而非綜合分析的能力，題幹也較少提供足夠的情境及線索供受試者參考。測驗的重點與課程綱要在某方面有相呼應之處，例如著重文法及單項的語言知識；閱讀測驗的取材雖多樣化，設計卻有不當之處；另外, 課程綱要雖重視聽說讀寫技巧, 但測驗僅能測出讀的能力。 第二，88-89 年試題轉變為漸漸著重整合式試題及語言的使用，例如，發音及拼字不再出現，且因課程綱要鼓勵老師使用溝通式教學法，所以測驗的特質亦略具溝通式語文學習，取材的內容著重與生活相關。當時的課程綱要亦有此特色。然聽、說、寫能力並沒有被測驗出來，題幹所提供的線索不足，文法測驗停留於單句的測驗，都是與課程綱要不符合之處。 第三，90-97年為試題產生最大變革的階段，90年教育部成立技專校院入學測驗中心，負責命題、建立題庫、分析試題、開發新題型等工作。因此90年度之後考試題型固定，溝通式語文教學及測驗目標確定，多為整合式題型，題幹有足夠的線索及情境，整體理解重於局部理解。例如：文法測驗併入克漏字測驗, 閱讀測驗需應用閱讀策略。此時的課程綱要著重後設認知理論，要學生培養有效的學習策略，測驗有反應此特色。但聽、說、寫能力並沒有被測驗出來,這違反課程綱要的目標。 此研究結果對於四技二專英文科測驗及高職英文教學有相當的啟示，是值得投注更多關注與研究的領域。

國民中學基本學力測驗（簡稱「基測」）英語科屬高代價之語言測驗，測驗結果攸關每年三十萬左右國中畢業生的升學依據，影響層面甚廣，測驗對教學的影響即語言測驗文獻中所探討之回沖效應。 本研究旨在探討基測英語科對台灣北部地區九年級學生英語學習的回沖效應，以問卷調查台灣北部地區（基隆市、台北市、台北縣、桃園縣、新竹市、新竹縣）九年級學生，從以下四個面向探討基測英語科所造成的回沖效應：（1）學生對基測英語科試題的了解及看法；（2）學生的學習目的；（3）學生的學習方式；（4）課堂活動。本研究以四個行政層級（直轄市、市、鎮、鄉）作為抽樣依據，共計發出1,490份問卷。 研究主要結果如下： 一、大多數學生了解基測英語科之題型及測驗目的。僅有半數學生認為試題簡單，進一步分析顯示，比較了解題型及目的的學生認為試題簡單。 二、學生學習目的與《國民中小學九年一貫課程綱要》英語課程目標一致。不以基測作為主要英語學習目標的學生英語成績較好。 三、學生最常從事的學習活動是聽英語歌曲、看英語影片或節目，也會學習千字表之外的字詞。大部分的學生沒有藉由參考書、參加校內課後輔導或校外補習來準備基測英語科。 四、最常見的課堂活動是老師帶唸課文或單字、文法練習，及寫測驗卷或聽老師檢討內容。相較之下，口說聽力的教學活動較少。 結果顯示基測英語科對學生的學習影響不大，但對老師的課堂教學產生負面回沖效應，進而影響學生學習。綜合研究結果，本研究建議教師應增加課堂教學活動的多樣性，減少測驗相關教材及練習，以培養學生學習英語的興趣與方法。 / The effect of testing on teaching and learning is generally known as washback in the field of language testing and assessment. Despite a growing attention to the concept of washback, most washback studies focus on teachers, not on learners as test takers, who are the most important stakeholders in testing. The purpose of this study is to investigate the washback effect of the Basic Competence English Test (BCET), a high-stakes language test, on 9th-graders in northern Taiwan. The method of questionnaire survey was used to collect students’ perspectives on the following four areas: (a) perceptions of the BCET, (b) learning motivation, (c) learning activities, and (d) class activities. A convenient sample of a proportionate size was selected based on four administrative levels (i.e., national, municipal, town, and village levels) from Keelung City, Taipei City, Taipei County, Taoyuan County, Hsinchu City, and Hsinchu County. Altogether 1,490 ninth-graders from 42 classes at 22 schools were selected for the survey across northern Taiwan. The major findings of the study are as follows: 1. Most 9th-graders in northern Taiwan were aware of the BCET test format and the BCET test purpose. Nearly half of them found the BCET not to be easy. The more the students understood the BCET format, the more likely they thought that the BCET was easy. Similarly, the more the students understood the BCET purpose, the more likely they regarded the BCET as an easy test. 2. Three top motivations for the students to learn English coincide with the overall objectives of English language education in Grade 1-9 Curriculum. Students’ English achievement gets better if they do not regard the BCET as the primary goal of language learning. 3. Students’ most common learning activities include listening to English songs, watching movies/programs in English, and learning words and phrases beyond the word list of 1,000 basic English vocabularies. The BCET does not drive students to undertake the following practices: using practice books for examinations, attending cram schools, and attending self-study classes for the test at school. 4. It was common for the 9th-graders to do practice tests or mock examinations in class, and listen to their teachers explaining the test items. Instruction relating to formal aspects of English also occurred frequently in English language courses, while few aural/oral practice activities were undertaken in classroom teaching. Based on the findings of the BCET washback effects on students and teachers, this study suggests that English teachers should attempt to employ various types of classroom activities, as proposed in Grade 1-9 Curriculum, rather than relying on examination-oriented materials and practices, so as to promote learning interests and develop effective methods of learning.