Questions have been raised concerning the equivalency of the January, June, and August forms of the British Columbia provincial Grade 12 examinations for a given subject. The procedure for constructing these examinations has been changed as of the 1990/91 school year. The purpose of this study was to duplicate this new procedure and assess the equivalency of the forms that resulted. An examination construction team, all of whom had previous experience with the British Columbia Ministry of Education's Student Assessment Branch, simultaneously constructed two forms of a Biology 12 examination from a common table of specifications using a pool of multiple choice items from previous examinations. A sample of students was obtained in the Okanagan, Thompson, and North Thompson areas of British Columbia. Both forms were administered to each student, as required by the test equating design (Design II (Angoff, 1971)) chosen. The data sample consisted of responses from 286 students. The data were analyzed using a classical item analysis (LERTAP, Nelson, 1974) followed by a 2x2 order-by-form fixed effects ANOVA with repeated measures on the second factor. Item analysis revealed all items on both forms performed satisfactorily, ruling out an alternate hypothesis of flawed items being the cause of the lack of equivalence found. Results showed a significant (p<.05) difference in the means of the two forms, no significant (p>.25) order effect, and a significant (p<.25) order-by-form interaction. Linear and equipercentile equatings were carried out. The linear and equipercentile equatings yielded very similar results. Equating errors, judged using the conditional root mean square error of equating, was 4.86 points (9.35%) for both the equatings. Equivalency was also judged employing a graphical procedure in which the deviation of the equating function from the identity function was plotted with error bands produced using the standard error of equating. The procedure showed the two forms to be nonequivalent, particularly for the lower scoring students. The source of the nonequivalency was investigated by separating the forms into three subtests based on the pairs of items possessing or lacking item statistics at the time of test construction. The linear equating followed by the graphical analysis was repeated for the pairs of subtests. The pairs of subtests comprised of item pairs for which difficulty (p) values were present at time of construction for one or both of the items in an item pair were found to be equivalent. In contrast, the pair of subtests comprised of items for which p values were unavailable for either item in an item pair at time of construction were found to be not equivalent. It was concluded that the examination construction procedure in its present state cannot be relied on to produce equivalent forms. An experienced examination construction team was unable to accurately match items based on the level of difficulty for items which did not have prior item statistics. As such, a necessary requirement for construction of equivalent forms is that item statistics be present at the time of construction. / Education, Faculty of / Graduate

Test-wiseness, possessed in different amounts by different individuals, is the ability to use test format, test characteristics, and/or the testing situation to receive a high score. As such, test-wiseness is an unwanted source of variance which can inflate test scores, thus invalidating test results. Problems of inappropriate interpretation may arise when test scores are affected by test-wiseness. The present study addressed the relationship between test-wiseness and English ability, as measured by the British Columbia Provincial English 12 Examination for June of 1989. The English 12 examination contained both selection and supply items. This provided an opportunity to examine both types of items and their relationship to test-wiseness. Previous research had focussed on selection items. To provide a framework for understanding and presenting the results the present research was divided into two substudies. Substudy I addressed questions concerning the nature and strength of the relationship of test-wiseness to the selection, short-answer, and extended-answer items of the English 12 examination. The selection items of the English 12 examination provided reference for interpreting the results for the short-answer and extended-answer items. Test scores were adjusted for the presence of verbal ability and it was found, as previous research indicated, that test-wiseness and verbal ability were moderately correlated. To further clarify the concept of test-wiseness, differences between test-wise and test-naive students were examined in terms of means and variability on selection, short-answer, and extended-answer items of the English 12 examination. The results of the study are based on test data for 735 students collected from April to June of 1989. Each student completed the Test of Test-Wiseness (TTW), Language Proficiency Index (LPI), and a form containing ethnographic information. Test scores for the English 12 examination were provided by the Ministry. Based on the analyses of test data for 735 grade twelve students, test-wiseness accounted for less than four percent of the variance on the English 12 examination for selection, short-answer, and extended-answer items on Substudy I. These results were found to be statistically significant. Results for Substudy II indicated that there were differences between test-wise and test-naive students in terms of means for the selection and short-answer items. Results for the extended-answer (essay) item were non-significant. There were no differences in variability between the test-wise and test-naive samples for any of the item types. The results of the present study will be of interest to those involved in constructing the English 12 examination, as well as grade 12 teachers and students. The test score on the English 12 examination accounts for 40% of a student's English 12 final grade, with a student's graduation or failure based upon these results. As such, English 12 examination scores should be as accurate and valid as possible. / Education, Faculty of / Educational and Counselling Psychology, and Special Education (ECPS), Department of / Graduate

The objectives of this study were to investigate the cultural differences in test-wiseness between Chinese and Canadian students; to study the impact of test-wiseness on British Columbia Grade 12 provincial examination results among Chinese students in British Columbia; and to examine the relationship of test-wiseness, and four correlates (verbal ability, achievement motivation, length of residency in Canada, multiple choice experience) among the Chinese students. The Test of Test-wiseness and Student Survey questionnaire were administered to 41 Grade 12 foreign Chinese students who wrote the provincial examinations in January 1990. Completed test data were compared with the same data from Canadian and Chinese immigrant students who wrote the provincial examination in June 1989. The data were analyzed by a linear regression model. The results indicated that ethnicity accounted for a significant amount of variance in test-wiseness. Foreign Chinese students were significantly less test-wise than both Canadian and Chinese immigrant students. Foreign Chinese students scored significantly lower than the other two groups in absurd-options, different-options, stem-options link and guessing subtests in the Test of Test-wiseness. The results also showed that test-wiseness did not account for a significant amount of variance in provincial examination scores (English 12 and Algebra 12). The four correlates together accounted for very little variance in test-wiseness. None of them significantly correlated with test-wiseness, except that length of residency in Canada correlated positively with test-wiseness. Finally, the three ethnic groups showed no significant differences in achievement motivation but foreign Chinese students scored significantly lower than Chinese immigrant students who scored in turn lower than Canadian students in verbal ability. The findings imply that language plays a crucial part in test-taking situations. With inadequate English language skills, foreign Chinese students were unable to satisfactorily comprehend the test questions, and hence their test performance were very close to what would be expected from random guessing. It is suggested that these students require examination coaching skills as well as improvement in English language skills. / Education, Faculty of / Educational and Counselling Psychology, and Special Education (ECPS), Department of / Graduate