在一場競爭性的考試中，我們如何決定要錄取或是淘汰這個考生？傳統的紙筆測驗方式固定題目總數，考生回答相同的題目，60分以上為及格。隨著電腦科技的快速發展，測驗型式也由紙筆轉換成電腦操作，也就是電腦化測驗。所謂電腦化效標參照測驗（computerized criterion-referenced test）即是把考生能力分成兩個以上的程度區間，藉由考生的答題狀況來判斷考生應歸屬於哪個區間。這種測驗方式與傳統測驗不同的是：電腦化測驗是依據考生的答題表現來給題，考生能力越偏離分段點（thresholds），需要的題數就越少；越接近分段點，需要的題數就越多。 在這篇論文中，我們運用兩個參數的羅吉斯模型（two-parameter logistic model）來估計考生之於試題的答對機率。藉由電腦模擬來探討結合貝它保護（beta-protection）方法和適性測驗對平均測驗題數及誤判率（亦即考生真正的能力與電腦判斷的區間不同）的影響。在模擬過程中，我們也介紹了試題參數的選擇情形，估計考生能力的方法以及在貝它保護下，停止選題的規則。根據這些原則，電腦模擬結果證明使用適性測驗加上貝它保護方法能夠有效地控制誤判率在規定的範圍內，程度不同的考生也能控制有不同的測驗題數。 / In a traditional Paper-and-Pencil (p-and-p) test, all examinees have same test items and the number of items is fixed. The examinee fails or passes the exam depends on if his/her test score exceeds a predetermined scores, say, 60 out of 100. However, with the rapid advancement of modern computer technology, the test form has been converted from p-and-p to computer terminal. Computerized criterion-referenced classify the examinees into more than two categories according to his/her answers to the items. It differs from the conventional standardized test in that the selection of test items is tailored to each examinee’s ability level. Typically, those examinees with high ability or low ability will have shorter average test length (ATL) than examinees with ability that close to thresholds. In this thesis, we assume that the probability of choosing correct response to an item follows a two-parameter logistic (2-PL) model. Our goal is to study the performance of ATL and misclassification rate (MR) using beta-protection method and adaptive sequential item selection. On the simulation procedures, we also introduce the selection rule of item parameter, the methods used to estimate an examinee’s ability, and the stopping rule with beta-protection. Simulation results show that using adaptive test and beta-protection method can control the MR within specified level and the number of test items required depends on the examinee’s ability.

Although classroom assessment is one of the most frequent practices carried out by teachers in all educational programs, limited research has been conducted to investigate the dependability and validity of criterion-referenced tests (CRTs). The main purpose of this study is to develop a criterion-referenced test for first-year Japanese university students in a general English program. To this end, four research questions are formulated: (a) To what extent do the criterion-referenced items function effectively?; (b) To what extent do the facets of persons, items, sections, classes, and subtests contribute to the total score variation in two CRT forms?; (c) To what extent are two CRT forms dependable when administered as pretests and posttests?; and (d) To what extent are two CRT forms valid when administered as pretests and posttests? Two CRT forms made up of vocabulary (k = 25), listening (k = 20), and reading (k = 25) subtests were administered to 249 students using a counterbalanced design. Criterion-referenced item analyses showed that most items were working well for criterion-referenced purposes. Both univariate and multivariate generalizability studies indicated that the most of the variance was accounted for by the interaction effect, followed by the items effect, and then by the persons effect. FACETS analyses showed the separation for all the facets accounted for in the analyses and showed that item separation was greater than person separation. This indicated that the students' ability estimates were similar due to their having taken a placement test, whose results were used to form proficiency-based classes. Both univariate and multivariate decision studies indicated that the CRT forms were moderately to highly dependable. The content validity of the CRT forms was supported because the test content was strongly linked to what was taught in class. The construct validity was supported mainly because a fair amount of score gain was observed. This study elucidates how the statistical analyses used in this study can be applied to CRT development, and how CRT development can be carried out as part of curriculum development. / Educational Administration