論文名稱：線上題庫與適性測驗整合系統之發展研究 頁數：337 校所系別：國立政治大學教育學系 畢業時間及摘要別：九十學年度第二學期博士論文摘要 指導教授：林邦傑博士、余民寧博士 研究生：陳新豐 論文摘要內容 本研究係結合工具研發、理論驗證與效能評估的研究，旨在開發一個建構在全球資訊網的「線上題庫與適性測驗整合系統」，以提供教師在網際網路環境下的輔助教學評量系統，除了可動態新增題庫之外，並能針對學生提供童身訂做的適性測驗。因此，研究的兩個核心主軸為「線上題庫與適性測驗整合系統」之開發與「線上題庫建置」的理論驗證。 依循這兩個核心主軸，本研究的研究目的有三：（一）開發線上題庫與適性測驗整合系統。（二）驗證建置題庫的相關理論。（三）評估整合系統運作效能與使用者滿意程度。 為達成這三個研究目的，研究者採用結構分析中，Sehlly、Cashmen和Rosenblatt (2001)所提出的「系統開發生命週期」，將「線上題庫與適性測驗整合系統」開發過程分為「系統規劃」、「系統分析」、「系統設計」、「系統建置」、「系統運行與支援」等五個階段逐步開發。 研究樣本方面，第一次預試選用台南市崇明國中三年級學生115人，第二次預試選用台南市建興國中三年級學生191人，正式施測樣本則是台灣地區北、中、南、東、離島等共計九校2567位國中三年級學生為研究對象。此外，在需求調查報告部分，共調查十五位專家對開發系統的意見。 就研究工具來說，本研究主要研究工具為「線上題庫與適性測驗之整合系統」，另外，「功能需求調查問卷」、「硬體設備」、「軟體工具」、「系統評估量表」也是本研究的研究工具。 就資料處理來說，本研究運用ITEMAN、BILOG、MatLab和SPSS套裝軟體進行資料處理，所採用的統計方法包含古典測驗理論與試題反應理論等理論來分析，計有試題分析、IRT三參數估計、因素結構分析等。 研究結果部分，本研究得到如下結論：（一）線上題庫與通性測驗兩個系統可整合為一。（二）線上題庫與遍性測驗之整合系統具有多項功能。（三）結構化分析中之系統開發生命週期是開發整合系統的理想方法。（四）題庫等化轉換常數方法以Mean/Mean和Haebara等方法較佳。（五）線上測驗與紙筆測驗的試題訊息量相近，但難度偏高。（六）線上測驗連結效益良好。（七）本整合系統運作效能良好。（八）使用者對整合系統之功能感到滿意。根據研究結論，本研究針對工具研發、題庫建置、效能評估等提出具體建議。 關鍵字：試題反應理論、題庫、等化、電腦化適性測驗、系統開發生命週期 / A Reserach In The Development of An Integrated On-Line Item Bank and Computerized Adaptive Testing System Abstract This research is to develop an integrated internet system of on-line item bank and computerized adaptive testing (the "System"), which is comprised of the teaching tool development, theory verification, and efficiency evaluation. Except for the addition of new item bank dynamically, the System, an auxiliary teaching evaluation system for teachers, can also provide customarily made adaptive testing for students. Therefore, to develop an integrated on-line item bank and computerized adaptive testing system and to verify the theory of on-line item bank development constitute the two core spindles of this research. Following the aforementioned research spindles, the main purposes of this research are going to: (A). Develop an integrated on-line item bank and computerized adaptive testing system. (B). Verify the related theories concerning the development of on-line item bank. (C). Evaluate the operating efficiency of such System and the degree of users' satisfaction. The "systems development life cycle" (Sehily, Cashmen Rosenblatt, 2001), a structured analysis method, is adopted to conduct the research. The development process of an integrated on-line item bank and computerized adaptive testing system is divided into 5 separate and successive stages, starting from system planning, system analysis, system design, system development, to system operation and support. In terms of research sampling, the selected samples in the first preliminary testing are 115 ninth-grade students of Chiung-Ming High School in Tainan City, Taiwan. The selected samples in the second preliminary testing are 191 ninth-grade students of Cheng-Sing High School in Tainan City, Taiwan. The third and official sampling is 2,567 ninth-grade students who were selected from a total of 9 high schools ranging from Northern, Central, Southern, and Eastern Taiwan, and islands adjacent to Taiwan. Furthermore, in the demand side, an investigation has been conducted to consult with 15 teaching experts for their professional opinions in regard to such System development. As far as research tools are concerned, except for the main research tool - the integrated on-line item bank and computerized adaptive testing system, other research tools employed consist of functional demand questionnaires, hardware equipments, software tools, and scales for system evaluation. In the aspect of data processing, ITEMAN, BILOQ MatLab, and SPSSapplication softwares are used to perform the data processing. The statistical method,like classical true score theory and item response theory and etc., is applied to conduct the following analyses: item analysis, IRT three-parameter estimate, structured analysis of elements, and etc. The results of this research lead to the following conclusion: 1. The on-line item bank: system and the adaptive testing system, the two separate systems, can be integrated into one system. 2. The integrated on-line item bank and adaptive testing system can play multiple functions. 3. "Systems development life cycle" in the structured analysis is an ideal manner to develop an integrated system. 4. "Mean/Mean", "Haebara" and, etc. are the better methods to perform the item bank equating and constant conversion. 5. For users, the information volume provided by on-line testing and traditional written testing are quite similar, but the degree of difficulty of on-line testing is higher than that of traditional written testing. 6. The linking effect of on-line testing is fair. 7. The operating efficiency of the integrated system is fair. 8. Users are satisfied with the functions of the integrated system. Based on research conclusions drawn thereon, suggestions for tool development, item bank development, and efficiency evaluation are also provided. Keywords: item response theory, item bank, equating, computerized adaptive testing,systems development life cycle

本論文探討當自變數存在測量誤差時，羅吉斯迴歸模型的估計問題，並將此結果應用在電腦化適性測驗中的線上校準問題。在變動長度電腦化測驗的假設下，我們證明了估計量的強收斂性。試題反應理論被廣泛地使用在電腦化適性測驗上，其假設受試者在試題的表現情形與本身的能力，可以透過試題特徵曲線加以詮釋，羅吉斯迴歸模式是最常見的試題反應模式。藉由適性測驗的施行，考題的選取可以依據不同受試者，選擇最適合的題目。因此，相較於傳統測驗而言，在適性測驗中，題目的消耗量更為快速。在題庫的維護與管理上，新試題的補充與試題校準便為非常重要的工作。線上試題校準意指在線上測驗進行中，同時進行試題校準。因此，受試者的能力估計會存在測量誤差。從統計的觀點，線上校準面臨的困難，可以解釋為在非線性模型下，當自變數有測量誤差時的實驗設計問題。我們利用序貫設計降低測量誤差，得到更精確的估計，相較於傳統的試題校準，可以節省更多的時間及成本。我們利用處理測量誤差的技巧，進一步應用序貫設計的方法，處理在線上校準中，受試者能力存在測量誤差的問題。 / In this dissertation, we focus on the estimate in logistic regression models when the independent variables are subject to some measurement errors. The problem of this dissertation is motivated by online calibration in Computerized Adaptive Testing (CAT). We apply the measurement error model techniques and adaptive sequential design methodology to the online calibration problem of CAT. We prove that the estimates of item parameters are strongly consistent under the variable length CAT setup. In an adaptive testing scheme, examinees are presented with different sets of items chosen from a pre-calibrated item pool. Thus the speed of attrition in items will be very fast, and replenishing of item pool is essential for CAT. The online calibration scheme in CAT refers to estimating the item parameters of new, un-calibrated items by presenting them to examinees during the course of their ability testing together with previously calibrated items. Therefore, the estimated latent trait levels of examinees are used as the design points for estimating the parameter of the new items, and naturally these designs, the estimated latent trait levels, are subject to some estimating errors. Thus the problem of the online calibration under CAT setup can be formulated as a sequential estimation problem with measurement errors in the independent variables, which are also chosen sequentially. Item Response Theory (IRT) is the most commonly used psychometric model in CAT, and the logistic type models are the most popular models used in IRT based tests. That's why the nonlinear design problem and the nonlinear measurement error models are involved. Sequential design procedures proposed here can provide more accurate estimates of parameters, and are more efficient in terms of sample size (number of examinees used in calibration). In traditional calibration process in paper-and-pencil tests, we usually have to pay for the examinees joining the pre-test calibration process. In online calibration, there will be less cost, since we are able to assign new items to the examinees during the operational test. Therefore, the proposed procedures will be cost-effective as well as time-effective.