Back to search

University of Crete (UOC)

Κατασκευή μηχανογραφημένης προσαρμοστικής εξέτασης μέσω των στατιστικών υποδειγμάτων της θεωρίας απόκρισης ερωτήματος (IRT)

Abstract

dc:description

Introduction and Purpose: Traditional assessment methods (Paper and Pencil, P&P) in Higher Education present significant limitations regarding accuracy, efficiency, and psychological impact, particularly in subjects like Statistics. The purpose of this doctoral dissertation was the holistic investigation of Computerized Adaptive Testing (CAT) as an alternative, examining psychometric, psychological, and metacognitive dimensions. Methodology: The research was conducted in two main phases. In the first phase (Calibration), an item bank was developed through the psychometric analysis of data from a large initial sample (N=680), estimating the Item Response Theory (IRT) parameters (difficulty, discrimination) of the items based on the 2-Parameter Logistic (2PL) model. In the second phase (Main Study), 213 undergraduate students from a Department of Education participated in a quasi-experimental, within-subjects, counterbalanced design. Participants were assessed using both a traditional (P&P) test and the calibrated CAT system. Data collected included performance, time, anxiety, and perceptions. Phase 2 analyses included comparative tests, Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA), as well as Logistic and Linear Regression. Results: The findings demonstrated the clear superiority of CAT. Psychometrically, it was significantly more efficient (83% time reduction), achieving accuracy with ~10 items. Students achieved significantly higher performance in CAT (M=5.40) compared to P&P (M=1.92), as the algorithm overcame the "floor effects" of the P&P test. Psychologically, CAT induced significantly less anxiety, increased self-confidence, and was preferred by the majority (55.4%). Psychometric analysis (EFA and CFA) revealed and validated a three-factor model of the CAT experience, distinguishing the dimensions of "Practicality" (F1), "Trust" (F2), and "Affect" (F3), with excellent model fit to the data (e.g., CFI=.976, SRMR=.036). Logistic Regression (Push-Pull model, AUC=0.968) showed that preference for CAT is predicted by low P&P performance and high P&P anxiety (Push factors), as well as by high Trust (F2) in the CAT method and more time spent in CAT (Pull factors). Linear Regression (Adjusted R²=0.588) indicated that performance in CAT is strongly predicted by knowledge (as measured by P&P performance), test time in the CAT, number of items (negatively), and the student's perceived performance in the CAT. The low predictive power of subjective experience alone supports the objectivity of CAT measurement, as performance was determined almost exclusively by objective factors (knowledge, behavior) rather than by students' affective perceptions. Finally, the research highlighted the trade-off between psychometric optimization and the psychological experience of the examinees, documenting "Metacognitive Pessimism" (systematic underestimation of performance in CAT) and the "Paradox of the High-Ability Student" (lower preference despite high performance). Conclusions and Contribution: This dissertation substantiates that CAT is a superior assessment method, capable of enhancing accuracy and fairness while reducing anxiety and increasing self-confidence of the examinees. The original contribution lies in the development and initial psychometric validation (via EFA and CFA) of the CAT experience model, the empirical demonstration of the psychological and metacognitive consequences of algorithmic design, and the quantification of factors (Push-Pull) determining acceptance and success in adaptive environments.

Degree

thesis:*
Grantor dc:publisher
University of Crete (UOC)
Year dc:date
2025

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Eleftherakis, Georgios
  • Ελευθεράκης, Γεώργιος

Subjects

dc:subject × 42

Rights

Language dc:language
gre

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/61652

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Eleftherakis, Georgios; Ελευθεράκης, Γεώργιος. Κατασκευή μηχανογραφημένης προσαρμοστικής εξέτασης μέσω των στατιστικών υποδειγμάτων της θεωρίας απόκρισης ερωτήματος (IRT). University of Crete (UOC), 2025. http://hdl.handle.net/10442/hedi/61652