This study investigates the psychometric issues and viability of cellphone-based-testing, a novel test administration modality whereby test-takers use a cellphone to respond to items on a web-based assessment. The study explored mode-dependent differences in scores from a web-based version of the Self-Monitoring Scale (SMS) administered across two modalities: desktop computer and cellphone. The selection of the SMS was based on several pre-established criteria. The instrument was simple and brief. Its text-based items included true/false response categories. Its rights of use fell under public domain and it had been previously validated for online administration. The study includes a comprehensive overview of recent literature related to the topic of psychometric equivalency and incorporates numerous methodological approaches to determine test score equivalence, including: comparisons of central tendency, dispersions, and rank order; the Kolmogorov-Smirnov test of equal score distributions; the Pitman procedure for detecting differences in reliability coefficients; a confirmatory factor analysis of the equality of factor structures using LISREL; and an analysis of differential item functioning based on item response theory using BILOG-MG. The study employed a counterbalanced repeated measures design whereby 234 participants took an 18-item web-based version of the SMS using a desktop computer and/or a cellphone. The psychometric equivalency of scores from the two modes of administration was analyzed. All statistical comparisons provided overwhelming support for one general conclusion: There were no mode-dependent differences in scores on the web-based version of the SMS when administered by desktop computer versus cellphone. The study also explored participants’ attitudes toward using cellphones as a test-taking tool. The participants correctly anticipated that their scores would not be affected by using a cellphone, but they categorically rated the cellphone as less enjoyable, more difficult, and more cumbersome than a desktop computer. However, one cannot ignore the tendency of our modern society for being obsessed with information on demand. As cellphone technology continues to improve and the text-messaging generation begins to influence the field of educational and psychological measurement, cellphone-based-testing will likely become an accepted standard for both academic and clinical practice.
| Identifer | oai:union.ndltd.org:MSSTATE/oai:scholarsjunction.msstate.edu:td-5597 |
| Date | 03 May 2008 |
| Creators | Edwards, John Francis |
| Publisher | Scholars Junction |
| Source Sets | Mississippi State University |
| Detected Language | English |
| Type | text |
| Format | application/pdf |
| Source | Theses and Dissertations |
Page generated in 0.0026 seconds