Matrices reasoning tests, which contain missing pieces in matrices that participants attempt to figure out, are one of the most popular types of tests to measure general intelligence. This thesis introduces several methods to develop matrices items, and presents them in different test forms to assess general intelligence. Part 1 introduces the development of a matrices test with reference to Carpenter’s five rules of Raven’s Progressive Matrices. The test items developed were administered together with the Standard Ravens’ Progressive Matrices (SPM). Results based on confirmatory factor analysis and inter-item correlation demonstrate good construct validity and reliability. Item characteristics are explored with Item-Response Theory (IRT) analyses. Part 2 introduces the development of a large item bank with multiple alternatives for each SPM item, with reference to the item components of the original SPM. Results showed satisfactory test validity and reliability when using the alternative items in a test. Findings also support the hypothesis that the combination of item components accounts for item difficulty. The work lays the foundation for the future development of computer adaptive versions of Raven’s Progressive Matrices. Part 3 introduces the development of an automatic matrix item generator and illustrates the results of the analyses of the items generated using the distribution-of-three rule. Psychometric properties of the items generated are explored to support the validity of the generator. Figural complexity, features, and the frequency at which certain rules were used are discussed to account for the difficulty of the items. Results of further analyses to explore the underlying factors of the difficulty of the generated items are presented and discussed. Results showed that the suggested factors explain a substantial amount of the variance of item difficulty, but are insufficient to predict the item difficulty. Adaptive on-the-fly item generation is yet to be possible for the test at this stage. Overall, the methods for creating matrices reasoning tests introduced in the dissertation provide a useful reference for research on abstract reasoning and fluid intelligence measurements. Implications for other areas of psychometric research are also discussed.

Not available / text

This case study examined the efficiency and precision of computer classification and adaptive testing to elicit responses from test coordinators on implementing a high-stakes computer-based testing. Test coordinators from five elementary schools located in a Georgia school district participated in the study. The school district administered state-made, high-stakes tests using paper and pencil; locally developed tests via the computer or paper and pencil. A post-hoc simulation program, Comprehensive Simulation of Computerized Adatpive Testing, used 586 student item responses to produce results with a variable termination point and classification termination point. Results from the simulation were analyzed and used in the case study to elicit interview responses from test coordinators. The photographs of computer-labs and test schedule documents were collected and analyzed to validate school test coordinators' responses. Test coordinators responded positively to the efficiency and precision of simulation results. Some test coordinators preferred the use of computer-adaptive tests for diagnostic purposes only. Test coordinators experiences focused on the security, the emotions, and the management of testing. The findings of this study will benefit those interested in implementing a high-stakes, computer-based testing program by recommending a simulation study be conducted and feedback by solicited from test coordinators prior to an operational test administration.

This quantitative study examined seventh-grade reading scores to determine the extent to which certain demographic variables (race/ethnicity, gender, socioeconomic status) explain and MAP reading scores predict reading scores on the State of Texas Assessment of Academic Readiness (STAAR) in a selected northeast Texas public school. Standardized assessments only compare the relative performance of an individual student to other groups of students using scaled scores, which can vary from year to year and from state to state. With the advent of computer adaptive testing, this study provides information on the predictive validity of benchmark assessments. Specifically, this study looked for predictive evidence that indicates how accurately test data can predict criterion scores. Findings revealed, through a multiple regression analysis, that the fall MAP Rasch Unit (RIT) scores predicted the STAAR scale scores. Using SPSS version 22, the data were entered and analyzed in a multiple regression model to determine the presence of a statistical trend or lack thereof. Demographic data and MAP scores were entered into the regression model to examine the predictive validity of the MAP assessment in determining student performance on the STAAR seventh-grade state-mandated reading assessment. The statistical analysis revealed that MAP RIT scores explain a significant variance related to seventh-grade STAAR reading scale scores. There is a vital need for tools that improve a student's academic development and MAP assessments have been found to predict performance on state-mandated assessments.

The paper deals with the current perspective of computer aided assessment of students, particularly with computer adaptive testing based on Item Response Theory. Although this testing method has become a world standard, in the Czech Republic it is still, unfortunately, rarely used. The aim of this paper is to describe the current state of the testing techniques used and to create a functional dll library, which will provide basic functionality needed for successful launching of a Web interface for computer adaptive testing of students. In addition, the library should serve as a support to authors in developing tests based on Item Response Theory. The first part provides a basic description of educational assessment methods currently used, and gives an insight into the matters of creating professional testing tools. Other chapters depict the operation of Classical Test Theory, which is now used as a cornerstone of the test development, and further it presents Item Response Theory, as a possible successor to the Classical theory. Both theories described are compared at the end. The second part of the paper is then divided into a theoretical part, which describes the possibility of computer adaptive testing, gives and overview of its origins and development, presents essential forms, and highlights the advantages and disadvantages of this testing method. The practical part contains illustrative examples and descriptions of the methods enabled by the presented dll library.

Technology has provided the means for the creation of many tools to facilitate the teaching and learning of foreign languages. These tools include computer programs designed to aid language learning by providing various levels of control to the language learner. This control allows the learner to make decisions regarding some or all of the elements of a program such as the pace, sequence, and content to name but a few. Because the amount of learner control can be varied, many research efforts have sought to determine the optimal level of control for learning. These efforts have produced mixed results, with some research suggesting that learners perform better with less control while other findings suggest the opposite. The purpose of this research was to investigate the use of remediation, a computer-adaptive control, in Swahili 101 university level courses and its effect on achievement and time. Participants included Swahili language learners from two universities, three colleges, and one student not affiliated with any school. The study required that participants complete a pretest, a background survey, the Swahili 101 online lesson materials, and an opinion survey. “Gate pages," or webpage-based assessments, were used throughout the online course to assess participant progress. The participants were randomly assigned to control and experimental groups. Those in the control group completed the online component at their own discretion, working through assigned tasks with review as desired. Those in the experimental group were required to obtain a minimum score on each lesson's gate page in order to advance in the lesson and thus in the course. If that score was not met, then the software assigned a series of remediation or review pages. Scores from the gate pages and time spent on the gate pages were stored and analyzed for both groups. The results from this study suggest that language learners benefit from computer intervention and guidance (remediation). The participants in the experimental group learned more Swahili than the control group, despite spending the same amount of time on the gate pages as the control group. Therefore, remediation, as defined and implemented in this study, can increase language learning while at the same time not requiring the learners to significantly increase the initial time they spend responding to questions presented on the gate pages.