Student misconceptions in science are common and may be present even for students who are academically successful. Concept inventories, multiple-choice tests in which the distractors map onto common, previously identified misconceptions, are commonly used by researchers and educators to gauge the prevalence of student misconceptions in science. Distractor analysis of concept inventory responses could be used to create profiles of individual student misconceptions which could provide deeper insight into the phenomenon and provide useful information for instructional planning, but this is rarely done as the inventories are not designed to facilitate it. Researchers in educational measurement have suggested that diagnostic cognitive models (DCMs) could be used to diagnose misconceptions and to create such misconception profiles. DCMs are multidimensional, confirmatory latent class models which are designed to measure the mastery/presence of fine-grained skills/attributes. By replacing the skills/attributes in the model with common misconceptions, DCMs could be used to filter students into misconception profiles based on their responses to concept inventory-like questions. A few researchers have developed new DCMs that are specifically designed to do this and have retrofitted data from existing concept inventories to them. However, cognitive diagnostic assessments, which are likely to display better model fit with DCMs, have not been developed. This project developed a cognitive diagnostic assessment to measure knowledge and misconceptions about Newton's laws and fitted it with the deterministic input noisy-and-gate (DINA) model. Experienced physics instructors assessed content validity and Q-matrix alignment. A pilot test with 100 undergraduates was conducted to assess item quality within a classical test theory framework. The final version of the assessment was field tested with 349 undergraduates. Results showed that response data displayed acceptable fit to the DINA model at the item level, but more questionable fit at the overall model level; that responses to selected items were similar to those given to two items from the Force Concept Inventory; and that, although all students were likely to have misconceptions, those with lower knowledge scores were more likely to have misconceptions. / Doctor of Philosophy / Misconceptions about science are common even among well-educated adults. Misconceptions range from incorrect facts to personal explanations for natural phenomena that make intuitive sense but are incorrect. Frequently, they exist in people's minds alongside correct science knowledge. Because of this, misconceptions are often difficult to identify and to change. Students may be academically successful and still retain their misconceptions. Concept inventories, multiple-choice tests in which the incorrect answer choices appeal to students with common misconceptions, are frequently used by researchers and educators to gauge the prevalence of student misconceptions in science. Analysis of incorrect answer choices to concept inventory questions can be used to determine individual student's misconceptions, but it is rarely done because the inventories are not known to be valid measures for this purpose. One source of validity for tests is the statistical model that is used to calculate test scores. In valid tests, student's answers to the questions should follow similar patterns to those predicted by the model. For instance, students are likely to get questions about the same things either all correct or all incorrect. Researchers in educational measurement have proposed that certain types of innovative statistical models could be used to develop tests that identify student's misconceptions, but no one has done so. This project developed a test to measure knowledge and misconceptions about forces and assessed how well it predicted student's misconceptions compared to two statistical models. Results showed that the test predicted student's knowledge in good agreement and misconceptions in moderate agreement with the statistical models; that students tended to answer selected questions in the same way that they answered two similar questions from an existing test about forces; and that, although students with lower test scores were more likely to have misconceptions, students with high test scores also had misconceptions.
| Identifer | oai:union.ndltd.org:VTETD/oai:vtechworks.lib.vt.edu:10919/105262 |
| Date | 12 October 2021 |
| Creators | Norris, Mary Armistead |
| Contributors | Counselor Education, Skaggs, Gary E., Kniola, David J., Miyazaki, Yasuo, Glasson, George E. |
| Publisher | Virginia Tech |
| Source Sets | Virginia Tech Theses and Dissertation |
| Detected Language | English |
| Type | Dissertation |
| Format | ETD, application/pdf |
| Rights | In Copyright, http://rightsstatements.org/vocab/InC/1.0/ |
Page generated in 0.0148 seconds