The purpose of this study was to develop an instrument to provide valid and reliable measures of misconceptions in the area of photosynthesis and respiration among fourth, fifth, and sixth grade students in Thailand. A 17-item, two-tier multiple-choice test was developed and validated using construct-related evidence to determine the appropriateness of inferences from a set of test scores. The test was administered to 4,346 students in the fourth, fifth, and sixth grades in schools throughout Thailand. The study attempted to identify misconceptions held by Thai elementary students in the area of photosynthesis and respiration, and to examine the effects of grade level and gender on students' understandings of these concepts. A two-way analysis of variance, with grade level and gender as independent variables, was used to analyze test scores. Findings revealed that: (a) The two-tier multiple-choice test was valid and reliable; (b) mean scores among grade levels were significantly different (p=0.001); (c) mean scores between male and female genders were not significantly different (p=0.180); and (d) there was a significant interaction effect between grade level and gender (p=0.005). From these findings, the following was concluded: (a) The two-tier multiple-choice test provided valid and reliable measures of student misconceptions of selected science concepts at the elementary school level in Thailand; (b) most Thai students in grades 4, 5, and 6 held a large number of misconceptions of the selected concepts; (c) a majority of the students answered the content questions correctly, but did not understand the justification for the correct responses; (d) the higher the grade level, the greater the understanding of the concepts; and (e) female and male Thai students in grades 4, 5, and 6 did not differ significantly in their understanding of photosynthesis and respiration. / Graduation date: 1994

Differential item functioning (DIF) and differential distractor functioning (DDF) are methods used to screen for item bias (Camilli && Shepard, 1994; Penfield, 2008). Using an applied empirical example, this mixed-methods study examined the congruency and relationship of DIF and DDF methods in screening multiple-choice items. Data for Study I were drawn from item responses of 271 female and 236 male low-income children on a preschool science assessment. Item analyses employed a common statistical approach of the Mantel-Haenszel log-odds ratio (MH-LOR) to detect DIF in dichotomously scored items (Holland & Thayer, 1988), and extended the approach to identify DDF (Penfield, 2008). Findings demonstrated that the using MH-LOR to detect DIF and DDF supported the theoretical relationship that the magnitude and form of DIF and are dependent on the DDF effects, and demonstrated the advantages of studying DIF and DDF in multiple-choice items. A total of 4 items with DIF and DDF and 5 items with only DDF were detected. Study II incorporated an item content review, an important but often overlooked and under-published step of DIF and DDF studies (Camilli & Shepard). Interviews with 25 female and 22 male low-income preschool children and an expert review helped to interpret the DIF and DDF results and their comparison, and determined that a content review process of studied items can reveal reasons for potential item bias that are often congruent with the statistical results. Patterns emerged and are discussed in detail. The quantitative and qualitative analyses were conducted in an applied framework of examining the validity of the preschool science assessment scores for evaluating science programs serving low-income children, however, the techniques can be generalized for use with measures across various disciplines of research.

Thesis (Ph. D.)--Mississippi State University. Department of Curriculum and Instruction. / Title from title screen. Includes bibliographical references.

Thesis (Ph.D.)--Mississippi State University. Department of Counseling and Educational Psychology. / Title from title screen. Includes bibliographical references.

This thesis investigated the testing effect, which is the tendency of testing to enhance learning and memory retention. Specifically, the thesis examines the extent to which test-induced retention benefits extend to the questions as well as the answers; are these benefits symmetrical? The results in the laboratory experiment (Experiment 1) demonstrated that the symmetry in retention benefits is highly dependent on the accuracy in the initial test. For the items students answered correctly in the initial test, the retention benefit was found to be symmetrical across the questions and answers supporting the retrieval hypothesis of the testing effect. However, for initially incorrect items, the retention performance was substantially better for the questions than the answers suggesting an asymmetrical effect from testing, a result that is also consistent with the retrieval hypothesis. These results were replicated using educationally-relevant materials with a delay up to five and a half months in two hybrid studies combining actual classroom experiences with controlled assessments in the laboratory (Experiment 2A & 2B). A series of follow-up studies were conducted to examine the difference in retention performance of the questions and the answers for the initial incorrect items. Particularly, the experiments focused on investigating various factors that might contribute to or eliminate the difference in retention performance, including prior study (Experiment 3), verification feedback (Experiment 4A & 4B) and answer feedback (Experiment 5). The thesis also touched upon various issues related to the temporal dimension of the testing effect. The previous notion that testing slows down forgetting rate is not supported by the data. Instead, testing seems to provide a short-term insulation against immediate forgetting, but then memory for the tested materials decays in the same way as the non-tested materials.

The study takes a close look at relationships between scores on a Mathematics standardized test in two different testing formats - Multiple-Choice (MC) and Constructed Response (CR). Many studies have been dedicated to finding correlations between item format characteristics with regards to race and gender. Few studies, however, have attempted to explore differences in the performance of English Learners in a low performing, predominantly Latino high school. The study also determined relationships between math scores and gender and math scores and language proficiency, as well as relationships between CAHSEE and CST scores. Statistical analyses were performed using correlations, descriptive statistics, and t-tests. Empirical data were also disaggregated and analyzed by gender, and language proficiency. Results revealed significant positive correlations between MC and CR formats. T-tests displayed statistically significant differences between the means of the formats, with boys and English Only students having better scores than their counterparts. Frequency tables examining proficiency levels of students by gender and language proficiency revealed differences between MC and CR tests, with boys and English Only students earning better levels of proficiency. Significant positive correlations were shown between CST scores and multiple-choice items, but none were found for CST scores and constructed response items.

Multiple Choice Questions (MCQs) have long been the backbone of standardized testing in academia and industry. Correspondingly, there is a constant need for the authors of MCQs to write and refine new questions for new versions of standardized tests as well as to support measuring performance in the emerging massive open online courses, (MOOCs). Research that explores what makes a question difficult, or what questions distinguish higher-performing students from lower-performing students can aid in the creation of the next generation of teaching and evaluation tools. In the automated MCQ answering component of this thesis, algorithms query for definitions of scientific terms, process the returned web results, and compare the returned definitions to the original definition in the MCQ. This automated method for answering questions is then augmented with a model, based on human performance data from crowdsourced question sets, for analysis of question difficulty as well as the discrimination power of the non-answer alternatives. The crowdsourced question sets come from PeerWise, an open source online college-level question authoring and answering environment. The goal of this research is to create an automated method to both answer and assesses the difficulty of multiple choice inverse definition questions in the domain of introductory biology. The results of this work suggest that human-authored question banks provide useful data for building gold standard human performance models. The methodology for building these performance models has value in other domains that test the difficulty of questions and the quality of the exam takers.

Submitted by Renata Lopes (renatasil82@gmail.com) on 2016-04-07T19:50:02Z No. of bitstreams: 1 jafetealbertomabote.pdf: 1055155 bytes, checksum: 344fb0d09a786d5b0d859a578b951ff9 (MD5) / Approved for entry into archive by Adriana Oliveira (adriana.oliveira@ufjf.edu.br) on 2016-05-02T00:33:46Z (GMT) No. of bitstreams: 1 jafetealbertomabote.pdf: 1055155 bytes, checksum: 344fb0d09a786d5b0d859a578b951ff9 (MD5) / Made available in DSpace on 2016-05-02T00:33:46Z (GMT). No. of bitstreams: 1 jafetealbertomabote.pdf: 1055155 bytes, checksum: 344fb0d09a786d5b0d859a578b951ff9 (MD5) Previous issue date: 2013-08-15 / O objetivo desta dissertação é avaliar, dentro do programa de reformas na área dos exames nacionais em Moçambique a política de Perguntas de Múltipla Escolha nos Exames Nacionais da 12ª classe introduzida em 2008. Esta política foi desenhada para reduzir ou eliminar a subjetividade na classificação das respostas dos alunos, facilitar a introdução da correção eletrónica e centralizada, combater a corrupção, obter um banco de dados de resultados que possam permitir retroalimentar o sistema educativo nacional para melhorar a qualidade do ensino, reduzir o tempo gasto e o número de intervenientes no processo de correção das provas. Decorridos cinco anos desde que esta política foi implementada considera-se pertinente analisar como está sendo implementada, avaliando a sua eficácia e eficiência. Esta dissertação compreende três partes fundamentais: na primeira procura-se fazer um enquadramento das mudanças introduzidas no sistema educativo com o programa de reformas do setor público; na segunda, faz-se uma análise das políticas de educação e do sistema de avaliação de Moçambique e na terceira, propõem-se melhorias e/ou aperfeiçoamento nos exames de certificação e sobre a expansão das mudanças para o ciclo imediatamente inferior. Para atingir-se os objetivos da pesquisa fez-se uma combinação dos métodos quantitativo e qualitativo para avaliar as ações desenvolvidas, corrigir possíveis distorções, propor estratégias para a consolidação da implementação da política nos exames da 12ª classe e para a sua extenção para os exames de outros níveis. Concluiu-se que a política está sendo bem aplicada e aceite no seio da comunidade escolar embora persistam alguns desvios que merecem atenção no futuro. Deste modo, elaborou-se um Plano de Ação para a consolidação da política na 12ª classe e para a sua extensão a 10ª classe nos próximos 3 anos. / The objective of this dissertation is to evaluate within the reforms programme in the area of exams in Mozambique the Multiple Choice Question Policy in Grade 12 National Examinations introduced in 2008. This policy was designed to reduce or eliminate the subjectivity in the classification of students’ answers, facilitate the introduction of electronic and centralized marking, fight corruption, obtain an item bank of answers that may allow to support the national education system to improve the quality of education and reduce the time wasted and the number of interveners in the process of marking tests. After five years since this policy was implemented it is considered pertinent to analyse how it is being implemented, evaluating its efficacy and efficiency. This dissertation comprises three main parts: in the first, it is made the framing of the changes introduced in the educational system with public sector reform programmes; in the second, it is made an analysis of Mozambique education and evaluation system policies and in the third, it is proposed improvements and or progress in certification exams and about the expansion of the changes for the cycle immediately lower. To hit the research objectives was made a combination of quantitative and qualitative methods to evaluate the developed actions, correct possible distortions, suggest strategies to consolidate the implementation of policy in grade 12 exams and to its extension to other levels exams. It was concluded that the policy is being well applied and accepted within the school community although prevailing some deviations that deserve attention in the future. In this way, was developed an Action Plan to consolidate the policy in grade 12 and for its extension in grade 10 in the next 3 years.

The purpose of this essay then is to gain insight into the utilisation of MCQ’s within the field of law at Swedish upper secondary schools and in this question what kind of knowledge requirements can be tested with MCQ. Also, the difference in test results between non-MCQs and MCQ was analyses in large and when it came to gender. Here a MCQ test was evaluated based upon the knowledge requirements while quantitative date gathered from a utilisation of MCQ within the field of employment law was analysed. It was found that it should be possib- le to utilise MCQ within the field of law. With regards to difference in results between non- MCQ and the MCQ under scrutiny it was found that high performing students scored above one grade lower on MCQ than the average of three non-MCQ’s in other subjects. Low per- forming students experienced a little improvement in their results. There was a slightly diffe- rence when it came to score in between gender. Although the average female score on all tests where higher than their male counterpart, females scored a little lower on their first MCQ test. It was felt that the scope of this research is too small to allow any conclusions to be drawn.

Multiple-choice (MCQs) and short-answer questions (SAQs) are the most common test formats for assessing English reading proficiency. While the former provides test-takers with prescribed options, the latter requires short written responses. Test developers favor MCQs over SAQs for the following reasons: less time required for rating, high rater agreement, and wide content coverage. This mixed methods dissertation investigated the impacts of test format on reading performance, metacognitive awareness, test-completion processes, and task perceptions. Participants were eighty English as a second language (ESL) learners from a Midwestern community college. They were first divided into two groups of approximately equivalent reading proficiencies and then completed MCQ and SAQ English reading tests in different orders. After completing each format, participants filled out a survey about demographic information, strategy use, and perceptions of test formats. They also completed a 5-point Likert-scale survey to assess their degree of metacognitive awareness. At the end, sixteen participants were randomly chosen to engage in retrospective interviews focusing on their strategy use and task perceptions. This study employed a mixed methods approach in which quantitative and qualitative strands converged to draw an overall meta-inference. For the quantitative strand, descriptive statistics, paired sample t-tests, item analyses, two-way ANOVAs, and correlation analyses were conducted to investigate 1) the differences between MCQ and SAQ test performance and 2) the relationship between test performance and metacognitive awareness. For the qualitative strand, test-takers’ MCQ and SAQ test completion processes and task perceptions were explored using coded interview and survey responses related to strategy use and perceptions of test formats. Results showed that participants performed differently on MCQ and SAQ reading tests, even though both tests were highly correlated. The paired sample t-tests revealed that participants’ English reading and writing proficiencies might account for the MCQ and SAQ performance disparity. Moreover, there was no positive relationship between reading test performance and the degree of metacognitive awareness generated by the frequency of strategy use. Correlation analyses suggested whether a higher or lower English reading proficiency of the participants was more important than strategy use. Although the frequency of strategy use did not benefit test performance, strategies implemented for MCQ and SAQ tests were found to generate interactive processes allowing participants to gain deeper understanding of the source texts. Furthermore, participants’ perceptions toward MCQs, SAQs, and a combination of both revealed positive and negative influences among test format, reading comprehension, and language learning. Therefore, participants’ preferences of test format should be considered when measuring their English reading proficiency. This study has pedagogical implications on the use of various test formats in L2 reading classrooms.