In this modern era many educational institutes and business organisations are adopting the e-Learning approach as it provides an effective method for educating and testing their students and staff. The continuous development in the area of information technology and increasing use of the internet has resulted in a huge global market and rapid growth for e-Learning. Multiple Choice Tests (MCTs) are a popular form of assessment and are quite frequently used by many e-Learning applications as they are well adapted to assessing factual, conceptual and procedural information. In this thesis, we present an alternative to the lengthy and time-consuming activity of developing MCTs by proposing a Natural Language Processing (NLP) based approach that relies on semantic relations extracted using Information Extraction to automatically generate MCTs. Information Extraction (IE) is an NLP field used to recognise the most important entities present in a text, and the relations between those concepts, regardless of their surface realisations. In IE, text is processed at a semantic level that allows the partial representation of the meaning of a sentence to be produced. IE has two major subtasks: Named Entity Recognition (NER) and Relation Extraction (RE). In this work, we present two unsupervised RE approaches (surface-based and dependency-based). The aim of both approaches is to identify the most important semantic relations in a document without assigning explicit labels to them in order to ensure broad coverage, unrestricted to predefined types of relations. In the surface-based approach, we examined different surface pattern types, each implementing different assumptions about the linguistic expression of semantic relations between named entities while in the dependency-based approach we explored how dependency relations based on dependency trees can be helpful in extracting relations between named entities. Our findings indicate that the presented approaches are capable of achieving high precision rates. Our experiments make use of traditional, manually compiled corpora along with similar corpora automatically collected from the Web. We found that an automatically collected web corpus is still unable to ensure the same level of topic relevance as attained in manually compiled traditional corpora. Comparison between the surface-based and the dependency-based approaches revealed that the dependency-based approach performs better. Our research enabled us to automatically generate questions regarding the important concepts present in a domain by relying on unsupervised relation extraction approaches as extracted semantic relations allow us to identify key information in a sentence. The extracted patterns (semantic relations) are then automatically transformed into questions. In the surface-based approach, questions are automatically generated from sentences matched by the extracted surface-based semantic pattern which relies on a certain set of rules. Conversely, in the dependency-based approach questions are automatically generated by traversing the dependency tree of extracted sentence matched by the dependency-based semantic patterns. The MCQ systems produced from these surface-based and dependency-based semantic patterns were extrinsically evaluated by two domain experts in terms of questions and distractors readability, usefulness of semantic relations, relevance, acceptability of questions and distractors and overall MCQ usability. The evaluation results revealed that the MCQ system based on dependency-based semantic relations performed better than the surface-based one. A major outcome of this work is an integrated system for MCQ generation that has been evaluated by potential end users.

Traditional undergraduate biology courses are content intensive, requiring students to understand and remember large amounts of information in short periods of time. Yet most students maintain little of the material encountered during their education. Poor knowledge retention is a main cause of academic failure and high undergraduate attrition rates. Characterizing strategies that support robust learning is critical for ensuring student success. One such strategy is testing effect, the observation that repeated testing can improve the fidelity and durability of retained knowledge more than an equal quantity of restudy. Numerous investigations have described the nature and boundaries of testing effect. Very few, however, have characterized its efficacy in actual classroom practice. The current study investigated whether repeated testing or repeated study affected student retention and understanding of complex biological concepts. The study was conducted in a large (~320 students) introductory biology class. All study conditions and assessments were required components of the course. Student retention of two fundamental molecular biology "big ideas" was targeted; (1) the relationship between genotype and phenotype, and (2) the relationship between gene expression and cell function. Students were randomly assigned to one of three repeated quiz or study conditions. For four weeks, students encountered various combinations of multiple-choice (MC) questions and review material related to big ideas 1&2 and/or unrelated lecture topics. Five weeks after the last quiz, all students completed identical MC final exam questions related to both big ideas. To determine the quality of "understanding" assessed by the MC questions, a subset of students also completed a short answer (SA) test prior to the final exam. Both question formats assessed the same knowledge (2 big ideas) at the same level (comprehension and application). Final exam performance supported the finding that repeated retrieval improves long-term retention of knowledge relative to repeated study. Novel to other previous work conducted at the undergraduate level, the current findings suggest that repeated testing affects student retention and understanding of sophisticated concepts. Careful design and analysis of parallel multiple-choice and short answer questions demonstrated that each can target and elicit similar qualities and types of knowledge.

Multiple-choice questions are acknowledged to be difficult for both English mother-tongue and second-language university students to interpret and answer. In a context in which university tuition policies are demanding explicitly that assessments need to be designed and administered in such a way that no students are disadvantaged by the assessment process, the thesis explores the fairness of multiple-choice questions as a way of testing second-language students in South Africa. It explores the extent to which two multiple-choice Linguistics examinations at Unisa are in fact ‘generally accessible’ to second-language students, focusing on what kinds of multiple-choice questions present particular problems for second-language speakers and what contribution linguistic factors make to these difficulties. Statistical analysis of the examination results of two classes of students writing multiple-choice exams in first-year Linguistics is coupled with a linguistic analysis of the examination papers to establish the readability level of each question and whether the questions adhered to eight item-writing guidelines relating to maximising readability and avoiding negatives, long items, incomplete sentence stems, similar answer choices, grammatically non-parallel answer choices, ‘All-of-the-above’ and ‘None-of-the-above’ items. Correlations are sought between question difficulty and aspects of the language of these questions and an attempt is made to investigate the respective contributions of cognitive difficulty and linguistic difficulty on student performance. To complement the quantitative portion of the study, a think-aloud protocol was conducted with 13 students in an attempt to gain insight into the problems experienced by individual students in reading, understanding and answering multiple-choice questions. The consolidated quantitative and qualitative findings indicate that among the linguistic aspects of questions that contributed to question difficulty for second language speakers was a high density of academic words, long items and negative stems. These sources of difficulty should be addressed as far as possible during item-writing and editorial review of questions.

It is hypothesized that useful diagnostic information can reside in the wrong answers of multiple-choice tests, and that properly designed distractors can yield indications of misinformation and missing information in algorithmic reasoning on the part of the test taker. In addition to summarizing the literature regarding diagnostic research as opposed to scoring research, this study proposes a methodology for analyzing test results and compares the findings with those from the research of Birenbaum and Tatsuoka and others. The proposed method identifies the conditions of misinformation and missing information, and it contains a statistical compensation for careless errors. Strengths and weaknesses of the method are explored, and suggestions for further research are offered.

Electronic cigarettes have grown in popularity across the U.S. and concerns have been raised about their abuse liability. The purpose of the current study was to evaluate and compare the abuse liability of an electronic cigarette with and without nicotine to a nicotine inhaler (the Nicotrol inhaler) and participants’ own brand of cigarettes. A total of 24 smokers attended four sessions in which the abuse liability of each product was examined using the Multiple-Choice Procedure (MCP), nicotine delivery, nicotine abstinence symptom suppression, and subjective reinforcing effects. Results revealed that the nicotine containing and non-nicotine containing electronic cigarette had a higher reinforcing efficacy on the MCP than the nicotine inhaler, but on average had a lower reinforcing efficacy than participants own brand of cigarettes. The nicotine containing electronic cigarette delivered nicotine to participants in amounts that did not differ significantly from participants’ own brand of cigarettes. The electronic cigarette with nicotine reduced nicotine abstinence symptoms to a greater degree than the electronic cigarette without nicotine, and both electronic cigarettes were rated as subjectively more reinforcing than the inhaler but less reinforcing than participants’ own brand of cigarettes. In sum, the results from this study suggest that the electronic cigarette examined had a moderate level of abuse liability that was higher than an FDA-approved nicotine inhaler but lower than traditional cigarettes. Furthermore, findings also suggest that electronic cigarette abuse liability may extend beyond factors related to nicotine delivery.

The main purpose of this study was to compare the strategies used by Chinese- speaking students when confronted with familiar versus unfamiliar topics in a multiple-choice format reading comprehension test. The focus was on describing what students do when they are taking reading comprehension tests by asking students to verbalize their thoughts. The strategies were further compared with participants' level of familiarity with different reading topics and their reading scores. Twenty Chinese-speaking participants at the University of Iowa performed three tasks: a topical knowledge vocabulary assessment that served as an indicator of each participant's topical knowledge about the four selected content areas in this study (law, business, language teaching, and engineering); two Test of English as a Foreign Language (TOEFL) internet-based test (iBT) practice reading comprehension passages, one with a familiar topic and the other with an unfamiliar topic, and both with retrospective think-aloud protocols; and an interview related to participants' test-taking strategies. Two stages of analysis, qualitative and quantitative, were undertaken in this study. For the qualitative analysis, all verbal reports provided by participants in the think-aloud protocols and the interviews were recorded and transcribed. Six categories of strategies emerged: general approaches to reading the passages, identification of important information by the discourse structure of the passages, vocabulary/sentence-in-context approaches, multiple-choice test-management strategies, test-wiseness, and background knowledge. For the quantitative analysis, an analysis of variance (ANOVA) with repeated measures was completed to determine if there were significant differences based on the frequency of strategy use and level of topic familiarity. The results showed that the types of test-taking strategies adopted by Chinese-speaking graduate students remained similar when they read passages with familiar versus unfamiliar topics. However, participants all reported feeling more relief and more confidence when reading passages related to their background knowledge. The second ANOVA employed a split-plot statistical design to examine whether there were significant differences based on participants' strategy use and their reading scores as measured by the iBT reading comprehension tests. High scorers employed strategies in categories one, two, three, and four significantly more frequently than low scorers. However, low scorers adopted significantly more strategies in category five than high scorers. In category six, high and low scorers seemed to use a similar number of strategies. Findings that emerged from the two perspectives are discussed; implications related to test-taking and reading pedagogy are provided in the conclusion.

It is the standard practice in colleges and universities to administer to all entering students a battery of tests which are designed to measure the aptitude and ability of students. Each institution uses in its test battery those tests which are best suited to its particular needs, but in the main they consist of measures of general mental ability, the ability to understand and use the English language and the ability to understand and to use mathematical symbols. These tests are available to the institution through companies which construct and standardize tests of all kinds. In addition to these tests, others may be constructed by the departments of an institution and validated for use in that department or school. The test battery or the Utah State Agricultural College consists or a combination of these two types. The English examination is the Educational Testing Service Cooperative English test Battery, the Mathematics test is a department constructed test, originally designed and constructed by Dr. Arden N. Frandsen.

Instructional design and delivery may be one tool available to teachers to increase the academic and social behaviors of all students in the classroom. Effective instruction is an evidence-based teaching strategy that can be used to efficiently educate our youth across all learning environments. One effective instructional strategy includes increasing students’ opportunities to respond to instructor-posed questions during lectures. Students may respond to questions using a response card system as a way to promote active engagement. This study examined the most common form of instructor-posed questions presented during lecture, recall and recognition questions, to determine the differential effects on students’ academic and participation behavior in a college classroom. Results found no differentiation in students’ academic behavior with respect to question type. Students’ participation behavior was greater when the instructor used class wide active responding procedures than observed in baseline conditions that represented typical college instruction.

Multiple choice testing has been introduced as an assessment instrument in almost all educational systems during the past twenty years. A growing body of research seems to indicate that tests structured to a multiple choice format favour males. In the ACT, Queensland and Western Australia, a multiple choice examination known as ASAT was used to moderate student scores. Using data from the 1989 ASAT Paper 1, as well as data from the ACT Year 12 cohort of that year, an investigation was made of the items in the ASAT paper. This investigation attempted to identify specific types of questions that enabled males, on average, to perform better than females. Questions, which had a statistically significant difference between the results of males and females, were examined further. An ASAT unit was given to students to complete and their answers to a questionnaire concerning the unit were taped and analysed. The study found that males performed better, on average, than females on the 1989 ASAT Paper 1. The mean difference in the quantitative questions was much greater than in the verbal questions. A number of factors appear to contribute to the difference in performance between males and females. A statistically significant number of females study Mathematics at a lower level, which appears to contribute to females lower quantatitive scores. Females seem to be considerably more anxious about taking tests and this anxiety remains throughout a multiple choice test. Females lack confidence in their ability to achieve in tests and are tentative about "risktaking" which is an element of multiple choice tests. The language of the test and male oriented content may contribute to females' negative performance in multiple choice testing.

The objectives of this study were to construct and develop a two-tiered multiple-choice test for the measurement of student misconceptions in optics at the level of secondary education in Thailand, as well as to investigate the effects of gender, grade level, and type of school upon levels of student misconceptions. Misconceptions were measured by the Two-Tiered Optics Misconception (TTOM) test instrument developed for this study. Based upon criteria-evidence, the validity or appropriateness, meaningfulness, and usefulness of inferences derived from scores obtained from administration of the instrument was demonstrated at acceptable levels. The TTOM was administered to 932 high school subjects from randomly selected all-male, all-female, and coeducational 10th through 12th grade classes in Bangkok, Thailand. A mixed analysis of variance model was used to investigate the effects of gender, grade level, and type of school upon subject misconception scores in the field of optics. Analysis of the data revealed the following: 1) a paper-pencil test based upon a pattern of two-tiered multiple-choice questions and reasons developed for this study provided a valid and reliable measure of student misconceptions in optics at the level of secondary education in Thailand; 2) gender, grade level, and type of school did not have significant effects upon subject misconception scores among Thai high school students; 3) the subgroup school within type was found to have a significant effect upon subject misconception scores; 4) when fixed grade levels were considered for type of school, there were significant differences in misconceptions among 12th grade subjects from different types of schools; 5) certain consistent misconceptions in the area of optics, related to the properties of a converging lens, images from a plane mirror and a converging lens, light rays, regions of light travel, shadow formation, and lightning flashes, existed among Thai physics students at the level of secondary education. / Graduation date: 1994