Academics encounter problems with the selection, evaluation, testing and implementation of e-assessment software tools. The researcher experienced these problems while adopting e-assessment at the university where she is employed. Hence she undertook this study, which is situated in schools and departments in Computing-related disciplines, namely Computer Science, Information Systems and Information Technology at South African Higher Education Institutions. The literature suggests that further research is required in this domain. Furthermore, preliminary empirical studies indicated similar disabling factors at other South African tertiary institutions, which were barriers to long-term implementation of e-assessment. Despite this, academics who are adopters of e-assessment indicate satisfaction, particularly when conducting assessments with large classes. Questions of the multiple choice genre can be assessed automatically, leading to increased productivity and more frequent assessments. The purpose of this research is to develop an evaluation framework to assist academics in determining which e-assessment tool to adopt, enabling them to make more informed decisions. Such a framework would also support evaluation of existing e-assessment systems. The underlying research design is action research, which supported an iterative series of studies for developing, evaluating, applying, refining, and validating the SEAT (Selecting and Evaluating an e-Assessment Tool) Evaluation Framework and subsequently an interactive electronic version, e-SEAT. Phase 1 of the action research comprised Studies 1 to 3, which established the nature, context and extent of adoption of e-assessment. This set the foundation for development of SEAT in Phase 2. During Studies 4 to 6 in Phase 2, a rigorous sequence of evaluation and application facilitated the transition from the manual SEAT Framework to the electronic evaluation instrument, e-SEAT, and its further evolution. This research resulted in both a theoretical contribution (SEAT) and a practical contribution (e-SEAT). The findings of the action research contributed, along with the literature, to the categories and criteria in the framework, which in turn, contributed to the bodies of knowledge on MCQs and e-assessment. The final e-SEAT version, the ultimate product of this action research, is presented in Appendix J1. For easier reference, the Appendices are included on a CD, attached to the back cover of this Thesis.. / Computing / PhD. (Information Systems)

政治知識之於民主社會有其重要性，在政治學界中與政治知識相關的研究產出相當豐富，研究者利用政治知識此一變數進行相關研究前，對於題目如何選定、選項如何提供、題型的差異都是研究者需要去關注的重點，而本文主要的研究目的即是聚焦於討論何種題型更適合用來測量民眾的政治知識。 目前國內測量政治知識的問卷題型較為常見的為開放題與選擇題題型，在這兩類題型的討論上，前者被認為會低估受訪者政治知識程度，後者的測量結果則被質疑提供猜題空間導致高估了受訪者的政治知識程度，然而目前國內外卻缺乏足夠的實證研究來證明這兩個題型的適用性。 本文採用具有實驗設計性質的二手資料，利用前後測的方式讓受測者填答相同題目不同題型的問卷，藉此檢視各種知識程度的受測者在面對不同題型時是否會產生回應模式上的差異。本研究發現，開放題會使得較高政治知識程度的受訪者被低估，選擇題反而能準確測量出此類受訪者的知識程度。為了進一步證實受訪者在偏難的題目上所增加的猜題比例並非是來自於盲猜，本文採用多項機率單元模型來檢視受訪者於於選擇題選擇各個答項的機率。研究發現，儘管選擇題無法避免受訪者猜題，但受訪者並非是盲猜，反而會根據其具有的知識依據來答題，故政治知識程度高的受訪者能採用猜題方式答對題目，政治知識低的受訪者無法利用猜題方式猜中答案。整體而言，選擇題比起開放題更適合用來測量民眾的政治知識。 / Political knowledge plays an important role in the democratic society, and therefore there has been much research on political knowledge in the discipline of political science. To study political knowledge, political scientists have to understand the way of questions and options presented, and also the differences between a variety of question formats. This paper aims to analyze which question format is better for measuring the political knowledge of the public. The open-ended and multiple-choice items are both common formats for measuring political knowledge in Taiwan. The open-ended question is always considered to underestimate the respondents’ level of political knowledge, while the multiple-choice format is thought of overestimating the levels of political knowledge for providing the respondents with opportunity to guess. However, a strong evidence to decide the most suitable format for the measurement of political knowledge is still lacking. This paper uses the secondary data which is collected by a pretest-posttest questionnaire to examine whether guessing behavior will emerge or not when the respondents facing the same question with different formats. This research finds that open-ended questions underestimate the respondents’ knowledge levels who has higher level of political knowledge originally, but the multiple-choice questions can estimate the levels more accurately. To further confirm that the higher guess proportions in the more difficult questions are not resulted from the blind guessing, the study examines the probabilities of options selected by Multinomial Probit Model. The research finds that though the respondents may have guess more in multiple-choice question, however, they tend to answer the questions based on their knowledge instead of blind guessing. Therefore, the respondents who have higher levels of political knowledge can guess correctly, while those who have lower levels of political knowledge cannot. In summary, the multiple-choice questions are more suitable to measure people’s political knowledge.

Submitted by admin tede (tede@pucgoias.edu.br) on 2016-09-12T14:39:55Z No. of bitstreams: 1 Paulo Henrique de Freitas Miranda.pdf: 2959134 bytes, checksum: 19e3a755635c112cbe515610ba863dda (MD5) / Made available in DSpace on 2016-09-12T14:39:55Z (GMT). No. of bitstreams: 1 Paulo Henrique de Freitas Miranda.pdf: 2959134 bytes, checksum: 19e3a755635c112cbe515610ba863dda (MD5) Previous issue date: 2015-09-10 / The evaluation evolves constantly in search of better educational practices through the development of education. In this perspective, this study aims to contribute by analyzing reviews of multiple choice and open questions, submitted on paper and computer. Four tests in four different models (multiple choices on your computer, multiple choices on paper, open questions on the computer and open questions in the paper) were performed. The focus of the research was analysis of the notes in each model, the duration of the test and student satisfaction at the end of the tests. The results were presented and discussed considering a context of controlled assessment, interpreting the conditions under which the models can be equivalent, advantages and disadvantages. A test of multiple-choice paper presented better results in terms od test notes. / A avaliação evolui constantemente em busca de melhores práticas educacionais mediante o desenvolvimento da educação. Nesta perspectiva, este estudo propõese contribuir através da análise de avaliações de múltipla escolha e questões abertas, apresentadas em papel e computador. Foram realizados quatro testes em quatro modelos diferentes (múltipla escolha no computador, múltipla escolha no papel, questões abertas no computador e questões abertas no papel). O foco da investigação foram análises das notas em cada modelo, do tempo de duração do teste e da satisfação do aluno ao término dos testes. Os resultados foram apresentados e discutidos considerando o contexto do teste controlado, interpretando as condições sob os quais os modelos podem apresentar equivalência, vantagens ou desvantagens. O teste de múltipla escolha em papel apresentou melhores resultados em termos de nota no teste.

Made available in DSpace on 2016-04-29T14:23:36Z (GMT). No. of bitstreams: 1 Antonio Carlos Bento.pdf: 19317474 bytes, checksum: 3345ae7a48f0769e446ab9888361725d (MD5) Previous issue date: 2015-06-18 / The research focuses on the performance evaluation methodology within the models of multiple-choice questions from methodeutics criteria of Fuzzy Logic proposed by Lotfi Zadeh associated with statistical reasoning presented by Santos, Pigari and Tolentino present in the theories of uncertainty discussed by Russell and Norvig and item response theory presented by Fontanive. The study was based on an exploratory and experimental research used questionnaires that were submitted to academic groups. The collected data were compared with a bibliography on the field evaluation, education and logic, resulting in the organization of a methodology that enabled the construction of a prototype computer system to validate the realized and expected outcomes studies. Following the classification s heuristics Clancey it concludes by identifying a set of rules that have been identified as strong enough to guide work and software modeling, aimed at building models of multiple-choice questions based Fuzzy Logic rules. Finally, the results were gathered and presented by means of a system prototype able to characterize the main established rules, as well as studies, being able to be fully utilized in the implementation of performance assessments that may present evidence with expected results, supporting identifying random responses during evaluations / A pesquisa enfoca a metodologia da avaliação de desempenho, dentro dos modelos de questões do tipo múltipla escolha, a partir de critérios metodêuticos da Lógica Fuzzy (Lógica Difusa ou nebulosa) proposta por Zadeh Lotfi associados ao raciocínio estatístico apresentado por Santos, Pigari e Tolentino presente nas teorias da incerteza discutido por Russell e Norvig e teoria da resposta ao item apresentado por Fontanive. O estudo partiu de uma pesquisa exploratória e experimental que utilizou questionários que foram submetidos a grupos de acadêmicos. Os dados colhidos foram cotejados com uma bibliografia sobre o campo de avaliação-educação e lógica, resultando na organização de uma metodologia que possibilitou a construção de um sistema protótipo computacional para a validação dos estudos realizados e resultados esperados. Seguindo a Classificação Heurística de Clancey a pesquisa conclui pela identificação de um conjunto de regras que foram identificadas como suficientemente fortes para nortear trabalhos e de modelagem de softwares, visando a construção de modelos de questões do tipo múltipla escolha baseado em regras de Lógica Fuzzy. Finalmente, os resultados foram reunidos e apresentados por meio de um sistema protótipo capaz de caracterizar as principais regras estabelecidas, bem como os estudos realizados, sendo capaz de serem utilizados plenamente na aplicação de avaliações de desempenho que possam apresentar evidências com resultados esperados, oferecendo suporte a identificação de respostas ao acaso durante avaliações

Nursing programs have traditionally used teacher-developed multiple-choice (MCQ) examinations to prepare students for licensure. Researchers have determined that poorly constructed MCQ tests used as formative and summative evaluations may penalize nursing students and impact progression and retention in nursing programs. The purpose of this exploratory case study was to examine issues related to the use of teacher-developed MCQ examinations as the only method of student assessment in the theory component of nursing courses. The National League for Nursing Core Competencies for Nurse Educators and the revised Bloom's Taxonomy were used as the conceptual frameworks for this study. The Director of the Nursing Program and 9 faculty members participated. Data were collected from a review of documents, 2 focus groups, faculty-maintained diaries, and an interview. During data analysis, categories were identified and themes emerged, revealing the key findings. Using a single method alone to assess student learning limited the opportunity for formative assessment, the ability to assess higher order thinking, and the development of metacognition on the part of students. To assist faculty in creating assessments of student learning that would address these themes, a 3-day faculty professional development project followed by 4 monthly lunch and learn sessions was designed. Providing additional faculty development in assessment methods may promote positive social change as it may ultimately increase the retention of qualified students to meet the demand for registered nurses within the community.

The essay analyses the discursive construction of national identity in the present naturalisation test of the Federal Republic of Germany. The essay includes an overview of immigration to Germany, and a survey of political measures to improve the integration of immigrants. The language and structure of representative multiple-choice questions (and answers) of the naturalisation test are analysed by using the method of critical discourse analysis (CDA). The theoretical background of this study is grounded in cultural studies. The methodological framework consists of a combination of critical discourse analysis and textual analysis. Criteria of the linguistic analysis are: the situational context of the text, thematic roles, deixis, lexical repetitions, modality, coherence (including implicit meanings and presuppositions), intertextuality and interdiscursivity, competence and performance. The analysis shows that national identity is conceptualized by the multiple-choice questions of the naturalisation test. National identity is above all constructed by the German language. One of the qualifications which the examinee has to fulfil is competence in German on the level B1 (of the Common European Framework of Reference for Languages). Linguistic competence is necessary in order to answer the questions. National identity is linguistically created by using alterity. Binary oppositions are constructed by stipulations and presumptions about migrants living in Germany. These oppositions are created by giving three alternative answers, which represent prejudices about foreigners; we is represented by an idealized construction of Germans, and the other is represented by stereotypical assumptions about foreigners. National identity is created by the content of the questions, too. Many questions deal with German laws and standards, which implies that being a German means to be law-abiding. The present study shows that German identity is constructed by language and the construction of alterity.

No description available.

<p>The test context is of paramount importance in language testing as it provides an understanding of the kind of tasks to be included in the test, how these tasks are executed by the test takers and how they can be efficiently administered. The objective of this study was to investigate the extent to which the context of the DR Congo English state examination (ESE) is valid and to come out with some useful suggestions that are likely to improve its validity. Two basic theories, the modern validity theory and the schema theory, informed this study. Weir‟s (2005) socio-cognitive framework was used to build the validity argument for the evaluation of the English state examination.</p> <p>A mixed method was used where the research design consisted of the combination of both qualitative and quantitative data during the collection and analysis stages. The content document analysis method was used to examine the content of the different state examination papers so as to identify the main features of the test, and the statistic (descriptive) method was used to quantify observations identified in the state examination papers and to evaluate the context validity of the ESE. Three techniques were used to collect the research data: the questionnaire, the<br /> test, and the interview.</p> <p>Three main findings of this study were reported: (1) the conditions under which the ESE tasks are performed and the relevance of these tasks to the test domain and characteristics are still far to contribute to the quality of evaluation of high school finalist students / (2) the extent to which the ESE includes tasks that take into consideration the nature of information in the text as well as the knowledge required for completing the task is globally good / (3) the conditions under which the test takes place are poor and these conditions affect the validity of test scores. The study recommends the test developers to approximate test tasks to those students have been exposed to in classroom situations and those they are likely to encounter in real life. It also recommends all the people involved in the administration of the test to adhere to high ethical standards.</p>

<p>The test context is of paramount importance in language testing as it provides an understanding of the kind of tasks to be included in the test, how these tasks are executed by the test takers and how they can be efficiently administered. The objective of this study was to investigate the extent to which the context of the DR Congo English state examination (ESE) is valid and to come out with some useful suggestions that are likely to improve its validity. Two basic theories, the modern validity theory and the schema theory, informed this study. Weir‟s (2005) socio-cognitive framework was used to build the validity argument for the evaluation of the English state examination.</p> <p>A mixed method was used where the research design consisted of the combination of both qualitative and quantitative data during the collection and analysis stages. The content document analysis method was used to examine the content of the different state examination papers so as to identify the main features of the test, and the statistic (descriptive) method was used to quantify observations identified in the state examination papers and to evaluate the context validity of the ESE. Three techniques were used to collect the research data: the questionnaire, the<br /> test, and the interview.</p> <p>Three main findings of this study were reported: (1) the conditions under which the ESE tasks are performed and the relevance of these tasks to the test domain and characteristics are still far to contribute to the quality of evaluation of high school finalist students / (2) the extent to which the ESE includes tasks that take into consideration the nature of information in the text as well as the knowledge required for completing the task is globally good / (3) the conditions under which the test takes place are poor and these conditions affect the validity of test scores. The study recommends the test developers to approximate test tasks to those students have been exposed to in classroom situations and those they are likely to encounter in real life. It also recommends all the people involved in the administration of the test to adhere to high ethical standards.</p>

Written exams, which are intended to examine students on the course goals, can sometimes be passed without any conceptual knowledge by memorizing procedures or facts. Because future courses depend on the students’ understanding of concepts in the required courses, not knowing the concepts could be a major issue for the student, for the teacher and for the program board. Here, we focus on developing conceptual multiple-choice questions and the algorithms for understanding the answers to the questions. The goal is to be able to answer questions such as “For how long do the students remember the key concepts?” and “Which concepts do the courses have any positive (or negative) effect on?”. To do that, a courses-concepts matrix was created and the most central concepts were identified. Multiple-choice questions were written on those concepts and the questions were imported into the test creator Respondus. Feedback was added to the questions, they were grouped by concept and exported to a quiz bank in the educational platform Blackboard. A set of answers from a survey on report writing was obtained and statistics were written to answer the second question that was posed. An issue with the probability function is that it only considers whether the student had a setback or an improvement, not how significant it was. The next step would be to use the slopes more effectively by considering the magnitude of the improvement or setback. / Developing Concept Inventory Tests for Electrical Engineering