This Research and Design (R&D) study models the methodology necessary to replicate an online assessment instrument designed to assess student skills and facets of thought while understanding Multiple Meanings and Models of Fractions (MUL) in college level developmental mathematics. The researcher used cognitive research done in the area of fractions to design this instrument that both documents and assesses facets of thought or reasoning strategies used by students. The final facet cluster is a table that ranks these facets from least to most problematic, documenting the student facets of thought across the content objective MUL. Over 500 student and teacher participants were used in the design and development of Fraction Diagnoser. All participants were affiliated with college developmental mathematics in Texas, representing four colleges and universities. Forty-eight student participants were individually interviewed to ascertain facets of understanding on the topic of MUL. Seven teacher participants were individually interviewed as to the effectiveness of Fraction Diagnoser in the classroom after the final step of the R&D cycle. Content experts were used to design the questions assessing skills and facets. Fraction Diagnoser was built using the Borg and Gall R&D cycle as its blueprint. Nine of the ten steps of the R&D cycle were used in the development of the instrument, excluding just the final product revision due to cost and time restraints. According to Borg and Gall (1996), a dissertation R&D should be limited to a few steps, but all of the steps used for this R&D allowed for the researcher to completely address all of the research questions. During the steps of the R&D cycle, validation and reliability analyses were done to statistically address the effectiveness of Fraction Diagnoser. Final interviews with the teacher participants supported findings in recent research on the effective use of online assessment. Implications for practice and recommendations for further study were also addressed.

Peer review is used as an effective quality assurance measure in many contexts, including science, business, programming or education. In education, several studies confirmed the positive effects of peer reviewing on student learning. Based on recent research concerning the role of media in the peer review process this study investigates how students perceive the process, content and effects of peer reviews. We also analyze students' opinions on different modes of peer reviewing activities, e.g. online vs. face-to-face reviewing. In the context of a computer science course on scientific writing, these research questions were addressed by administering an online questionnaire (n=38) and analysis using quantitative and qualitative methods. Results indicate that students value the peer review activity, take peer reviews seriously and provide comprehensive and constructive reviews. Findings also show that students prefer written online reviews with the possibility of oral follow-up questions to reviewers.

The purpose of this study was to describe the frequencies and types of cheating behaviors occurring within proctored and unproctored testing environments for students enrolled in online courses and taking assessments through an online format. In addition, this study sought to examine relationships between demographic variables of gender, age, GPA, discipline of study, undergraduate/graduate status and knowledge of the institutional honor code and online cheating behaviors for students who had taken online assessments through proctored methods. Participants in this study included students enrolled as distance learning campus students who took online courses and online assessments through a large, 4-year, public, degree-granting institution located in the southeastern region of the United States during the spring 2015 semester. Participants were asked to report their frequency in engaging in online cheating behaviors through the Online Assessment Cheating Behaviors Survey (OACBS). The study found that distance students who took unproctored exams reported more frequently engaging in overall cheating behaviors than proctored students. No differences were found in overall cheating behaviors for those taking exams through face to face and remotely proctored methods. Individual item analyses revealed that those taking unproctored exams reported more frequently using web searches during online exams to search for answers than those taking proctored exams. The study also found differences in overall cheating behaviors for those taking proctored online exams based on gender, with female distance students more frequently reporting engaging in cheating behaviors than male distance students. Individual level item analyses revealed females, those with a “C” GPA, and undergraduate distance students more frequently utilized web searches during an online proctored exam and used brain dump sites to obtain test questions and answers.

The goal of the study was to determine whether the Occupational Personality Questionnaire (OPQ32i) yielded comparable results when two different modes of administration, namely paperand- pencil and Internet-based administration, were used in real-life, high-stakes selection settings. Two studies were conducted in which scores obtained online in unproctored settings were compared with scores obtained during proctored paper-and-pencil settings. The psychometric properties of the paper-and-pencil and Internet-based applications were strikingly similar. Structural equation modelling with EQS indicated substantial support for the hypothesis that covariance matrices of the paper-and-pencil and online applications in both studies were identical. It was concluded that relationships between the OPQ32i scales were not affected by mode of administration or supervision.

The goal of the study was to determine whether the Occupational Personality Questionnaire (OPQ32i) yielded comparable results when two different modes of administration, namely paperand- pencil and Internet-based administration, were used in real-life, high-stakes selection settings. Two studies were conducted in which scores obtained online in unproctored settings were compared with scores obtained during proctored paper-and-pencil settings. The psychometric properties of the paper-and-pencil and Internet-based applications were strikingly similar. Structural equation modelling with EQS indicated substantial support for the hypothesis that covariance matrices of the paper-and-pencil and online applications in both studies were identical. It was concluded that relationships between the OPQ32i scales were not affected by mode of administration or supervision.

This study examined the potential use of keystroke dynamics to create keyprints (typing fingerprints) to authenticate individuals in online assessment situations. The implications of this study are best understood in terms of the keystroke behavioral biometric. While previous studies considered the degree to which keystroke typing patterns are unique, this study was set up to determine how well keyprints are able to identify individuals when typing under various treatment conditions (copy typing, free typing, and typing with mild or moderate impediments). While authentication can be difficult when attempting to correctly identify individual users, the results of this study indicate that keyprints can be a solid indicator of negative cases (i.e., flagging situations where a typing sample is likely not the correct individual). As anticipated, typing with a temporary impediment does diminish the algorithms' ability to identify students. This is also the case when user samples are typed under conditions different from those in which the keyprint baseline signature was captured (i.e., copy versus free typing). The ability to identify individuals is also challenging when using small comparison samples. However, the ability of the system to identify negative cases functions fairly well in each instance.

This study investigated the examinees&rsquo / perceptions about an online assessment tool and an online assessment center. In this study, a computer literacy exam (Bilisim Seviye Tespit Sinavi, BSTS) was delivered in purposively designed environment, an online assessment center, at CEC, METU (Continuing Education Center, Middle East Technical University). The data were collected from 117 participants through the &ldquo / User evaluation questionnaire&rdquo / , in depth interviews with 43 examinees and 7 experts. 7 participants and the experts were interviewed individually while the rest 36 participants were interviewed in 6 focus groups (composed of 6 examinees). Descriptive statistics, frequency distributions and comments of participants and experts were used to express the results of the study. The results showed that the examinees perceived the online assessment tool and the online assessment center suitable for delivering online assessments. However, both participants and experts reported that a few changes could be done to the interface of online assessment tool. They also suggested that the computer literacy assessments be more authentic.

Diagrams are increasingly used in many design methods, and are being taught in a variety of contexts in higher education such as database conceptual design or software design in computer science. They are an important part of many assessments. Currently computer aided assessments are widely used for multiple choice questions. They lack the ability to assess a student's knowledge in a more comprehensive way, which is required for diagram-type student work. The aim of this research is to develop a semi-automatic assessment framework, which enables the use of computer to support the assessment process of diagrammatic solutions, with the focus of ensuring the consistency of grades and feedback on solutions. A novel trace model, that captures design traces of student solutions, was developed as a part of the framework and was used to provide the matching criteria for grouping the solutions. A new marking style, partial marking, was developed to mark these solution groups manually. The Case-Based Reasoning method is utilised in the framework to mark some of the groups automatically. A guideline for scenario writing was proposed to increase the efficiency of automatic marking. A prototype diagram editor, a marking tool and scenario writing environment were implemented for the proposed framework in order to demonstrate proof of concept. The results of experiments show that the framework is feasible to use in the formative assessment and it provides consistent marking and personalised feedback to the students. The framework also has the potential to significantly reduce the time and effort required by the examiner to mark student diagrams. Although the constructed framework was specifically used for the assessment of database diagrams, the framework is generic enough to be used for other types of graph-based diagram.

The professional practice of educational design normally involves collaborating with a subject matter expert on a one-to-one basis and is only occasionally undertaken in teams. This thesis argues that a team-based approach to educational design is powerful and particularly so when transdisciplinary collaborations are facilitated. Transdisciplinary educational design is the process of standing outside one’s discipline to collaborate with colleagues from the technical sphere, the library and other disciplines. The common ground shared by the transdisciplinary teams in this research was student assessment.The core data collection for this research was completed between July 2002 and June 2005. Using an overarching action research methodology, three cycles of data collection were completed by action learning sets. Suitable members of the sets were identified through a series of online staff development workshops that were designed and facilitated by the researcher. Two supplementary data collection activities were also undertaken. The first of these was a Web survey that broadly mapped design practices for online assessment in four Australian regional universities. Three rounds of telephone interviews then followed up on survey responses. The second supplementary data collection was undertaken between the second and third action learning cycles to contextualise the online assessment design activities at Southern Cross University within the broader framework provided by the other three regional universities in the original sample. It included focus groups with educational designers and face-to-face interviews with three academics at each of these universities. The entire series of data collection activities was reflectively managed to heighten its effectiveness. This management included screening of suitable participants, negotiation of manageable session times and duration, and establishment of ground rules for attendance and interactions, as well as drawing out a commitment to observe silences as creative spaces in the design process.In keeping with the action research paradigm, an extensive examination of the literature not only provides a background for the research questions but also continues to be threaded throughout the thesis as data collection cycles directed further literature review. The thesis narrative is given an original form through the use of a gardening metaphor that serves to highlight the rewarding, delicate and transitional nature of this kind of educational design. Such transitional aspects of educational design allow for innovation and creativity not evident in the systems-based approaches to designing instruction. This research also supports current initiatives in Australian higher education concerning the first year experience, embedding graduate attributes in the curriculum, and blending on-campus and off-campus learners into one class. The transdisciplinary approach to educational design explored through this research responds effectively to the varied issues in designing online assessment and developing innovative approaches by academic staff

This research develops a new approach to the development of training inexperienced journalists in news writing using a web-based platform of instruction delivery. E-training is growing in the world as an instructional setting, and offers not only financial benefits, but also a range of specific advantages over the traditional classroom type of setting. Such advantages include the ability to personalise the content of training to the trainees' current competencies, to facilitate regular multi-faceted monitoring of the changes in these competencies and to combine learning with the immediate practising of what was learned. Two e-training tools have been created and validated in this research: the news text assessment system (NTA) - a comprehensive and effective online scoring rubric, i.e. a matrix describing different levels of competency in several dimensions of the assessed performance - to assess the quality of news writing; and a scoring e-textbook (SET), an asynchronous news writing training tool. The SET is built around the NTA as its core element and contains hundreds of self-learning modules including exercises, examples, instructional texts, and quizzes to be used in a non-linear fashion according to the specific needs of trainees. Both the exercises and the NTA are elements of corrective feedback, which in psychological literature has been shown to be most effective in changing the subsequent performance of trainees. The two tools help both the trainee and the instructor. They assist the instructor to identify and address journalists' weak and absent competencies in news writing and consistently upgrade the learning modules when needed. They help trainees to monitor their progress and to learn from their own mistakes in the short periods of spare time they have during their work or in other time they can spare for the training. To create the NTA, 53 top journalism experts, both practitioners and academics, used the prototype of the assessment rubric with 30 criteria of news writing to assess the quality of several supplied news stories. The results were then subjected to statistical analysis and the NTA rubric was created as a compromise between its comprehensiveness and user-friendliness. To evaluate the NTA and the SET, an experiment was conducted with journalists in one post-Communist country in the form of an action research project, where this researcher was also the instructor. The experiment consisted of four months of training and reflection on its results by both the journalists and the researcher. The results show improvement in news writing competencies to an internationally 'acceptable' news writing standard for most of the trainees. The suggested tools have been well received and the trainees appreciated the interactivity that was provided during the training.