The aim of this thesis is to examine how teachers use computer based assessment in their work and what user barriers they experience while working with computer based assessments. The result of the thesis is meant to provide an understanding of which factors affect the use of computer based assessments and how a company who provide computer based assessments applications can increase the rate of use amongst teachers.To investigate this, the author examined a company in Sweden who provide a computer based assessment application for both universities and high schools. The study was carried out at two different schools, a university and a high school, both located in Stockholm. In total nine teachers were interviewed. The author also conducted observations at the company whose computer based assessment application was examined.The author’s findings suggest that a lack of functions within the application and lack of costumer support from the company all negatively impacted the teacher’s use of the application. There were also findings that could suggest that a lack incentive to use the application may have negatively impacted teacher’s rate of use.To increase the rate of use, the author suggest that companies must work closer with teachers to understand their routines and develop applications accordingly. Companies should also be using teachers who already are using computer based assessment to develop incentives so that other teachers may start to use computer based assessments applications more frequently

Thesis advisor: Ina V.S. Mullis / This dissertation describes the foundation for maintaining TIMSS’ 20 year trend measurements with the introduction of a new computer- and tablet-based mode of assessment delivery—eTIMSS. Because of the potential for mode effects on the psychometric behavior of the trend items that TIMSS relies on to maintain comparable scores between subsequent assessment cycles, development efforts for TIMSS 2019 began over three years in advance. This dissertation documents the development of eTIMSS over this period and features the methodology and results of the eTIMSS Pilot / Item Equivalence Study. The study was conducted in 25 countries and employed a within-subjects, counterbalanced design to determine the effect of the mode of administration on the trend items. Further analysis examined score-level mode effects in relation to students’ socioeconomic status, gender, and self-efficacy for using digital devices. Strategies are discussed for mitigating threats of construct irrelevant variance on students’ eTIMSS performance. The analysis by student subgroups, similar item discriminations, high cross-mode correlations, and equivalent rankings of country means provide support for the equivalence of the mathematics and science constructs between paperTIMSS and eTIMSS. However, the results revealed an overall mode effect on the TIMSS trend items, where items were more difficult for students in digital formats compared to paper. The effect was larger in mathematics than science. An approach is needed to account for the mode effects in maintaining trend measurements from previous cycles to TIMSS 2019. Each eTIMSS 2019 trend country will administer the paper trend booklets to an additional nationally representative bridge sample of students, and a common population equating approach will ensure the link between paperTIMSS and eTIMSS scores. / Thesis (PhD) — Boston College, 2018. / Submitted to: Boston College. Lynch School of Education. / Discipline: Educational Research, Measurement and Evaluation.

A case study approach is used to examine two formative computer based assessments (CBAs) used in a School of Medicine and Faculty of Education at a Canadian university. Both assessments contained primarily scenario-based, multiple-choice items however the number of items and complexity was different. The formative CBAs were examined in terms of how feedback is provided to students, the effectiveness of different types of feedback, and the characteristics of items used for assessment. Feedback was channelled to students in one case, immediately following responding to an item and in the second case, at the end of a set of items. Feedback given to students immediately following an item was more effective given that the timing captured students’ construction of knowledge. The most favoured type of feedback provided additional information. Students also indicated that feedback using a live internet link to direct them to a resource would be acceptable. Although feedback that simply stated a response was correct or incorrect was previously shown to be ineffective, students from the School of Medicine indicated this type of feedback was acceptable for low cognitive items. In both case studies, students reported that more items were required and in one case study, students recommended these items be added at the higher end of the cognitive scale. / Thesis (Ph.D, Education) -- Queen's University, 2009-01-28 13:55:37.823

International studies of reading achievement demonstrate that significant inequalities in reading outcomes continue to exist among some of the world’s wealthiest countries, despite strong investment in initiatives directed towards raising literacy achievement for all children (United Nations Educational, Scientific and Cultural Organisation—UNESCO, 2009; United Nations Children’s Fund—UNICEF, 2010). One approach towards the elevation of reading achievement is to investigate how key predictors of reading success are incorporated into everyday classroom literacy practices. Phonological awareness (PA) is widely recognised as a powerful predictor and underlying precursor to early reading success for both typically developing and at-risk readers (Al Otaiba, Kosanovich, & Torgesen, 2012; Blachman, Ball, Black, & Tangel, 2000; Goswami, 2001; Pressley, 2006). A majority of research demonstrating the benefits of PA to literacy growth has been conducted under controlled research settings outside of the classroom environment (Ehri, Nunes, Willows, Schuster, Yaghoub-Zadeh, & Shanahan, 2001; Gillon, 2000a, 2005; Gillon & McNeill, 2009), and thus less is known about whether such benefits hold true when integrated into the heterogeneous classroom setting. For this reason, four experiments reported in this thesis investigated whether PA can be efficiently and effectively integrated into the classroom literacy programme with the overarching aim of raising reading achievement and equalising reading outcomes for the majority of children in the first year of formal education. In the first experiment (reported in Chapter 3), time-efficiency and congruency of scores between a computer-based PA screening and monitoring tool (described in Chapter 2) and a paper-based equivalent were examined. Thirty-three children aged between four years 10 months and five years zero months participated in the study, 12 of whom presented with moderate-severe speech delay (MSD). Participants were randomly allocated to either Group A or Group B experimental assessment conditions. A crossover research design was employed where Group A received the paper-based version of the PA assessment followed two weeks later by the equivalent computer-based assessment (CBA). Group B received the same assessments but in the reverse order of delivery. That is, the computer-based PA assessment first followed two weeks later by the paper-based counterpart. Results demonstrated that: 1) the CBA generated comparable scores to the paper-based equivalent for both children with typical development and children with MSD, and 2) CBA took 31 per cent less time than paper-based administration. These results demonstrate that CBA can provide educators with a time-efficient approach to the screening and monitoring of PA development in the classroom while maintaining equivalency of scores with paper-based testing. Having established the time-efficiency of CBA, the next step was to investigate the use of the computer-based PA screening and monitoring tool as part of the beginning classroom reading programme. In the second experiment (reported in Chapter 4), the influence of a short and intensive period of teacher-implemented classroom PA instruction on reading outcomes in the first year of education was investigated. One-hundred and twenty-nine children aged five-years participated in the study. Using a quasi-experimental design, thirty-four children in two classrooms received 10 weeks of PA instruction from their teachers, as an adjunct to the ‘usual’ reading programme. Ninety-five children from 10 classrooms continued with the ‘usual’ reading programme, which included phonics instruction but did not target PA. Results demonstrated that children exposed to classroom PA instruction performed significantly higher on reading and spelling measures compared to children who received the ‘usual’ reading programme only. Of importance, the number of children experiencing word decoding difficulties after one year of schooling reduced from 26 per cent among children who followed the ‘usual’ reading programme to 6 per cent among children who received classroom PA instruction. These results provide evidence that a short and intensive period of classroom-wide PA instruction in the first year of schooling can have a positive influence on raising reading achievement. In the third experiment (reported in Chapter 5), the effect of classroom PA instruction on raising reading achievement and reducing inequality in literacy outcomes for children with spoken language impairment (SLI) was examined. The data from 129 five-year-old children who participated in the second experiment were extracted and analysed. End-of-year reading outcomes between children with SLI who received classroom PA instruction (n = 7) was compared to: 1) children with typical language development (TD) who received classroom PA instruction (n = 27), 2) children with SLI who followed the ‘usual’ reading programme (n = 21), and 3) children with TD who followed the ‘usual’ reading programme (n = 74). Children with SLI who received classroom PA instruction showed significant improvements in PA, reading and spelling acquisition immediately and up to six months following PA instruction. However, this cohort, in comparison to children with TD, appeared less able to transfer their enhanced PA knowledge to reading and writing tasks. Of importance, children with SLI who received PA instruction performed significantly higher than children with SLI who followed the ‘usual’ reading curriculum; and on par with children with TD who followed the ‘usual’ reading programme. Children with TD who received classroom PA instruction significantly outperformed all other cohorts in this experiment on end-of year reading measures. These results indicate that both children with TD and children with risk for reading difficulties can benefit from classroom-wide teacher-directed PA instruction. These findings have positive implications for elevating reading achievement and reducing inequality between good and poor readers. In the fourth experiment (reported in Chapter 6), the validity and reliability of the computer-based PA screening and monitoring tool was investigated and established. Using a longitudinal research design, the responses of 95 children to test items in the CBA at the start, middle and end of the first year at school were collated and analysed to provide evidence of content, construct and criterion validity, in addition to test-retest and internal consistency reliability. A number of statistical analyses were employed including Rasch Model analysis, exploratory factor analysis and multiple regression analysis. Results demonstrated that the majority of test items were appropriate for five-year-old children in the first year of school and sampled a spectrum of ability levels that would be present in a typical classroom environment. Rhyme oddity, initial phoneme identity and letter-knowledge tasks were most appropriate at school-entry while tasks of final phoneme identity, phoneme blending and phoneme segmentation became more suitable by the middle and end stages of the first year at school. Importantly, performance on the CBA predicted end-of-year reading status with 94 per cent accuracy, and in conjunction with language abilities accounted for 68.9 per cent of the variance in end-of-year reading performance. These findings indicate that the computer-based PA screening and monitoring tool developed and applied in this thesis has sufficient validity and reliability to be used confidently as a time-efficient assessment tool in the classroom. The results from the experiments reported in this thesis provide evidence that PA can be efficiently and effectively integrated into the beginning classroom reading programme from two complementary perspectives: 1) through use of computer-based screening and monitoring of PA skills, and 2) through implementation of a short and intensive period of teacher-directed classroom-wide PA instruction. The results reported in this thesis demonstrate that the evidenced-based integration of key predictors of literacy success, such as PA, into existing classroom programmes can support national and international initiatives that seek to raise reading achievement and reduce inequalities in literacy outcomes for all children.

Due to the increasing demands of third-party payers to show measurable results in a decreased amount of time, mental health practitioners need new tools and techniques to quickly and efficiently assess their client's current status and progress in treatment. The goal of this study was to develop and validate a brief computerized measure of emotional states and personality traits for use as an assessment tool in a clinical setting. Adapted from the State-Trait Personality Inventory (STPI; Spielberger, 1998), the computerized instrument measured state and trait anxiety, anger, depression, and curiosity. The computerized STPI was examined for equivalence with the traditional STPI and compared with therapists' ratings of their clients' emotional states and personality traits. Forty individuals (24 women, 16 men) in psychotherapy were administered either a computerized version of the STPI or the original, paper version. Each client's therapist was asked to rate him or her on the same 4 states and 4 traits measured by the STPI. In addition, clients and therapists answered short questionnaires regarding their reactions to the STPI and the client report generated by the STPI, respectively. A comparison of mean STPI scale scores between the computerized and paper administration groups showed slightly higher levels of reported state emotion for the computerized administration. There was no other evidence of differences between the two experimental groups. Therapists who had previously conducted assessment with their client demonstrated a significantly higher correlation of their ratings with the corresponding client STPI self-reported scores. Feedback from the client and therapist exit questionnaires suggested that their reactions to the procedure and the usefulness of the STPI were quite favorable. Implications of increasing the use of computers in clinical assessment are discussed.

Pre-Trip Inspection of the truck and trailer is one of the components of the current Commercial Driver's License (CDL) test. This part of the CDL test checks the ability of the student to identify the important parts of the commercial vehicle and their potential defects. The Virtual Check Ride System (VCRS), a computer-based application, is an assessment and feedback tool that mirrors the inspection component of the actual CDL. The VCRS has provided an after action review (AAR) via a feedback session that helps in identifying and correcting drivers' skill in inspecting parts and for overall safety. The purpose of this research is to determine the effectiveness of the VCRS in truck driving training programs. An experimental study was conducted with truck driving students at Mid Florida Tech, located in Orlando, Florida. The students were divided into control and experimental groups. Students in the both groups received regular training provided by Mid Florida Tech. The experimental group received additional training by making use of the VCRS. A total of three paper-based tests were given to all subjects during first three weeks; one test at the end of a week. Both groups were given the same paper-based tests. A two-way analysis of variance was conducted to evaluate the effect of the VCRS in the experimental group. This analysis found a significant difference between control and experimental groups. This effect showed that the students in the experimental group increased their performance by using VCRS. Moreover, there was a main effect in the scores of each week. However, there was not an interaction between the two factors. Follow up Post Hoc tests were conducted to evaluate the pair-wise differences among the means of the test week factors using a Tukey HSD test. These Post Hoc comparisons indicated that the mean score for the third week's test scores were significantly better than the first week's test score in the experimental group. It was concluded that the VCRS facilitated learning for the experimental group and that learning also occurred for both groups as a result of repeated testing.

This study investigated the examinees&rsquo / perceptions about an online assessment tool and an online assessment center. In this study, a computer literacy exam (Bilisim Seviye Tespit Sinavi, BSTS) was delivered in purposively designed environment, an online assessment center, at CEC, METU (Continuing Education Center, Middle East Technical University). The data were collected from 117 participants through the &ldquo / User evaluation questionnaire&rdquo / , in depth interviews with 43 examinees and 7 experts. 7 participants and the experts were interviewed individually while the rest 36 participants were interviewed in 6 focus groups (composed of 6 examinees). Descriptive statistics, frequency distributions and comments of participants and experts were used to express the results of the study. The results showed that the examinees perceived the online assessment tool and the online assessment center suitable for delivering online assessments. However, both participants and experts reported that a few changes could be done to the interface of online assessment tool. They also suggested that the computer literacy assessments be more authentic.

There has recently been an increased emphasis on the importance of learning programming languages, not only in higher education but also in secondary schools. Students of a variety of departments such as physics, mathematics and engineering have also started learning programming languages as part of their academic courses. Assessment of students programming solutions is therefore important for developing their programming skills. Many Computer Based Assessment (CBA) systems utilise multiple-choice questions (MCQ) to evaluate students performance. However, MCQs lack the ability to comprehensively assess students knowledge. Thus, other forms of programming solutions are required to assess students knowledge. This research aims to develop a semi-automated assessment framework for novice programmers, utilising a computer to support the marking process. The research also focuses on ensuring the consistency of feedback. A novel marking process model is developed based on the semi-automated assessment approach which supports a new way of marking, termed segmented marking . A study is carried out to investigate and demonstrate the feasibility of the segmented marking technique. In addition, the new marking process model is developed based on the results of the feasibility study, and two novel marking process models are presented based on segmented marking, namely the full-marking and partial-marking process models. The Case-Based Reasoning (CBR) cycle is adopted in the marking process models in order to ensure the consistency of feedback. User interfaces of the prototype marking tools (full and partial) are designed and developed based on the marking process models and the user interface design requirements. The experimental results show that the full and partial marking techniques are feasible for use in formative assessment. Furthermore, the results also highlight that the tools are capable of providing consistent and personalised feedback and that they considerably reduce markers workload.

Title from PDF of title page. / Document formatted into pages; contains 89 pages. / Thesis (M.A.)--University of South Florida, 2003. / Includes bibliographical references. / Text (Electronic thesis) in PDF format. / ABSTRACT: Due to the increasing demands of third-party payers to show measurable results in a decreased amount of time, mental health practitioners need new tools and techniques to quickly and efficiently assess their client's current status and progress in treatment. The goal of this study was to develop and validate a brief computerized measure of emotional states and personality traits for use as an assessment tool in a clinical setting. Adapted from the State-Trait Personality Inventory (STPI; Spielberger, 1998), the computerized instrument measured state and trait anxiety, anger, depression, and curiosity. The computerized STPI was examined for equivalence with the traditional STPI and compared with therapists' ratings of their clients' emotional states and personality traits. Forty individuals (24 women, 16 men) in psychotherapy were administered either a computerized version of the STPI or the original, paper version. / ABSTRACT: Each client's therapist was asked to rate him or her on the same 4 states and 4 traits measured by the STPI. In addition, clients and therapists answered short questionnaires regarding their reactions to the STPI and the client report generated by the STPI, respectively. [BR][BR] A comparison of mean STPI scale scores between the computerized and paper administration groups showed slightly higher levels of reported state emotion for the computerized administration. There was no other evidence of differences between the two experimental groups. Therapists who had previously conducted assessment with their client demonstrated a significantly higher correlation of their ratings with the corresponding client STPI self-reported scores. Feedback from the client and therapist exit questionnaires suggested that their reactions to the procedure and the usefulness of the STPI were quite favorable. / ABSTRACT: Implications of increasing the use of computers in clinical assessment are discussed. / System requirements: World Wide Web browser and PDF reader. / Mode of access: World Wide Web.

Frühe literale Kompetenzen gelten als wichtige Lernvoraussetzungen, auf denen der schulische Schriftspracherwerb aufbaut. Mithilfe einer frühzeitigen Diagnostik kann Schwierigkeiten im Erwerbsprozess vorgebeugt werden. In diesem Beitrag werden die Gütekriterien eines neu entwickelten, digitalen Instruments zur Erfassung früher literaler Kompetenzen für das erste Schuljahr vorgestellt. Das Verfahren umfasst Aufgaben zu basalen Kompetenzen, phonologischer Bewusstheit, Buchstabenkenntnis und erstem Lesen. Es liegt für den Beginn und die Mitte des Schuljahres in Parallelformen vor und wurde an zwei großen Stichproben (N = 1.042 bzw. 809) im Land Brandenburg normiert. In Teilstichproben wurde die Konstruktvalidität anhand des IEL-1 und des CFT 1-R ermittelt. Die Itemkennwerte ebenso wie die Befunde zu den Hauptgütekriterien weisen darauf hin, dass sich das Verfahren zur Routinediagnostik von Schulbereitschaft bzgl. früher literaler Kompetenzen in der Jahrgangsstufe 1 eignet. Damit liegt ein Verfahren vor, welches aus psychometrischer Sicht einen Zugewinn für die pädagogische Praxis liefert und in der schulischen Anwendung eine hohe Akzeptanz erfährt. / Early literacy skills are crucial for later reading achievements. Therefore, classroom assessment is essential for promoting reading success. Low-threshold assessment by educational staff is presently not very common in German primary schools. This study examined the validity and reliability of a digital assessment tool designed to measure early literacy skills in grade 1 of German primary school (ILeA plus Deutsch A). Two cohorts of children (N = 1,042/809) completed the assessment for phonological awareness, alphabet, and reading skills using a PC / laptop. The tool exhibited satisfying item properties (item-total correlation range = .87 to .98) and good internal consistency (all > .90). Correlations between scores for the tool and paper-pencil control tests were high (IEL-1, r = .58 / .67). The psychometric properties suggest that digital assessment of early literacy skills has the potential to improve assessment practices for classroom use. In addition, the easy application and automated calculations of the children's results were well received by the teachers.