Grades serve as one of the primary indicators of student learning, directing subsequent actions for students, instructors, and administrators, alike. Therefore, grade validity—that is, the extent to which grades communicate a meaningful and credible representation of what they purport to measure—is of utmost importance. However, a grade cannot be valid if one cannot trust that it will consistently and reliably result in the same value, regardless of who makes a measure or when they make it. Unfortunately, such reliability becomes increasingly challenging to achieve with larger class sizes, especially when utilizing multiple evaluators, as is often the case with mandatory introductory courses at large universities. Reliability suffers further when evaluating open-ended tasks, as are prevalent in authentic, high-quality engineering coursework.<div><br></div><div>This study explores grading reliability in the context of a large, multi-section engineering course. Recognizing the number of people involved and the plethora of activities that affect grading outcomes, the study adopts a systems approach to conduct a human reliability analysis using the Functional Resonance Analysis Method. Through this method, a collection of data sources, including course materials and observational interviews with undergraduate teaching assistant graders, are synthesized to produce a general model for how actions vary and affect subsequent actions within the system under study. Using a course assignment and student responses, the model shows how differences in contextual variables affect expected actions within the system. Next, the model is applied to each of the observational interviews with undergraduate teaching assistants to demonstrate how these actions occur in practice and to compare graders to one another and with expected behaviors. These results are further related to the agreement in system outcomes, or grades, assigned by each grader to guide analysis of how actions within the system affect its outcome.<br></div><div><br></div><div>The results of this study connect and elaborate upon previous models of grader cognition by analyzing the phenomenon in engineering, a previously unexplored context. The model presented can be easily generalized and adapted to smaller systems with fewer actors to understand sources of variability and potential threats to outcome reliability. The analysis of observed outcome instantiations guides a set of recommendations for minimizing grading variability.<br></div>

Looking into and Evaluation Quallity, Efficiency and Effects of Further Education A thesis centers upon description and comparison of methods that gain a feedback in the field of the quality of further education. It considers both institutions providing further education and institutions/organisations (e.g. schools, offices of public administration) representing source of participants of further education courses. A theoretical part the thesis compares well known methods reporting on the quality and effectivity of further education in a view of both a provider and a customer. A research aims for an analysis of gaining a feedback in the field of further education from 50 providing institutions and 340 institutions/organizations in a role of a customer. Those 340 institutions/organizations include 32 offices of public administrative and another 308 publicly figuring organizations (e.g. schools). This part of the thesis also presents positives and negatives of chosen methods. A conclusion of the thesis summarizes findings and provides recommendations for both surveyed groups.

The desire to produce a learning environment which promotes student motivation, collaboration, and higher order thinking is common within the higher education system of today. Such learning environments also have the ability to address challenges’ Mechanical Engineering Technology (MET) students face entering the workforce. Through the vertical and horizontal integration of courses, this research presents how a scaffolded learning environment with a centralized theme of energy can increase motivation and conceptual retention within students. The integration of courses allows students to systematically translate their competency of concepts between energy based courses through experiential learning. The goal of this work is to develop a competency based learning model where students earn a professionally recognizable credential. The credential is earned through demonstrating their mastery of industry desired skills at a level that goes above and beyond the stock curriculum. The result is a more continuous curriculum that enhances multi-disciplinary problem solving while better preparing MET students for the workforce.

Increasing enrollment of students and declining availability of qualified and experienced faculty are leading to increased assessment loads of the existing faculty. Moreover, the assessment techniques are changing drastically due to the ever-increasing demand of new knowledge and abilities from the students. The tools offered by information technology can now be effectively used in enhancing the productivity of a teacher. This thesis proposes a mechanism for creating both summative and formative assessment instruments for a course in an engineering program. The assessment instruments will vary widely in nature depending on the subject. With increasing prevalence of digital devices in all walks of life a first level knowledge of digital systems is considered necessary for all engineers especially under electrical and computer engineering curricula. The first level course ‘Basics of Digital Systems’ is chosen for developing a framework of computer assisted evaluation. Creation of assessment instruments is best done in the context of an instructional system design (ISD) model. ADDIE, a generic model is chosen for the study. Bloom’s classification of levels of cognition, Vincenti’s categorization of engineering knowledge, and ‘Gronlund 2-level’ method for writing the learning objectives are integrated to create a ‘Bloom-Vincenti–Gronlund’(BVG)framework for preparing the learning objectives/assessment instruments. Developing tools for evaluation of performance of students in the assessment tests requires consideration of many issues: analysis of problems and their solution methods, errors normally committed by students, grading preferences of the instructor and feedback to students. A set of tools are developed that are able to evaluate the truth tables, state tables, excitation tables, timing diagram and VHDL codes. The developed tools are validated. The submission of the assignment and the integration of all the tools for evaluation will be more effective if they can be integrated in a learning management system (LMS). ‘MOODLE’, an open source LMS, is identified for the integration of the tools. The developed tools execute the files submitted by the students, evaluate them, and provide feedback to the students. In summary, the thesis addressed some key issues related to “assessment and evaluation of students’ performance” and proposed an integrated computer assisted system for the evaluation of students’ performance in the course ‘Basics of Digital Systems’.

<div>Service-learning is an organized volunteer activity in which learners serve the community while utilizing and enhancing their own skills, thus benefiting both the learners and the community. Studies have shown that students gain various benefits from participating in a service-learning activity, especially in their academic skills and civic growth through continued reflections (Eyler, Giles, & Braxton, 1997; Eyler & Giles, 1999; Billig, 2000; Grassi et al., 2004; Steinberg, Bringle, & Williams, 2010), often increasing their motivation to learn the related subject (Steinberg et al., 2010). Service-learning has been implemented in foreign language courses in the United States, especially Spanish (Barreneche & Ramos-Flores, 2013). However, service-learning literature on Japanese as a foreign language is limited.</div><div>The researcher founded a service-learning program in the Japanese language. In the program, the university students enrolled in intermediate- or higher-level Japanese courses help Japanese children with their schoolwork as volunteer tutors. The researcher conducted a qualitative case study on four of the student-tutors to examine the program's potential benefits to maintain and enhance the student-tutors' various motivations toward learning Japanese. The Volunteer Functions Inventory (VFI) (Clary, Snyder, & Ridge, 1998) was used as an analysis scheme, which reports six most commonly found functions, or varying motivations, for participating in a volunteer activity. The student-tutors indicated five out of the six VFI functions, showing a connection between their service-learning experience and their personal growth. They built strong connections with the Japanese community and kept their motivation to improve their Japanese skills to better help the children. It is hoped that the present research will contribute to providing an example of Japanese service-learning in the U.S.</div>

<p>Internationalization of higher education is a collaborative responsibility academic and non-academic programs share to facilitate the integration of various student populations within the broader culture of the university. My dissertation project links First Year Writing (FYW) classes of domestic and international students to promote and evaluate their intercultural competence development. My research questions explore the use of reflective writing as a genre for formative assessment in the writing classroom and investigate the data it provides about students’ continuous learning. My research methodology combines qualitative analysis of reflective writing and quantitative analysis of intercultural competence development. Participants come from four sections of FYW courses spanning two semesters – Spring 2016 and Fall 2017. I collected reflective writing data from four embedded reflective journals and a final reflective essay assigned to students in each section. Using a grounded scheme, I applied thematic coding analysis of reflective writing and traced frequencies of codes. I also mapped students’ reflections onto the Developmental Model of Intercultural Sensitivity (DMIS; Bennett, 1993). Results from both coding methods contextualize and interpret students’ development in both intercultural competence and writing skills. I also share pedagogical, assessment, and administrative implications for more effective teaching of reflective writing and better continuous assessment of intercultural competence skills within the context of the linked course model curriculum. </p> <p> </p>

<p>Content about learners with gifts and talents is not necessarily a part of most teacher education programs. Without high quality training and professional development opportunities, preservice and inservice teachers are left with no tools to identify and serve the students with gifts and talents. However, adding more content is not enough. The successful translation of training and professional development into effective practice depends on understanding teacher motivation, debunking misconceptions, building adequate knowledge base, and building teacher identity. I adopted several theoretical perspectives in this study: teacher identity formation (Gardner & Kaplan, 2018), Teacher Efficacy (Tschannen-Moran et al., 1998), Teacher Goal Orientations (Butler, 2007), beliefs about gifted learners and gifted education (Gagne & Nadeau, 1991; McCoach & Siegle, 2007), desire to teach (Watt & Richardson, 2007). My participants were 236 preservice teachers who desire to teach learners with gifts and talents and inservice teachers in gifted education.</p><p>The objectives of this mixed-methods investigation were: (1) identifying the differences between preservice and inservice teachers in measures of identity, beliefs, motivation, and desire to teach learners with gifts and talents, (2) modeling the structural relationships among dimensions of identity, motivation, beliefs, and desire to teach, and (3) understanding how participants experiences and perceptions inform their identity, motivation, and belief systems. I used a combination of Multivariate Analysis of Variance (MANOVA), Structural Equation Modeling (SEM) and qualitative thematic analysis to answer my research questions.</p><p>Findings revealed inservice teachers (n=155) have high levels of relational goals, instructional efficacy, positive beliefs, and teacher identity, while preservice teachers (n=81) have high levels of intrinsic motivation and social value for gifted education. SEM showed that teacher identity, mastery goals, influenced positive beliefs; teacher identity was influenced by efficacy, mastery and relational goals. The strongest predictors of desire to teach learners with gifts and talents were teacher identity, teacher efficacy, and relational goals. Qualitative findings indicated that self-perceptions as gifted played a meaningful role in participants deciding to become teachers, understanding the needs of gifted learners, and advocating for gifted education.</p>

<p>Innovations in technology are changing not only everyday life for many individuals around the world but are also influencing the expansion of online learning opportunities at an accelerated rate (Collins & Halverson, 2<a></a>018; Mah, 2016). Online learning platforms allow for scalability, flexibility, greater global access, and innovative and new ways to deliver education (Goodman, Melkers, & Pallais, 2019; Kizilcec et al., 2019). Enrollments in online learning programs and opportunities have seen significant growth in recent years (Seaman, Allen, & Seaman, 2018; U.S. Department of Education, 2018) with continued and steady growth expected into the future. The ubiquity and newness of new online learning formats present a challenge in linking research and practice. Through three separate academic papers, the following dissertation discusses and considers key questions and topics with regards to the use of digital badges and Massive Open Online Courses (MOOCs), two types of emerging online innovations and platforms, and aspects of their efficacy. The three papers respectively 1) identify and discuss the theoretical and empirical foundations digital badges use in specific learners groups by reviewing current literature; 2) highlight the application of a use case in which digital badges have been implemented as a means to offer training; and 3) explore the perceptions of MOOC instructors toward quality learning in their courses in a case study. Conclusions are drawn and solutions as well as potential future directions for research and practice of discussed. </p>

<p>In an era of widespread automation—from grocery store self-checkout machines to self-driving cars—it is not outrageous to wonder: can teachers be automated? And more specifically, can automated computer teachers instruct students how to write? Automated computer programs have long been used in summative writing evaluation efforts, such as scoring standardized essay exams, ranking placement essays, or facilitating programmatic outcomes assessments. However, new claims about automated writing evaluation’s (AWE) formative educational potential mark a significant shift. My project questions the effectiveness of using AWE technology for formative educational efforts such as improving and teaching writing. Taken seriously, these efforts portend a future embrace of semi, or even fully, automated writing classes, an unprecedented development in writing pedagogy.</p><p>Supported by a summer-long grant from the Purdue Research Foundation, I conducted a small-<i>n </i>quasi-experiment to test claims by online college tutoring site Chegg.com that its EasyBib Plus AWE tool can improve both writing and writers. The experiment involved four college English instructors reading pairs of essays comprising one AWE-treated and untreated version per pair. Using a comparative judgment model, a rubric-free method of writing assessment based on Thurstone’s law, raters read and designated one of each pair “better.” Across four raters and 160 essays, I found that AWE-treated essays were designated better only 30% of the time (95% confidence interval: 20-40%), a statistically significant difference from the null hypothesis of 50%. The results suggest that Chegg’s EasyBib Plus tool offers no discernible improvement to student writing, and potentially even worsens it.</p><p>Finally, I analyze Chegg’s recent partnership with the Purdue Writing Lab and Online Writing Lab (OWL). The Purdue-Chegg partnership offers a useful test case for anticipating the effects of higher education’s embrace of automated educational technology going forward. Drawing on the history of writing assessment and the results of the experiment, I argue against using AWE for formative writing instruction. In an era of growing automation, I maintain that a human-centered pedagogy remains one of the most durable, important, effective, and transformative ingredients of a quality education.</p>

<p dir="ltr">Despite educators implementing bias-reducing strategies, accurately evaluating team members’ individual contributions remains a challenge. This study investigated the extent of bias in team member evaluations within a design thinking course. Self-, peer, and undergraduate teaching assistants (mentors) used the Comprehensive Assessment for Team Member Effectiveness (CATME) tool to rate team members based on their contributions to a capstone project.</p><p dir="ltr">To measure alignment between self-, peer, and mentor ratings, statistical analyses such as Kendall’s W Coefficient of Concordance and Spearman’s rank correlations were performed to assess interrater reliability. Other non-parametric tests were used to analyze the ratings’ correlations with grades and demographic data. The analyses revealed moderate alignment between peer and mentor ratings, which suggested that external perspectives were consistent and reliable measures of contributions. Self-ratings had weaker alignment with peer and mentor ratings, indicating a disconnect between students’ internal self-perception and how their teams and mentor saw them contribute. Although there were no statistically significant relationships between ratings and grades, a negative relationship between self-ratings and grades was found, which suggested that students who inflated self-evaluations performed slightly worse on assignments. Further analyses split by course section exposed how demographic factors influenced mentor ratings, illuminating hidden biases that were veiled in the pooled dataset.</p><p dir="ltr">The findings from this study highlight the importance of refining bias-reducing strategies. The objective was to enhance the fairness and accuracy of team member evaluations, thereby reducing frustration from unfair grading and improving the educational experience and outcomes for students engaged in collaborative projects.</p>