In both engineering and physics education, a common objective is that students should learn to use theories and models in order to understand the relation between theories and models, and objects and events, and to develop holistic, conceptual knowledge. During lab-work, students are expected to use, or learn to use, symbolic and physical tools (such as concepts, theories, models, representations, inscriptions, mathematics, instruments and devices) in order both to understand the phenomena being studied, and to develop the skills and abilities to use the tools themselves. We have earlier argued that this learning should be seen as the learning of a complex concept, i.e. a “concept” that makes up a holistic system of “single” interrelated “concepts” (i.e. a whole made up of interrelated parts). On the contrary, however, in education research it is common to investigate “misconceptions” of “single concepts”. In this paper we will show the power of analysing engineering students’ learning as the learning of a complex concept. In this model “single concepts” are illustrated as nodes or “islands” that may be connected by links, while the links that students actually make are represented by arrows. The nodes in our model are found by looking for “gaps” in the actions and conversations of students. A gap corre­sponds to a non-established link, and when a gap is filled and the students establish a relation between two nodes, this is represented by a link. The more links that are made, the more complete the knowledge. In this study we report an analysis of a sequence of labs about AC-electricity in an electric circuit theory course. In for example electric circuit theory the “concepts” of current, voltage and impedance are interdependent. Rather, the central physical phenomenon is “electricity” represented by Ohms law as a generalization of the current/voltage/impedance/frequency-relationship of a circuit or circuit element. The results show the learning of “electricity” as a complex concept with students’ knowledge becoming more complete. Furthermore, according to our analysis “entities” that in later labs were fused into one were separate in the earlier labs. For example in a later lab we could note that “the physical circuit” and “the circuit drawing” had fused into a single “real circuit”. Our results suggest that the learning of a complex concept first start with establishing more and more links. As links become well established, “entities” that have been separate fuse into a whole. Our model suggests a method for finding “learning difficulties” since these corresponds to “gaps” and non-established links. As teachers and experts in a field we can miss to uncover these since for us the ‘complex concept’ has become a conceptual whole and we may no longer be able to distinguish the parts in the complex. In line with the thesis of M. Holmberg we also argue that learning problems in electric circuit theory may be due to the common failure to appreciate that concepts are relations.

<p>Ongoing efforts seek to develop engineering students into entrepreneurially minded engineers. Often, work to achieve that goal relies on theories drawn from entrepreneurship research from business disciplines to develop interventions and ground research on engineering entrepreneurship education. However, despite repeated warnings by multiple scholars, there has been limited evaluation of whether such theories are appropriate to design interventions or understand the development of students’ entrepreneurial expertise. Theories of entrepreneurship developed in the field of entrepreneurship typically make several assumptions or research design choices pertinent to their usefulness in education. Those assumptions include assuming those studied make no errors, building expert-comparative rather than expert-novice theories, and mythicizing and reifying certain types of entrepreneurs. One such theory, the <i>Theory of Effectuation</i>, is representative of these assumptions as well as being commonly used in entrepreneurship education as a model of correct decision-making. Prior studies have used the Theory of Effectuation to compare experts and students and track students’ growth, but have presumed error free reasoning by both experts and students.</p> <p>My dissertation focuses on empirically evaluating the appropriateness of one assumption from the Theory of Effectuation when applying the theory to engineering students’ decision-making. The assumption I focus on is what errors engineering students make when working on typical early stage entrepreneurship decisions. The existence of such errors would call into question whether the Theory of Effectuation, which does not allow for such errors, can usefully describe engineering students’ decision-making. Interpreting the resulting errors can also help educators inform educators about pre-existing knowledge and beliefs that students bring to entrepreneurship classrooms. This can enable the design of more effective research studies and interventions to improve the state of the field</p> <p>To do so, I completed a verbal protocol study with engineering students at two universities. The verbal protocol used is based on one previously used to develop the Theory of Effectuation and asks participants to think aloud while making decisions typical of an early-stage entrepreneurial venture. I then coded the transcribed data from those protocols for conceptual errors related to business and management concepts. A thematic analysis of the results showed several consistent patterns of errors. Those included misinterpreting market research data as representative of their company’s financial performance, misunderstanding and using faulty analogies to analyze different outside investment options, and perceiving that they would personally receive all proceeds from a company’s sale. In general, two overarching patterns emerged – overestimating the value of their venture and overestimating their control.</p> <p>I end by interpreting the results through three existing areas of literature to provide new knowledge to engineering entrepreneurship educators. First, the patterns of errors appear similar to other misconceptions in that a potential alternative ontology that students rely on may exist in mythicization work, however more evidence is necessary to formally establish that the patterns of errors are in fact ontological miscategorizations. Second, the patterns of errors are strikingly similar to the myths about entrepreneurs that have been identified in media and research that reports on entrepreneurs. This suggests a specific source of students’ preconceptions about entrepreneurship that educators should actively engage with. Third, the findings validate existing theoretical critiques of how entrepreneurship theory is used in engineering education. Specifically, theories developed in entrepreneurship literature appear to be a poor fit for engineering education research because of their embedded assumptions.</p>

<p>There is a large variation in the quantity and quality of ethics that U.S. engineering students learn. Why is there so much room for improving the state of engineering ethics education in the United States? Recognizing the interplay between individual agency, structural factors, and historical contingency, this dissertation is a three-part approach to answering that question – I present three distinct, mutually informative threads for studying engineering ethics education from different angles. The first thread is an historical approach. The second thread is an empirical study of the mental models that faculty members have regarding engineering ethics education. The third thread applies theoretical constructs from political science and economics to analyze structural factors impinging on engineering ethics education.</p><p><br></p> <p>From the studies, first we see that trailblazers of engineering ethics developed the new knowledge required of this emerging field through interpersonal relationships; they leveraged existing organizations and built new institutional mechanisms for sharing knowledge and creating a community of scholars and an engineering ethics curriculum; they utilized resources from supportive colleagues and administrators to corporate, governmental, and nongovernmental funding that legitimated their work. Their efforts ultimately created pedagogical materials, prevalent ideas, publication outlets, meetings, and foundations that not only contributed to the current state of U.S. engineering ethics education but also the launching point for future generations to build upon and continue developing that state. Second, mapping the mental models of engineering ethics education among engineering faculty members provided a typology for analyzing the state of engineering ethics education and places where one can expect to find variation, deepening our understanding of the state of engineering ethics education. Third, outlining a theory of the political economy of engineering education highlighted factors that could be influencing curricular and pedagogical decisions in engineering departments. Furthermore, I supplemented the outlined theoretical phenomena with data from the mental models interviews in order to provide a proof of concept and relevant grounding for the phenomena.</p><p><br></p> <p>In sum, faculty members make decisions based on their mental models. Structural factors shape the broader environment and institutions in which those faculty members operate. Those structures and institutions change over time, leading to the current state of engineering ethics education. Having all three pieces has provided a more complete understanding of the state of U.S. engineering ethics education.</p><p><br></p> <p>Ultimately, my dissertation accomplishes multiple goals. First, I have provided additional evidence for understanding and explaining the qualitative and quantitative discrepancies of engineering ethics coverage in U.S. undergraduate engineering education at multiple levels of analysis. Second, I have amassed evidence that can inform future research efforts. Third, I have demonstrated the use of certain theories and methods infrequently employed in engineering education research. Finally, I have outlined potential new avenues for interdisciplinary research, especially at the nexus of political economy, education, engineering, and society. </p>

<div>Disciplinary perspectives, as a core element of interdisciplinary work, represent the ways individuals may see and approach a situation based on their unique disciplinary background and training. Interdisciplinary collaboration requires individuals to leverage disciplinary perspectives and knowledge from diverse fields to build a shared understanding of the problem situation. However, based on the diversity of background and experiences within a team, interdisciplinary collaboration can be a challenge because collaborators must negotiate disciplinary differences, while also fundamentally experiencing the collaborative situation in different ways. Therefore, it is important to understand how individuals engage and experience disciplinary perspectives in their practice of collaboration. In this study, I investigated the nature of disciplinary perspectives in the context of educational development. </div><div><br></div><div>The profession of educational development broadly aims to support the teaching and learning mission of higher education institutions, where educational developers work with faculty, graduate students, and administration on teaching, instruction, curriculum, and organizational development across disciplines. As such, educational developers play a significant role in engineering education transformation and offer a unique context to investigate interdisciplinary practice. In this work, educational developers bring their diverse disciplinary perspectives to their collaborative interactions. </div><div><br></div><div>In this dissertation, a phenomenographic study was conducted to investigate the following research question: how do educational developers experience disciplinary perspectives in the work of educational development? Phenomenography is a qualitative research approach that focuses on the variation in how a phenomenon is experienced and conceptualized. I adopted a situative theoretical perspective to see disciplinary perspectives in relation to the contexts, social interactions, and activities through which interdisciplinary work is performed. I conducted semi-structured interviews with eighteen educational developers from Centers for Teaching and Learning across the United States and Canada. Participants were recruited from various disciplinary backgrounds and levels of experience. In the interview, participants shared general descriptions about their work, and specific descriptions of an experience where they worked with others who contributed different disciplinary perspectives. Additionally, a scenario-based elicitation exercise was used to frame participants’ description of how diverse disciplinary perspectives appear in their work. The analysis followed an iterative and generative process to discern features and qualities of disciplinary perspectives. </div><div><br></div><div>The findings of this study are presented as a phenomenographic outcome space consisting of five categories of description as distinct ways that disciplinary perspectives are experienced by educational developers. Additionally, the findings illustrate how disciplinary perspectives become externalized as an object that is brought forward and shaped in collaborative interactions. This research contributes to further understanding interdisciplinary collaboration in two ways. First, for interdisciplinary practice, the findings provide an integrated view of the variation in ways of experiencing disciplinary perspectives such that educational developers may attune and attend to different collaborative interactions. Second, with the situative perspective, I provide insight into the situated knowledge that constitutes how disciplinary perspectives become meaningful based on educational developers’ position in relation to different disciplinary spaces. My findings highlight the situative relationships between the individual educational developer, their practice with disciplinary perspectives, and their work tasks in educational development. As educational developers continue to develop their practice to advance teaching and learning in higher education, this research contributes to the professional knowledge of educational developers in support of interdisciplinary collaboration. </div><div><br></div>

The duration of exile in refugee communities has grown immensely over the last two decades. Recent humanitarian reports have called for actors to create more coordinated global support for the refugee crises. In these recent calls, the desire to break a cycle of dependency between the refugee community and international aid has been a clear priority. Hence, education has emerged as a strategic action to foster refugee self-reliance, particularly higher education (HE) and technical and vocational education and training (TVET). There are many opportunities to use HE and TVET to benefit the refugee community, including: developing solutions to improve living conditions, enabling new opportunities for learning pathways, allowing refugees to contribute to the economy in hosting countries, or preparing them to rebuild their lives once they return to their home countries. However, the economic, political, and cultural complexities of refugee communities often add layers of challenges to typical formal HE and TVET programs. In addition, the existing literature in refugee education still lacks a coherent analysis of these factors and conditions for adoption of HE and TVET programs, especially for refugees living in camps. <div>To address these gaps, this dissertation presents three studies that investigate an undergraduate introductory engineering course for refugees called Localized Engineering in Displacement (LED). Specifically, I draw on effective learning and policy frameworks to understand how to situate engineering education across HE and TVET and advance LED in refugee camps. The first study presents a case study examining the iterative processes of creation and implementation of the LED course in the Azraq refugee camp in Jordan. As a general outcome of my study, I describe the novel approach to teaching engineering design for learners in the Azraq refugee camp and its applications to other contexts. The second study examines the LED course implemented in the Kakuma refugee camp. The Kakuma refugee camp is situated in Kenya and considered the largest refugee camp in the world, thus providing a different context of refugee camps. I discuss the contextual challenges to transfer, develop, and implement to a new context and present the course outcomes and experiences based on the course participants’ reflections. The third study extends findings from the first and second studies by using a comparative case study to critically examine the development process and challenges of engineering education in refugee camps. Central to my analysis is the connection between the challenges identified in both camps and existing actors involved with refugee education. </div><div>My research uses two case studies to underscore the complexity of the LED course development in the Azraq and Kakuma camps. I seek to foster a debate about the challenges that influence the development of higher engineering education programs in refugee camps and how different actors can collaborate to advance high-quality engineering education initiatives in refugee contexts. Overall, this dissertation clarifies some of the biggest challenges to implement engineering education in refugee settings, how different actors can collaborate to mitigate these challenges, and how these findings expose the misalignment between the international rhetoric and reality on the ground in refugee camps.</div>

<p>  </p> <p>Clustering, a prevalent class of machine learning (ML) algorithms used in data mining and pattern-finding—has increasingly helped engineering education researchers and educators see and understand assessment patterns at scale. However, a challenge remains to make ML-enabled educational inferences that are useful and reliable for research or instruction, especially if those inferences influence pedagogical decisions or student outcomes. ML offers an opportunity to better personalizing learners’ experiences using those inferences, even within large engineering classrooms. However, neglecting to verify the trustworthiness of ML-derived inferences can have wide-ranging negative impacts on the lives of learners. </p> <p><br></p> <p>This study investigated what student clusters exist within the standard operational data of a large first-year engineering course (>1500 students). This course focuses on computational thinking skills for engineering design. The clustering data set included approximately 500,000 assessment data points using a consistent five-scale criterion-based grading framework. Two clustering techniques—N-TARP profiling and K-means clustering—examined criterion-based assessment data and identified student cluster sets. N-TARP profiling is an expansion of the N-TARP binary clustering method. N-TARP is well suited to this course’s assessment data because of the large and potentially high-dimensional nature of the data set. K-means clustering is one of the oldest and most widely used clustering methods in educational research, making it a good candidate for comparison. After finding clusters, their interpretability and trustworthiness were determined. The following research questions provided the structure for this study: RQ1 – What student clusters do N-TARP profiling and K-means clustering identify when applied to structured assessment data from a large engineering course? RQ2 – What are the characteristics of an average student in each cluster? and How well does the average student in each cluster represent the students of that cluster? And RQ3 – What are the strengths and limitations of using N-TARP and K-means clustering techniques with large, highly structured engineering course assessment data?</p> <p><br></p> <p>Although both K-means clustering and N-TARP profiling did identify potential student clusters, the clusters of neither method were verifiable or replicable. Such dubious results suggest that a better interpretation is that all student performance data from this course exist in a single homogeneous cluster. This study further demonstrated the utility and precision of N-TARP’s warning that the clustering results within this educational data set were not trustworthy (by using its W value). Providing this warning is rare among the thousands of available clustering methods; most clustering methods (including K-means) will return clusters regardless. When a clustering algorithm identifies false clusters that lack meaningful separation or differences, incorrect or harmful educational inferences can result. </p>

No description available.

A recurring question in science and engineering education is why the students do not link knowledge from theoretical classes to the real world met in laboratory courses. Mathematical models and visualisations are widely used in engineering and engineering education. Very often it is assumed that the students are familiar with the mathematical concepts used. These may be concepts taught in high school or at university level. One problem, though, is that many students have never or seldom applied their mathematical skills in other subjects, and it may be difficult for them to use their skills in a new context. Some concepts also seem to be "too difficult" to understand. One of these mathematical tools is to use Laplace Transforms to solve differential equations, and to use the derived functions to visualise transient responses in electric circuits, or control engineering. In many engineering programs at college level the application of the Laplace Transform is considered too difficult for the students to understand, but is it really, or does it depend on the teaching methods used? When applying mathematical concepts during lab work, and not teaching the mathematics and practical work in different sessions, and also using examples varied in a very systematic way, our research shows that the students approach the problem in a very different way. It shows that by developing tasks consequently according to the Theory of Variation, it is not impossible to apply the Laplace Transform already in the first year of an engineering program. The original aim of this thesis was to show: how students work with lab-tasks, especially concerning the goal to link theory to the real world how it is possible to change the ways students approach the task and thus their learning, by systematic changes in the lab-instructions During the spring 2002 students were video-recorded while working with labs in Electric Circuits. Their activity was analysed. Special focus was on what questions the students raised, and in what ways these questions were answered, and in what ways the answers were used in the further activities. This work informed the model ”learning of a complex concept”, which was used as well to analyse what students do during lab-work, and what teachers intend their students to learn. The model made it possible to see what changes in the lab-instructions that would facilitate students learning of the whole, to link theoretical models to the real world, through the labactivities. The aim of the thesis has thus become to develop a model: The learning of a complex concept show how this model can be used as well for analysis of the intended object of learning as students activities during lab-work, and thus the lived object of learning use the model in analysis of what changes in instruction that are critical for student learning. The model was used to change the instructions. The teacher interventions were included into the instructions in a systematic way, according to as well what questions that were raised by the students, as what questions that were not noticed, but expected by the teachers, as a means to form relations between theoretical aspects and measurement results. Also, problem solving sessions have been integrated into the lab sessions. Video recordings were also conducted during the spring 2003, when the new instructions were used. The students' activities were again analysed. A special focus of the thesis concerns the differences between the results from 2002 and 2003. The results are presented in four sections: Analysis of the students' questions and the teachers' answers during the lab-course 2002 Analysis of the links students need to make, the critical links for learning Analysis of the task structure before and after changes Analysis of the students' activities during the new course The thesis ends with a discussion of the conclusions which may be drawn about the possibilities to model and develop teaching sequences through research, especially concerning the aim to link theoretical models to the real world. / En stående fråga som lärare i naturvetenskapliga och tekniska utbildningar ställer är varför elever och studenter inte kopplar samman kunskaper från teoretiska kursmoment med den verklighet som möts vid laborationerna. Ett vanligt syfte med laborationer är att åstadkomma länkar mellan teori och verklighet, men dessa uteblir ofta. Många gånger används avancerade matematiska modeller och grafiska representationer, vilka studenterna lärt sig i tidigare kurser, men de har sällan eller aldrig tillämpat dessa kunskaper i andra ämnen. En av dessa matematiska hjälpmedel är Laplacetransformen, som främst används för att lösa differentialekvationer, och åskådliggöra transienta förlopp i ellära eller reglerteknik. På många universitet anses Laplacetransformen numera för svår för studenterna på kortare ingenjörsutbildningar, och kurser eller kursmoment som kräver denna har strukits ut utbildningsplanerna. Men, är det för svårt, eller beror det bara på hur man presenterar Laplacetransformen? Genom att låta studenterna arbeta parallellt med matematiken och de laborativa momenten, under kombinerade lab-lektionspass, och inte vid separata lektioner och laborationer, samt genom att variera övningsexemplen på ett mycket systematiskt sätt, enligt variationsteorin, visar vår forskning att studenterna arbetar med uppgifterna på ett helt annat sätt än tidigare. Det visar sig inte längre vara omöjligt att tillämpa Laplacetransformen redan under första året på civilingenjörsutbildning inom elektroteknik. Ursprungliga syftet med avhandlingen var att visa hur studenter arbetar med laborationsuppgifter, speciellt i relation till målet att länka samman teori och verklighet hur man kan förändra studenternas aktivitet, och därmed studenternas lärande, genom att förändra laborationsinstruktionen på ett systematiskt sätt. Under våren 2002 videofilmades studenter som utförde laborationer i en kurs i elkretsteori. Deras aktivitet analyserades. Speciellt studerades vilka frågor studenterna ställde till lärarna, på vilket sätt dessa frågor besvarades, och på vilket sätt svaren användes i den fortsatta aktiviteten. Detta ledde fram till en modell för lärande av sammansatta begrepp, som kunde användas både för att analysera vad studenterna gör och vad lärarna förväntar sig att studenterna ska lära sig. Med hjälp av modellen blev det då möjligt att se vad som behövde ändra i instruktionerna för att studenterna lättare skulle kunna utföra de aktiviteter som krävs för att länka teori och verklighet. Syftet med avhandlingen är därmed att ta fram en modell för lärande av ett sammansatt begrepp visa hur denna modell kan användas för såväl analys av önskat lärandeobjekt, som av studenternas aktivitet under laborationer, och därmed det upplevda lärandeobjektet använda modellen för att analysera vilka förändringar som är kritiska för  studenters lärande. Modellen användes för att förändra laborationsinstruktionerna. Lärarinterventionerna inkluderades i instruktionerna på ett systematiskt sätt utifrån dels vilka frågor som ställdes av studenterna, dels vilka frågor studenterna inte noterade, men som lärarna velat att studenterna skulle använda för att skapa relationer framför allt mellan teoretiska aspekter och mätresultat. Dessutom integrerades räkneövningar och laborationer. Videoinspelningar utfördes även våren 2003, då de nya instruktionerna användes. Även dessa analyserades med avseende på studenternas aktiviteter. Skillnader mellan resultaten från 2002 och 2003 står i fokus. Avhandlingens resultatdel består av: Analys av studenternas frågor och lärarnas svar under labkursen 2002 Analys av de länkar studenterna behöver skapa för att lära Analys av laborationsinstruktionerna före och efter förändringarna Analys av den laborationsaktivitet som blev resultatet av de nya instruktionerna, och vilket lärande som då blev möjligt Avhandlingen avlutas med en diskussion om de slutsatser som kan dras angående möjligheter att via forskning utveckla modeller av undervisningssekvenser för lärande där målet är att länka samman teori och verklighet

<p>This thesis is an article-based (3-paper format) dissertation. In the first article, the research team adapted an input-process-outcome (IPO) model of group-level processes in the classroom, as a theoretical framework, to examine students’ experiences regarding pre-college engineering curricula, classroom environments, and their experiences with the creative process in the two engineering courses offered in a university-based summer enrichment program. Applying provisional and open coding to semi-structured interview data from 16 participants, an Input-Process-Outcome Model of Collaborative Creativity (IPOCC model) was developed. In this study, I grouped our findings under Inputs, Group Processes, Outcomes, and Mediating Factors. The IPOCC model expands the 4P model of creativity to incorporate more collaborative contexts. According to the 4P model, creativity can be viewed from four different perspectives: Person, Process, Product, and Press. The IPOCC model suggests that in K-12 collaborative practice, creativity involves group-level considerations in addition to individual-level components. The IPOCC model offer insights for educators in terms of input components, group processes, and mediating factors that can facilitate learners’ engagement in creative teamwork. Findings of this study indicated that a combination of challenging tasks, open-ended problems, and student teamwork provides a rich environment for learners’ engagement to think creatively.</p> <p>The purpose of the second study was to systematically investigate how novice/K-12 students’ visual representation of design ideas has been operationalized, measured, or assessed in the research literature. In the different phases of screening in this systematic review, inclusion, exclusion, and quality criteria were applied. From an initial sample of 958 articles, 40 studies were included in the final step of the coding process and qualitative synthesis. Applying provisional and open coding, three broad themes, and 23 characteristics were identified that have been used by researchers to conceptualize sketching of ideas, in novice/K-12 design activities: Communicating Ideas, Visual-Spatial Characteristics, and Design Creativity. We propose this Three-pronged Design Sketching (3-pDS) framework to examine K-12 design sketches. </p> In K-12 settings, one major challenge of conducting research on the influence of engineering education programs and curricula involves assessment. There is a need for developing alternative, effective, and reliable assessment measures to evaluate students’ design activities. The third study aimed to address this need by developing the idea-Sketching Early Engineering Design (i-SEED) Scale to assess pre-college learners’ freehand sketches in response to a design task. Applying the Three-pronged Design Sketching (3-pDS) as a theoretical framework, the purpose of this study was to examine evidence of content validity, construct validity, and internal consistency of the i-SEED Scale data. The data collection took place in a residential summer enrichment program for students with gifts and talents at a Midwestern university. Following different stages of scale-development design, a sample of 113 design sketches were scored in this study, and the scores were used to provide evidence of the validity of the data for the i-SEED Scale. The sketches were generated by 120 middle- and high-school students in a collaborative design-oriented course. Exploratory factor analysis results supported a three-factor model for the i-SEED Scale, including Visual-Spatial Characteristics, Design Creativity, and Communicating Ideas.

Grades serve as one of the primary indicators of student learning, directing subsequent actions for students, instructors, and administrators, alike. Therefore, grade validity—that is, the extent to which grades communicate a meaningful and credible representation of what they purport to measure—is of utmost importance. However, a grade cannot be valid if one cannot trust that it will consistently and reliably result in the same value, regardless of who makes a measure or when they make it. Unfortunately, such reliability becomes increasingly challenging to achieve with larger class sizes, especially when utilizing multiple evaluators, as is often the case with mandatory introductory courses at large universities. Reliability suffers further when evaluating open-ended tasks, as are prevalent in authentic, high-quality engineering coursework.<div><br></div><div>This study explores grading reliability in the context of a large, multi-section engineering course. Recognizing the number of people involved and the plethora of activities that affect grading outcomes, the study adopts a systems approach to conduct a human reliability analysis using the Functional Resonance Analysis Method. Through this method, a collection of data sources, including course materials and observational interviews with undergraduate teaching assistant graders, are synthesized to produce a general model for how actions vary and affect subsequent actions within the system under study. Using a course assignment and student responses, the model shows how differences in contextual variables affect expected actions within the system. Next, the model is applied to each of the observational interviews with undergraduate teaching assistants to demonstrate how these actions occur in practice and to compare graders to one another and with expected behaviors. These results are further related to the agreement in system outcomes, or grades, assigned by each grader to guide analysis of how actions within the system affect its outcome.<br></div><div><br></div><div>The results of this study connect and elaborate upon previous models of grader cognition by analyzing the phenomenon in engineering, a previously unexplored context. The model presented can be easily generalized and adapted to smaller systems with fewer actors to understand sources of variability and potential threats to outcome reliability. The analysis of observed outcome instantiations guides a set of recommendations for minimizing grading variability.<br></div>