The Vessel Traffic Service (VTS) is a complex system tasked with ensuring the safety of navigation within specified areas known as VTS areas. Earlier research in the domain has often focused on the decision support systems and other tools employed by the VTS operators to provide the vessels in the area with VTS services. Consequently, less effort has gone into looking at the system itself and the human factors aspects of the system. This study uses the Functional Resonance Analysis Method (FRAM) to create a functional model of the VTS. It looks at how a VTS works, what the different components are and how these components are related. The main purpose of the FRAM model is to serve as a basis for future application by identifying the functions that constitute the system, and to illuminate the potential variability therein. To demonstrate how it might be used, an instantiation of an observed scenario will be presented. A structural description of the VTS is also presented, which aims to serve as an introduction to the domain for readers who are previously unfamiliar with it. The functional model shows that a lot of the potential variability seems to lie in the functions that rely heavily on human interaction, which is to be expected, as human performance is highly variable. It also shows that the availability and reliability of relevant information is crucial in order to be able to provide the VTS services, and if the information for some reason is unavailable or insufficient it seems likely to cause variability.

Grades serve as one of the primary indicators of student learning, directing subsequent actions for students, instructors, and administrators, alike. Therefore, grade validity—that is, the extent to which grades communicate a meaningful and credible representation of what they purport to measure—is of utmost importance. However, a grade cannot be valid if one cannot trust that it will consistently and reliably result in the same value, regardless of who makes a measure or when they make it. Unfortunately, such reliability becomes increasingly challenging to achieve with larger class sizes, especially when utilizing multiple evaluators, as is often the case with mandatory introductory courses at large universities. Reliability suffers further when evaluating open-ended tasks, as are prevalent in authentic, high-quality engineering coursework.<div><br></div><div>This study explores grading reliability in the context of a large, multi-section engineering course. Recognizing the number of people involved and the plethora of activities that affect grading outcomes, the study adopts a systems approach to conduct a human reliability analysis using the Functional Resonance Analysis Method. Through this method, a collection of data sources, including course materials and observational interviews with undergraduate teaching assistant graders, are synthesized to produce a general model for how actions vary and affect subsequent actions within the system under study. Using a course assignment and student responses, the model shows how differences in contextual variables affect expected actions within the system. Next, the model is applied to each of the observational interviews with undergraduate teaching assistants to demonstrate how these actions occur in practice and to compare graders to one another and with expected behaviors. These results are further related to the agreement in system outcomes, or grades, assigned by each grader to guide analysis of how actions within the system affect its outcome.<br></div><div><br></div><div>The results of this study connect and elaborate upon previous models of grader cognition by analyzing the phenomenon in engineering, a previously unexplored context. The model presented can be easily generalized and adapted to smaller systems with fewer actors to understand sources of variability and potential threats to outcome reliability. The analysis of observed outcome instantiations guides a set of recommendations for minimizing grading variability.<br></div>