Cognitive Work Analysis (CWA) is a framework for the analysis of complex systems that involve technological tools and human operators who must operate the systems in sometimes-unexpected situations. CWA consists of five phases that cover analysis of ecological aspects of systems, such as the tools in the system and the environment, and cognitive aspects, related to the human operators in the systems. Human operators often need to be trained to use complex systems, and training research has been conducted related to the ecological aspects of CWA; however, there is a gap in the training research related to the cognitive aspects of CWA. The Skills-Rules-Models (SRM) framework is currently the main method of cognitive analysis for CWA; therefore, to begin to fill the gap in the training literature at the cognitive end of CWA, this dissertation examines the need for training as related to SRM. In this dissertation, the current research related to training and CWA is reviewed and literature on the nature of expertise as related to SRM is examined. From this review, the need for training rules that can be used in unexpected situations, as a means of reducing cognitive demand, is identified. In order to assist operators with developing the knowledge to use such rule-based behaviour, training needs and methods to meet those needs must be identified. The reuse of knowledge in new situations is the essence of training transfer, so ideas from training transfer are used to guide the development of three guidelines for determining rules that might be transferable to new situations. Then methods of developing rules that fit those three guidelines by using information from the Work Domain Analysis (WDA) and Control Task Analysis (CTA) phases of CWA are presented. In addition to methods for identifying training needs, methods for meeting those training needs are required. The review of training transfer also identified the need to use contextual examples in training while still avoiding examples that place too high of a cognitive demand on the operator, possibly reducing learning. Therefore, as a basis for designing training material that imposes a reduced cognitive demand, Cognitive Load Theory (CLT) is reviewed, and methods for reducing cognitive demand are discussed. To demonstrate the methods of identifying and meeting training needs, two examples are presented. First, two sets of training rules are created based on the results of the application of the WDA and CTA phases of CWA for two different work domains: Computer Algebra Systems and the programming language Logo. Then, from these sets of rules, instructional materials are developed using methods based on CLT to manage the cognitive demands of the instructions. Finally, two experiments are presented that test how well operators learn from the instructional materials. The experiments provide support for the effectiveness of applying CLT to the design of instructional materials based on the sets of rules developed. This combined work represents a new framework for identifying and meeting training needs related to the cognitive aspects of CWA.

This Master of Science thesis is producedat SAAB Security and Defence Solutions.The purpose of the thesis is to analyzehow air surveillance work can be carriedout. This information is then used to givesuggestions for the design of a new systemcontaining only the most essentialfunctionality. This is done by examiningthe available frameworks which can informinterface design and applying a frameworkto analyze work in a complete system usedas the basis of the new Compact C2 system.The second part of the analysis isdirected towards the stripped system(Compact C2) and both parts of theanalysis are used to inform interfacedesign of the Compact C2 system. By usingthe full range of the chosen framework foranalysis of the identification process inSwedish air surveillance work, someessential functions were identified andshould also have support in a Compact C2 system.

Cognitive Work Analysis (CWA) is a framework for the analysis of complex systems that involve technological tools and human operators who must operate the systems in sometimes-unexpected situations. CWA consists of five phases that cover analysis of ecological aspects of systems, such as the tools in the system and the environment, and cognitive aspects, related to the human operators in the systems. Human operators often need to be trained to use complex systems, and training research has been conducted related to the ecological aspects of CWA; however, there is a gap in the training research related to the cognitive aspects of CWA. The Skills-Rules-Models (SRM) framework is currently the main method of cognitive analysis for CWA; therefore, to begin to fill the gap in the training literature at the cognitive end of CWA, this dissertation examines the need for training as related to SRM. In this dissertation, the current research related to training and CWA is reviewed and literature on the nature of expertise as related to SRM is examined. From this review, the need for training rules that can be used in unexpected situations, as a means of reducing cognitive demand, is identified. In order to assist operators with developing the knowledge to use such rule-based behaviour, training needs and methods to meet those needs must be identified. The reuse of knowledge in new situations is the essence of training transfer, so ideas from training transfer are used to guide the development of three guidelines for determining rules that might be transferable to new situations. Then methods of developing rules that fit those three guidelines by using information from the Work Domain Analysis (WDA) and Control Task Analysis (CTA) phases of CWA are presented. In addition to methods for identifying training needs, methods for meeting those training needs are required. The review of training transfer also identified the need to use contextual examples in training while still avoiding examples that place too high of a cognitive demand on the operator, possibly reducing learning. Therefore, as a basis for designing training material that imposes a reduced cognitive demand, Cognitive Load Theory (CLT) is reviewed, and methods for reducing cognitive demand are discussed. To demonstrate the methods of identifying and meeting training needs, two examples are presented. First, two sets of training rules are created based on the results of the application of the WDA and CTA phases of CWA for two different work domains: Computer Algebra Systems and the programming language Logo. Then, from these sets of rules, instructional materials are developed using methods based on CLT to manage the cognitive demands of the instructions. Finally, two experiments are presented that test how well operators learn from the instructional materials. The experiments provide support for the effectiveness of applying CLT to the design of instructional materials based on the sets of rules developed. This combined work represents a new framework for identifying and meeting training needs related to the cognitive aspects of CWA.

'Complex sociotechnical systems' are systems made up of numerous interacting parts, both human and non-human, operating in dynamic, ambiguous and safety critical domains. The system design, and representation, has significant implications for the usability of the system, its performance, errors and reliability. Due to its formative constraint-based approach, Cognitive Work Analysis (CWA) has been frequently proposed as a framework for modelling these complex systems, further it is contended that the approach can be extended to design for interaction without significant deviation from the accepted framework. This thesis presents a number of applications in complex military domains to explore and develop the benefits of CWA. Unlike much of the previous literature, particular attention is placed on exploring the CWA framework in its entirety. This holistic approach focuses on the system environment, the activity that takes place within it, the strategies used to conduct this activity, the way in which the constituent parts of the system (both human and non-human) interact and the behaviour required. Each stage of this analysis identifies the constraints governing the system; it is contended that through this holistic understanding of constraints, recommendations can be made for the design of system interaction; increasing the ability of users to cope with unanticipated, unexpected situations. This thesis discusses the applicability of the approach in system analysis, development and evaluation. It provides process to what was previously a loosely defined framework.

This research examines on the importance of considering context as design criteria for technology integration and technology design, and in particular, what that means for teacher education. While others (Fishman, Marx, Blumenfeld, Krajcik, & Soloway, 2004) have broadly considered context as a factor in supporting technology integration, this current research uses a Human-Tech framework (Vicente, 2003) to identify and examine the implications of individual contextual constraints for the design of technology-enhanced learning; therefore, the researcher begins to answer the important - and arguably unexplored - question of how the complexities of varying contexts ought to be used to inform design. Broadly, this design research study explores the impact of using a Human-Tech framework for understanding technology integration in education and specifically, to inform the design of technology-enhanced learning practices in the context of teacher education. Further, the research reports on the impact of an open online research support forum - designed using a Human-Tech framework - on the experiences of teacher candidates while carrying out classroom-based research as part of their initial teacher education program. Overall, results of the research study suggest that using a Human-Tech framework for understanding technology integration was helpful in supporting a broader and more systematic approach to designing for more effective use of technology in the context of teacher education. In terms of the design of the open online research support forum, findings suggest that there were key Organizational constraints that likely continued to have a limiting impact on the innovations across the two-year design study. Therefore, this research also points to future technology-specific and non technology-specific design strategies which may have implications for technology integration, and fulfilling the functional purposes of the program more broadly.

Cognitive Work analysis (CWA) as an analytical approach for examining complex socio-technical systems has shown success in modeling the work of single operators. The CWA approach allows room for social and team interactions, but a more explicit analysis of team aspects can reveal more information for systems design. CWA techniques and models do not yet provide sufficient guidance on identifying shared constraints, team strategies, or social competencies of team players. In this thesis, I explore whether a team approach to CWA can yield more information than a typical CWA. Team CWA techniques and models emerge and extend from theories and models of teamwork, past attempts to model teams with CWA, and the results of two sets of observational studies. The potential benefits of using Team CWA models in domains with strong team collaboration are demonstrated through the results of a two-week observation at the Labour and Delivery Department of The Ottawa Hospital and a fifteen-week observation at the IBM Ottawa Software Group.

This research examines on the importance of considering context as design criteria for technology integration and technology design, and in particular, what that means for teacher education. While others (Fishman, Marx, Blumenfeld, Krajcik, & Soloway, 2004) have broadly considered context as a factor in supporting technology integration, this current research uses a Human-Tech framework (Vicente, 2003) to identify and examine the implications of individual contextual constraints for the design of technology-enhanced learning; therefore, the researcher begins to answer the important - and arguably unexplored - question of how the complexities of varying contexts ought to be used to inform design. Broadly, this design research study explores the impact of using a Human-Tech framework for understanding technology integration in education and specifically, to inform the design of technology-enhanced learning practices in the context of teacher education. Further, the research reports on the impact of an open online research support forum - designed using a Human-Tech framework - on the experiences of teacher candidates while carrying out classroom-based research as part of their initial teacher education program. Overall, results of the research study suggest that using a Human-Tech framework for understanding technology integration was helpful in supporting a broader and more systematic approach to designing for more effective use of technology in the context of teacher education. In terms of the design of the open online research support forum, findings suggest that there were key Organizational constraints that likely continued to have a limiting impact on the innovations across the two-year design study. Therefore, this research also points to future technology-specific and non technology-specific design strategies which may have implications for technology integration, and fulfilling the functional purposes of the program more broadly.

New social networking and social web tools are becoming available and are easing the process of customizing online social environments. With these developments in technology, core design efforts are being extended beyond usability for individual users and beginning to include notions of sociability for the engagement of communities of users. This thesis is an investigation of these developments. It is guided by the principal research question: how do you design for social engagement in an online social environment intended to facilitate interaction in a community of users? To address this question, this thesis presents a domain-community model developed from the communities of practice concept and the Work Domain Analysis model used in Cognitive Work Analysis. The domain-community model provides a basis for the design a composition of web components for an online social environment that will addresses issues of social engagement and domain effectiveness. In a case study, the domain-community model was used as a basis for the redesign of a social networking portal used by an international development leadership community called UCP-SARnet. A social network analysis of core members of UCP-SARnet was conducted before and after the portal was redesigned. From the social network analysis, it was concluded that the structure of UCP-SARnet was positively affected by the redesign: core group members reported they knew one another significantly more after the redesign of the website than before the redesign. User experience measures of the UCP-SARnet portal, website usage data, and a tally of website communication activity also changed significantly with the redesign of the website. This provided more evidence that a design informed by Cognitive Work Analysis and communities of practice produced a measurable effect on the structure of the UCP-SARnet online community. As such, this model can provide a basis for designers of online communities to more systematically account for social phenomena in relation to collective efforts in a given work domain. Furthermore, it is expected the effectiveness of the model can be taken forward with future work by refining the domain-community model, developing techniques to translate the model into interface concepts, and building practices for community-based research and design.

This work presents the application of a cognitive engineering design method to the design of operational procedures and ground control station interfaces for uninhabited aerial vehicles (UAVs). Designing for UAV systems presents novel challenges, both in terms of selecting and presenting adequate information for effective teleoperation, and in creating operational procedures and ground control station interfaces that are robust to a range of UAV platforms and missions. Creating a coherent set of operating procedures, automatic functions and operator interfaces requires a systematic design approach that considers the system and the mission at different levels of abstraction and integrates the different element of the system. Several models are developed through the application of this cognitive engineering method. An analysis of the work of operating a UAV creates an abstraction decomposition space (ADS) model. The ADS helps identify the control tasks needed to operate the system. A strategies analysis then identifies methods for implementing these control tasks. The distribution of activities and roles between the human and automated components in the system is then considered in a social organization and cooperation analysis. These insights are applied to the design of coherent sets of operational procedures, ground control station interfaces and automatic functions for a specific UAV in support of a continuous target surveillance (CTS) mission. The importance of the coherence provided by the selected design method in the design of UAV operational procedures and ground control station interfaces is analyzed through a human in the loop simulation experiment for this mission. The results of the simulation experiment indicate that UAV controllers using coherently designed elements achieve significantly higher mission performance and experience lower workloads than those that when using incoherently matched elements.

Field managers have a complex job that requires coordinating a large number of variables that affect construction work. This makes their job primarily abstract, which complicates the observation of their activities; in turn, their knowledge is usually organized in a way that does not adequately consider the relationships between these work variables. As such, practical approaches to aid field managers’ work have often neglected these relationships and have been insufficient to aid practitioners’ coordination tasks. For example, many information tools have been developed to support decisions in the field, but their tendency to separate work activities makes it complicated to integrate the tools into work. Similarly, education programs often focus on individual concepts and tasks, which do not match the practical organization of work activities in the field. This dissertation presents the development of comprehensive cognitive models of field managers to gain insight into the approaches of expert practitioners to address the relationships between the major variables of work. Using the Applied Cognitive Work Analysis (ACWA) technique, it is possible to model work in terms of goals, decisions and information requirements of the job. These components are utilized to explain how the variables of work relate to each other and how they contribute towards attaining the objectives and responsibilities of the job. In particular, the inclusion of activities and tools as part of this set of variables gives insight as to how goals are attained through such means. The cognitive model obtained is applied to, first, analyze the integration of information tools into field managers’ work and, second, analyze the utilization of cognitive models to develop instructional programs and tools. As such, this dissertation makes three main contributions. The first contribution is the development of thorough cognitive models of field managers’ work. The second contribution is the use of these models to consider cognitive and behavioral aspects of adoption of information tools in the field. And the third contribution is the use of these models to present field management learners with the opportunity to use the knowledge and approaches of field managers in a robust learning environment. / text