Performance and efficacy are reciprocally causal; however, the effect of performance on subsequent perceptions of efficacy has received little attention, especially in the context of team training. In addition, the moderating effect of feedback accuracy on the relationship between team performance and team-efficacy is largely unexplored. As such, the objective of the present study was to investigate the relationship between team performance and team-efficacy in the context of after-action reviews (AARs). Specifically, this study examined the conjoint influence of (a) the accuracy of performance feedback available to trainees during AARs, and (b) time on the predictive validity of team performance on team-efficacy. Data were obtained from 492 undergraduate students assigned to 123 teams in a 5 hr team training protocol using a 3 (training condition: non-AAR, versus subjective AAR, versus objective AAR) x 3 (sessions) repeated measures design. Contrary to the first set of hypotheses, the positive relationship between performance and efficacy was strongest for teams trained without AARs and weakest for teams trained using subjective AARs. Although team-efficacy was predicted more strongly by more proximal team performance than by more distal team performance, this pattern of results was found only for teams trained either without AARs or with objective AARs. The predictive validity of performance on efficacy decreased as performance episodes became more proximal among teams trained using subjective AARs. Finally, within-team agreement of team-efficacy ratings decreased over time for teams that engaged in AARs and remained constant over time for teams that did not engage in AARs. The theoretical and practical implications of these findings are discussed. It is anticipated that this research will provide insight into the roles of feedback accuracy and time in the performance-efficacy relationship and provide guidance to researchers and practitioners in effectively integrating AAR design characteristics into team training environments.

The team-training literature provides favorable support for the after-action review (AAR)?s ability to improve cognitive, skill, and attitudinal outcomes in co-located and distributed environments. However, the comparative effectiveness of co-located and distributed AARs is unknown. Thus, the objective of the present study was to investigate the comparative effectiveness of co-located and distributed AARs. The present study examined the AAR?s effect on performance, declarative knowledge, team-efficacy, team voice, team cohesion, and team-level reactions. Data were obtained from 492 undergraduate students (47.66% female) assigned to 123 4-person teams who participated in a team training protocol using a 3 (type of AAR review: non-AAR versus subjective AAR versus objective AAR) x 2 (geographic dispersion: co-located and distributed training environments) x 3 (sessions) repeated measures design. The results indicate that AAR teams had significantly higher performance scores than the non-AAR teams. In addition, the AAR teams had higher perceptions of team-efficacy and higher levels of team cohesion than the non-AAR teams. With the exception of team-level reactions, there were no other significant differences between the distributed AAR and co-located AAR conditions. Similarly, there were no significant differences across any of the outcome variables between the objective and subjective AAR conditions, indicating that the type of AAR did not impact the results of the training. The findings of the present study highlight several practical and scientific implications that should be considered regarding AAR training. Primarily, regardless of the training environment or type of AAR, AAR training remains an effective intervention at increasing performance and attitudinal-based outcomes. In addition, the results suggest that the use of distributed AARs does not engender the proposed process losses that were hypothesized. Thus, the use of this training to reduce administrative costs may be a viable option for geographically dispersed organizations. Finally, practitioners should evaluate the extent to which increasing the amount of technology to allow for a more objective performance review, is providing the intended benefit to the trainees. The empirical research has consistently demonstrated that the use of objective review systems provides little to no benefit to the trainees. Future research is needed to determine the generalizability of these findings to other tasks, domains, team types, and levels of expertise.

The United States Army currently uses after action reviews (AARs) to give personnel feedback on their performance. However, due to the growing use of geographically distributed teams, the traditional AAR, with participants and a moderator in the same room, is becoming difficult; therefore, distributed AARs are becoming a necessity. However, distributed AARs have not been thoroughly researched. To determine what type of distributed AARs would best facilitate team training in distributed Army operations, feedback media platforms must be compared. This research compared three types of AARs, which are no AAR, teleconference AAR, and teleconference AAR with visual feedback, to determine if there are learning differences among these conditions. Participants completed three search missions and received feedback between missions from one of these conditions. Multiple ANOVAs were conducted to compare these conditions and trials. Results showed that overall the teleconference AAR with visual feedback improved performance the most. A baseline, or no AAR, resulted in the second highest improvement, and the teleconference condition resulted in the worst overall performance. This study has implications for distributed military training and feedback, as well as other domains that use distributed training and feedback.

The after–action review (AAR; also known as the after–event review or debriefing) is a training approach that is based on reviews of trainees’ performance on recently completed tasks or performance events. Used by the military for decades, the use of AAR–based training has increased dramatically in recent years. Empirical research investigating AARs, however, is almost non–existent, and theoretical work on the effectiveness AAR–based training and the underlying processes have been limited. The present study presents a theoretical framework for the AAR by integrating the AAR into the existing training literature. In addition, this study presents an empirical evaluation of the effectiveness of AAR–based training, and an investigation of whether objective AAR–based training is more effective than subjective AAR–based training. One–hundred twenty individuals were trained in 30 4–person teams on a cognitively complex performance task. Teams were trained using a non–AAR–, subjective AAR–, or objective AAR–based training approach. Declarative knowledge, team performance, and team–efficacy served as the measures of training effectiveness. It was hypothesized that AAR–based training (subjective AAR– and objective AAR– based training combined) would be more effective than non–AAR–based training. Further, it was hypothesized that objective AAR–based training would be more effective than subjective AAR–based training. The study results indicated that AAR–based training was more effective than the non–AAR–based training approach in terms of team performance and team–efficacy, but not team declarative knowledge. Objective AAR–based training was no more effective than subjective AAR–based training. Teams performed equally well on the training outcome measures regardless of whether they used an objective or subjective AAR– based training approach. It is anticipated that the theoretical framework and empirical results of this study will serve as a catalyst for the integration of AAR–based training into existing training literatures and to inform the design and practice of AAR–based training systems to take full advantage of their efficacy as training interventions.

This thesis will propose a system for feedback and after action review in serious games. The proposed system, called serious achievements, is built on common scoring systems in commercial games for entertainment. A number of games for training is evaluated to give an overview of how feedback is carried out in existing products and a literature review is made to gather background material to motivate and explain the serious achievement system. Guidelines are thereafter formed focusing on how to create serious achievements for a serious game for education by using the course goals from the course’s curriculum. A case where the game On Scene Commander is equipped with serious achievements from the course goals of the course Incident Commander Basic Level is made and evaluated by rescue training instructors. Conclusions are that the course goals often needs to be more specific and the easiest course goals to implement as serious achievements are the ones that is easily measurable.

This thesis aims to explore the challenges of translation of after-action reviews (AARs) from theory and best practices into real-life. Furthermore, the purpose was to explore if existing theories can be translated to another context and how. This is examined on the case of extreme action teams, more concretely firefighters in Sweden. In extreme action teams there is a high importance on learning and improvement. After-action reviews as a learning tool can be a very suitable approach to foster constant development and enhance extreme action teams’ resilience and effectiveness.  From the literature findings it is apparent that there are several frameworks for after-action reviews. However, these are mainly based on a specific organizational context, namely the U.S. Army’s. Practitioners and researchers should be aware of the fact that this process has been developed with the military in mind, a fact which may adversely affect the implementation in organizations with a different background. Translating an after-action review from one specific context to another and more generally from theory to practice comes with challenges that will be explored in this study.  This thesis was conducted using qualitative research to examine how after-action reviews are conducted in real-life. Data were mainly collected through semi-structured interviews with firefighters and team leaders from different teams in the case organization. The interviews helped us to understand how the after-action reviews were perceived by the interviewees. Based on the literature best practices for the implementation and conduction of after-action reviews were identified which were compared to the empirical results.  We identified two main challenges of translating after-action reviews which are that (1) it is not in all contexts possible to strictly follow the theory and (2) it is not clear to what extent the AAR frameworks and best practices are suited to the specific needs and capabilities of teams. We emphasize the importance of context and identify that when translating an after-action review based on the military context to the firefighting context, there are significant differences between the theoretical after-action review and the after-action review utilized in real life.  We developed two main recommendations for extreme action teams based on the context in which they operate. Firstly, after-action review frameworks and best practices should be deviated from if they are not realistic or if they are too constraining. Secondly, we would encourage that extreme action teams to change and adapt the theory according to their own needs, to make the after-action review as effective as possible for them.

An After Action Review (AAR) is the Army training system's performance feedback mechanism. The purpose of the AAR is to improve team (unit) and individual performance in order to increase organizational readiness. While a large body of knowledge exists that discusses instructional strategies, feedback and training systems, neither the AAR process nor the AAR systems have been examined in terms of learning effectiveness and efficiency for embedded trainers as part of a holistic training system. In this thesis, different feedback methods for embedded training are evaluated based on the timing and type of feedback used during and after training exercises. Those feedback methodologies include: providing Immediate Directive Feedback (IDF) only, the IDF Only feedback condition group; using Immediate Direct Feedback and delayed feedback with open ended prompts to elicit self-elaboration during the AAR, the IDF with AAR feedback condition group; and delaying feedback using opened ended prompts without any IDF, the AAR Only feedback condition group. The results of the experiment support the hypothesis that feedback timing and type do effect skill acquisition, retention and transfer in different ways. Immediate directive feedback has a significant effect in reducing the number of errors committed while acquiring new procedural skills during training. Delayed feedback, in the form of an AAR, has a significant effect on the acquisition, retention and transfer of higher order conceptual knowledge as well as procedural knowledge about a task. The combination of Immediate Directive Feedback with an After Action Review demonstrated the greatest degree of transfer on a transfer task.

The importance of looking at student and teacher interactions holistically is essential and results in reflective teaching practices; consequently, the reflective practice of teaching needs to incorporate all facets of the teacher, known and unknown. This study looks at the potential influence of hidden biases towards adolescent Hispanic males and students with Emotional Behavior Disorders by observing preservice teacher (PT) interactions with students within a simulated classroom environment. Factorial MANOVAs and Discriminant analyses revealed statistically significant interactions and relationships between participant level of bias and the identified student avatars. These exchanges were more prevalent with one student avatar by both experimental and control PTs; indicating that student characteristics and their differences are important factors that need to be considered when addressing issues related to bias.

Reflection is an important team learning activity, allowing team members to process feedback and to learn from experiences. Debriefing methods such as the After Action Review (AAR), proved to increase learning, can be used to guide teams through the process of reflection. This thesis, commissioned by the Creating Collaborative Resilience Awareness, Analysis and Action for Finance, Food and Fuel Systems in INteractive Games (CCRAAAFFFTING) project, aims to investigating how current post-exercise discussions within the CCRAAAFFFTING project support reflection and whether or not AAR could be implemented. This study used an explorative, semi-experimental research design, containing two conditions - the current post-exercise discussions (condition 1) and AAR’s (condition 2). Data was collected through observations and questionnaires. Due to issues regarding validity, data from questionnaires given to participants, as well as comparisons of participants' subjective experiences between the two conditions, could not be used to provide any support for choosing one post-exercise discussion over the other. However, observations showed that the current post-exercise discussion did not support reflection, thus, making it beneficial to implement AAR. Proper implementation of AAR could not only support team learning, but also organizational learning by facilitating the transference of knowledge back to participants home organizations. This transferred knowledge, if used right by the organizations, could help increased resilience. However, AAR cannot simply be implemented, and this thesis concludes with a list of identified recommendations.

Kyoto University (京都大学) / 0048 / 新制・課程博士 / 博士(情報学) / 甲第19114号 / 情博第560号 / 新制||情||99 / 32065 / 京都大学大学院情報学研究科社会情報学専攻 / (主査)教授 林 春男, 教授 田中 克己, 教授 喜多 一 / 学位規則第4条第1項該当