Keywords

1 Introduction

An AAR is a structured review or debrief process for analyzing differences between actual and expected performance after military training exercises or actual tactical events. AARs also provide a process for identifying and using lessons learned to improve tactical performance or change individual behaviors following scenario-based training. According to Army Doctrine Reference Publication (ADRP) 7-0 an AAR is “…a guided analysis of an organization’s performance, conducted at appropriate times during and at the conclusion of a training event or operation with the objective of improving future performance. It includes a facilitator, event participants, and other observers. Team members, or participants, provide responses to questions about what happened, why it happened, and agree on how to sustain strengths and improve performance. Often, a team leader directs an AAR and focuses on only what could have been done better, paying little attention to what was done well and why. Formal AARs were originally developed by the U.S. Army in response to the need for arriving at performance improvements by blending squad member inputs with objective performance measures. Effective AARs are usually centered on formative feedback, self-monitoring and self-reflection, which can deepen and expand learning [1, 2]. An AAR is essentially an opportunity to improve tactical performance.

The Integrated After Action Review (IAAR) was developed by the Squad Overmatch (SOvM) research program- a multi-year, joint US Army – US Navy research effort - to improve individual and team performance under stressful conditions. SOvM training integrates tactical skills and team behaviors in five skill areas to improve mission effectiveness [3, 4]. The integrated training approach includes classroom training (knowledge acquisition), participation in simulation-based training (opportunity to practice what was learned in the classroom), and participation in live training (opportunity to apply what was practiced virtually and learned in the classroom). SOvM scenario based training includes an IAAR after each virtual and live scenario. The IAAR is introduced as an AAR that covers multiple skill areas that are integrated through scenario based training and discussed during an IAAR. Because training resources are limited, it makes sense that training objectives should be collectively combined for training (when resources can be shared and it is complementary to the skills areas to do so). An important difference between the IAAR and traditional AARs is where the discussions are focused. During a SOvM IAAR, the squad shifts its focus to teamwork behaviors, instead of predominantly focusing on tactical skills. The IAAR creates an atmosphere where each squad member’s role shifts from Soldier being corrected to Soldier offering self-correction.

Both AARs and IAARs are opportunities to improve performance through facilitated discussions that start with agreement on an overall goal and training requirements. Each compares expected performance to actual performance and requires individual accountability for task performance. The main difference between an AAR and an IAAR is that an effective IAAR emphasizes collective learning across multiple skill areas (vs only tactical skills) and requires all squad members’ (from the lowest level up) participation and engagement (vs the team leader doing most of the talking). IAARs that address tactics and teamwork require members to be accountable for team performance and contribute to solutions and goals. Therefore, in order for an IAAR to be effective it should create a learning environment that provides opportunities for knowledge exchange, facilitates changes in behaviors, and is resourced to learn from information collected during scenario based training.

2 SOvM Training Effectiveness Evaluation

A Training Effectiveness Evaluation (TEE) of SOvM was conducted in June 2016 at Fort Benning, GA. It was led by the Program Executive Office for Simulation, Training, and Instrumentation, Army Research Laboratory Human Research and Engineering Directorate, Naval Air Warfare Center Training Systems Division, The MITRE Corporation, and Cognitive Performance Group. The U.S. Army Maneuver Center of Excellence, Maneuver Battle Lab, Clarke Simulation Center, and the McKenna training complex. These organizations provided the training and simulation resources at Fort Benning, GA.

Participants included four squads from the 82nd Airborne Division (Fort Bragg, NC) and four squads from the 75th Ranger Regiment (Fort Benning, GA). Each squad was augmented with a 68 W medic from the 690th Ground Ambulance, 14th Combat Support Hospital (Fort Benning, GA). Squads size ranged from eight to ten members. Four squads participated in an experimental condition and four squads participated in a control condition.

Squads in the experimental condition received classroom training, participated in two simulation-based training scenarios, participated in three live training scenarios, and engaged in an IAAR after each scenario. Control condition squads participated in only two live training scenarios and participated in a traditional AAR after each scenario.

Squads in the experimental condition received instruction from five instructors in five skill areas:

  • Tactical Combat Casualty Care (TC3) – Trains communication and team member roles and priorities in response to medical tactical situations.

  • Advanced Situation Awareness (ASA) – Trains human behavior pattern/threat recognition and decision making in complex environments.

  • Resilience and Performance Enhancement (RPE) – Develops squad member skills in maintaining tactical effectiveness under combat stress.

  • Team Development (TD) – Develops teamwork skills including Information Exchange, Communication Delivery, Supporting Behavior, and Team Initiative/Leadership.

  • IAAR – Develops an understanding of the IAAR process, skills in applying the Force of Four framework, and methods for identifying the characteristics of an effective IAAR.

The cornerstone of SOvM training is the IAAR. It is the culminating event that provides the foundation for the integration of the skill areas and offers the opportunity for teams to detect errors, reflect on behaviors, and self-correct their performance. These activities lead to improved team performance.

3 Preparing for and Conducting the SOvM IAAR

IAAR preparation includes a variety of techniques to observe and collect examples of skill area behaviors and to discuss them openly. This approach is based on years of team research findings. The research found that participant feedback is most effective in improving performance [1]. These improvements occur when the team recognizes its less than optimal behavior, acknowledges the consequences of that behavior, generates solutions, and sets goals to improve behavior. Each person on the squad actively participates in the process by identifying examples of good and poor performance during scenarios and by contributing to opportunities where the team recognizes team errors and discusses more effective solutions. This approach encourages the team to collaborate on improving its performance through goal setting.

The SOvM IAARs for the TEE included the skill area instructors, an IAAR Facilitator (for SOvM, the Facilitator was the squad’s Platoon Leader), and the Army squad itself. The IAAR followed a process with specific steps (see Fig. 1. ‘IAAR Process’ below).

Fig. 1.
figure 1

(Source: Mitre Corporation).

IAAR Process

Full size image

The IAAR model includes an instructor for each skill area and an IAAR Facilitator who guides the discussion. These individuals observe and collect squad performance data during virtual and live scenario-based training. Then, during the IAAR that follows each scenario the Facilitator reviews performance objectives and elicits squad inputs about the tactical timeline. Then instructors review skill area learning objectives and ask squad members (1) where they struggled and excelled (triggers); (2) to agree on what went wrong and right (teamwork behaviors); (3) to propose a workable solution (identify correct procedure); and (4) to discuss real world outcomes and consequences. This is called the Force of Four, which provides a framework for team self-correction during the IAAR. With the support of the Facilitator and instructors, squads also set goals and integrate them into the next mission’s planning. In this IAAR process, the Facilitator and instructors act as guides to keep the IAAR on track. Squad members contribute and engage in team self-correction across the integrated skill areas.

For this multiple skill area focused IAAR to work optimally a number of resources were developed and used to keep the IAAR on track and covering all the required objectives within each skill area within a 30–40 min timeframe.

For the SOvM TEE, job aids for Skill Area Observation and Assessment were provided so that each instructor could link skill area objectives to specific scenario events and injects, making it easier to identify whether specific behaviors occurred. These job aids were paired with individual Skill Area Scenario Event Timelines and Overlays (see Fig. 2 ‘Individual TD Skill Area Overlay for Scenario M-2’ below).

Fig. 2.
figure 2

(Source: Mitre Corporation)

Individual TD Skill Area Overlay for Scenario M-2

Full size image

Other resources such as Gridded Reference Graphics (GRGs) (geographical maps) were also included. Additionally, individual Skill Area IAAR Job Aids were provided to guide team self-correction during the IAAR (see Fig. 3. ‘TC3 AAR Job Aid example’ below). These job aids contained each learning objective (expected behavior) and questions to ask requiring squad members to monitor and reflect on their own and their squad’s performance following the Force of Four framework. Finally, a Set Goals Job Aid was used to allow the squad to identify, prioritize, and set goals. So each IAAR provided opportunities to review learning objectives, discuss performance, and agree on goals.

Fig. 3.
figure 3

TC3 AAR Job Aid (one page example from Scenario B-1)

Full size image

In order for an IAAR to work effectively, there must be a synergy between instructors and an IAAR Facilitator. The instructors, as experts in their areas, must be allowed to contribute to the IAAR by asking specific questions and guiding discussion related to their skill areas. The IAAR Facilitator must express the tactical scenario expertise and have an overall basic understanding of the skill areas to be able to offer an integrated perspective to the squad.

An IAAR rich with learning objectives across multiple skill areas demands active participation from all squad members and skilled facilitation. Allowing squad members to contribute freely allows them to be accountable and share their perspectives of what happened, why it happened, and how to learn from the experience without reservation. The SOvM IAAR Facilitator, skill area instructors, and squad members engaged in effective IAAR questioning, feedback, and response techniques during scenario-based training, fostering an environment that was conducive to detection, reflection, and self-correction (see Fig. 4. ‘IAAR with soldiers at Schofield Barracks, HI’ below).

Fig. 4.
figure 4

(Source: Mitre Corporation).

IAAR with soldiers at Schofield Barracks, HI

Full size image

4 Lessons Learned

Overall, the AAR approach used in the control condition and the IAAR approach in the experimental condition was well received during the TEE. Self-report surveys revealed the majority of soldiers in both conditions rated the AAR climate following the live scenarios as strongly supportive and positive [5].

Many procedures implemented for the IAAR were successful throughout the TEE. We found that having individual instructors for each skill area instead of one instructor or only the IAAR Facilitator attempting to cover all the integrated, yet distinct, skill areas in the IAAR proved to be a good approach. Each of the SOvM skill areas were condensed from much longer program of record courses. One instructor would have had a difficult time understanding the objectives of each area, within each scenario, and know all the critical issues to address. A skilled expert handled this more effectively and efficiently. Similarly, it would have been challenging for the IAAR Facilitator to provide the tactical AAR as well as knowledgeably cover critical skill area objectives during the IAAR. Individual Observation Job Aids used during the scenarios were useful in quickly identifying skill area behaviors around trigger events within scenarios. These were then easily used to verify where the squad performed well and where they had challenges. This data also informed which questions to ask on the IAAR Job Aids and which areas to cover during the IAAR. The Set Goals Job Aid was used effectively and provided direction on identifying, prioritizing, and setting goals. The IAAR Facilitator was also a necessary and well-functioning role. This individual facilitated the entire IAAR, provided a tactical debrief, made sure instructors stayed on track, and ensured they each had opportunities to discuss objectives within their skills areas. Other aspects of the IAAR approach we found needed to be streamlined and improved. Following is a listing of our lessons learned and how we modified the IAAR process to improve the approach. Most of these improvements were implemented during operational testing of SOvM training at Army and Marine Corp bases during 2017.

Lesson Learned 1. Combine the Scenario Event Timeline Display to Highlight a Sample of Learning Objectives. The Individual Skill Area Scenario Event Timelines made it challenging for the IAAR Facilitator to ensure specific learning objectives were discussed. Most of our resources developed specifically for the IAAR were focused on individual skill areas. Although it made it easier to focus on each skill area by addressing it separately, these individually focused job aids and resources made it more challenging to integrate the learning opportunities and present to the squads a unified training approach. The inter-relationships of these skill areas had been taught in the classroom, but this was not reinforced during the IAARs. This individual approach led to somewhat time consuming IAARs because each instructor needed time to discuss his objectives on separate job aids. To remedy this problem, the Individual Skill Area Scenario Event Timelines were combined for each scenario and these highlighted a sample of objectives instead of all objectives for each skill area in each scenario. Using this approach, objectives were still covered, but dispersed across scenarios, allowing opportunities for other skill area objectives to be presented together more fluidly. Only one Overlay was needed per scenario (see Fig. 5. ‘Updated Scenario Event Timeline’ below). Each instructor was provided time to cover their skill area objectives, but they focused on a subset and utilized one graphic to do so.

Fig. 5.
figure 5

(Source: Mitre Corporation)

Updated Scenario Event Timeline

Full size image

Lesson Learned 2. Design IAAR Job Aids to Include General Questions for Multiple Skill Areas for use by Each Instructor. We learned that the Individual Skill Area IAAR Job Aids resulted in a disjointed and less integrated IAAR. Therefore, we determined the IAAR Job Aids should be developed to include general questions that address multiple skill areas and can be used by each instructor for each scenario. Each instructor would simply need to determine which question(s) to ask to better meet specific learning objectives within a scenario, ensuring each had ample opportunities to engage with the squad in areas where performance deficiencies were observed. This determination could be made during the ‘huddle up’ suggested solution below. Certain scenarios might be better suited for one skill area (or more) over others. For example, in the SOvM scenarios, the earlier ones that focused on establishing a baseline were rich with opportunities for ASA behaviors to be observed and later scenarios that escalated with more TC3 events (e.g., casualties) provided numerous TC3 and TD (e.g., communication delivery, information exchange, supporting behavior, and initiative/leadership) behaviors to be exhibited. Discussing which and when certain skill area behaviors were largely utilized in the scenario and determining which of these are the most critical to debrief during the IAAR can each be better accomplished with integrated IAAR Job Aids. This would make it easier and provide greater flexibility for all instructors to more quickly determine IAAR direction and areas of emphasis.

Lesson Learned 3. Conduct a Huddle Up with the Facilitator and Instructors. We found that moving directly from a scenario into the IAAR led to confusion on sequence and timing of skill area discussions during the IAAR and which strengths and weaknesses to focus on. We determined that a Huddle Up for instructors and the IAAR Facilitator that takes place in between scenario end and IAAR start would ensure scenario challenge areas were agreed upon, strengths and weaknesses within each skill area would be addressed, and provide structure and direction to the IAAR resulting in less confusion. This critical step was added to Fig. 6. ‘IAAR Process’ in between Observe/Collect Squad Performance (during exercise) and Review Performance Objectives (during IAAR). The Huddle Up is essentially a planning session for conducting the IAAR, an opportunity for instructors to exchange notes and talk with the IAAR Facilitator about which trigger events and learning objectives to focus on within key squad challenge areas and also positive aspects that should be highlighted throughout the IAAR. Preparation for the Huddle Up should take no more than 5–7 min and the same timeframe should be sufficient for the Huddle Up itself. Before the Huddle Up, instructors should review notes and tie them to events/triggers in the Scenario Event Timeline based on training objectives/performance issues. Instructors should gather performance assessment information from all sources (e.g., role players, Medic) and talk with each other about integrated learning objectives.

Fig. 6.
figure 6

(graphic courtesy of Mitre Corporation)

Huddle Up Steps

Full size image

Huddle Up Steps are listed in Fig. 6. ‘Huddle Up Steps’ below. Accompanying video examples have been developed for SOvM operational implementation and transition efforts to emphasize key steps in the process and provide subject matter expertise in executing. During the Huddle Up, the IAAR Facilitator should ask instructors whether their training objectives were met, if there were any squad weaknesses, if goals were met, and when they want to talk in relation to the Scenario Event Timeline during the IAAR. Each instructor should define major training objectives, related errors, and examples of good performance. The instructor and IAAR Facilitator group should determine where overall focus should be placed during the IAAR. Finally, to ensure IAAR organization and to optimally utilize the time allotted, the IAAR Facilitator should determine who will run the slide deck and take notes on goals.

5 IAAR Interactive Dashboard

Throughout the SOvM training, data are collected to support review and analysis by instructors and the IAAR Facilitator during the IAAR process. There are challenges of transforming large quantities of data into information about squad performance in time and in a form to support team development and performance improvements. One solution is the creation of a dashboard that facilitates data aggregation, synthesis, and presentation of results during the IAAR. Currently, harvesting and making sense of the data for the IAAR has been difficult to accomplish. By using a big data approach for identifying relationships among the data sources, like Observation and Assess Job Aids and IAAR Job Aids, automated field notes, GRGs with location identifiers, and audio/video, we believe sufficient, high quality performance analytics are available. An IAAR dashboard that would allow the instructors and IAAR Facilitator to enter, track and report on the Squad’s progress during each stage of training is on the drawing board. We have conceived of an enterprise level dashboard solution that would link several data collection platforms; arrange and optimize results for reporting purposes; and deliver the information through an intuitive user interface during the IAAR. We believe that evidence-based displays not only reveal patterns of performance, they would also support near-transfer of essential feedback for team learning.

6 Conclusions

As standard as AARs have become as part of scenario-based training, IAARs provide a unique approach in ensuring multiple skill areas covered in training receive the attention needed to impact future performance. The SOvM TEE provided an environment where approaches to an IAAR could be tested and studied and valuable lessons learned derived. A number of resources are necessary for managing different skill areas and mitigating challenges with an increased number of individuals facilitating and running the IAAR. The effort involved is worth the benefits of covering multiple skill areas that complement each other when integrated and raise the potential level of learning and performance impacts. With the constrained and often limited resources available in training today, we will likely see more attempts at combining training topics and efforts. Embracing an IAAR approach can help ensure individual aspects that are combined receive similar attention to what they would have if trained separately.