Lessons Learned Part 5: Capture Using After Action Reviews
This article is part 5 of a series of articles exploring the Lessons Learned Life Cycle.
One of the most common methods for capturing lessons is the After Action Review (AAR). AARs have been around for a long time, the process having been created in the 1980s by the U.S. Army as a way of quickly capturing lessons after an event while that event is still fresh in the minds of those who experienced it. There are three premises behind the AAR: (1) localness, or what the Army calls “ground truth” – the idea that those who experienced an event directly are the ones best able to understand its significance; (2) forward focus – thinking about what just happened in the context of what is going to happen next time; and (3) iteration — that by keeping the AARs short and doing them frequently, participants are conditioned to blur the line between learning and doing, fostering emergent learning.
On the surface, the AAR is a very simple process consisting of asking four questions:
- What did you expect to happen?
- What actually did happen?
- Why did it happen?
- What will you do next time?
These questions are great because they are neutral. Although most AARs focus on capturing lessons from bad experiences, it’s also important to capture lessons from the times where things went well (perhaps by accident) so that they can be consciously replicated in the future. So these questions can just as easily be used to capture a positive scenario as a negative one.
While the AAR seems like an easy way to capture lessons, there was a portion of my career where I facilitated quite a few AARs and discovered along the way that the process isn’t always as simple as it seems. So the remainder of this post is all about the tips I found for making AARs more effective. Note that whether the changes I suggest here work for you is highly dependent on the culture of your team and organization, so you should always be open to adjustments if you’re not getting the results you want. Just as you want your team to be evaluating expectations during the AAR, you should be running a “meta-AAR” in the back of your mind to make sure the AAR itself is meeting expectations.
When and how to run an AAR
AARs are ideally suited for projects, since there is a clear end date after which a review can be held. But you can also use them for ongoing improvement simply by creating a regular cadence based on the calendar, such as once a week or once a month. When using them with projects, it’s important not to go too long before holding as session. If you wait until the very end of a long project to review it, participants will fall victim to what’s known as “recency bias.” They will vividly remember the most recent issues but probably forget that big problem way back at the beginning of the project that got resolved pretty quickly. I often worked with engineers doing product development, where the projects could typically last 12 to 18 months. The projects were in phases, so we found it logical to have an AAR at the end of each phase.
I have also found it very useful to have a facilitator for the meeting who is not part of the team, for two reasons. First, it can be hard for a team member to ensure that the conversation is unbiased and active with balanced participation from everyone while at the same time participating in it. Second, it’s often easy for team members who have been working together closely on a project for a long time to develop shortcuts and shared assumptions. While these can help speed the project because everyone is on the same page, they can sometimes hinder the review process because they go unnoticed. It can be useful to have a facilitator who is not as knowledgeable about the project as the team members, someone who can ask the “dumb questions” that might elicit deeper thinking about the lessons that were learned. I’ll have more to say about this in my next post, where I offer some tips for facilitators.
In terms of presentation, I like projecting a spreadsheet on the screen with the four AAR questions as column heading across the top and a list of expectations down the left side. If possible, have a scribe who captures the information that comes out of the discussion by typing into the spreadsheet so that everyone can follow along and correct any misconceptions. Now that we have a general idea of how you might run an AAR, let’s look at each question in detail.
1. What did you expect to happen?
Nailing down expectations in a way that drives lessons can be tricky. For example, let’s say your project was late and you felt it should have had more people. So in the AAR you say that your expectation was “I expected that we would have more people.” What actually happened? “We didn’t have enough people.” What to do next time? “Get more people.” The whole exercise if conducted this way seems pretty trivial and time-wasting. The problem is that the expectation as phrased above embeds the solution in it. The real expectation is that the project would have been completed on time. Having a discussion around that expectation might lead to alternative solutions in which more could be done with fewer people. These alternatives get suppressed when the expectation is phrased as a leading question. This is especially true when there are budget constraints. Some team members may see the AAR as a way to advocate for whatever their favorite resource is: more people, more time, better equipment, etc. While these concerns may be valid issues, it’s better when they come out of the discussion as lessons and solutions rather than going in as expectations.
For some situations where team members already had an overwhelming number of meetings, I would solicit expectations ahead of time by email rather than having them emerge in the meeting through conversation. This gave me an opportunity to push back on the framing as in the example above. It also provided time to see grouping patterns that would allow me to categorize the expectations into groups such as communication, quality, design, manufacturing, etc. Once people start thinking about a particular expectation, it’s much easier cognitively to move on to a similar one than to be jumping back and forth randomly between broad topics.
2. What actually did happen?
A funny thing can occur when you have good participation from passionate people talking about what they’ve just experienced. As they describe what actually happened, they can quickly veer off into war stories, similar issues, and all kinds of things that don’t directly relate to the specific expectation now being analyzed. After this happened to me several times, I changed the wording of this question slightly from What actually did happen? to Did it happen (y/n)? What I wanted out of the participants was a simple answer: Either the expectation was met or it was not. This provided two benefits. It not only kept the discussion on track (which was the original intent), but I found that it was a good indicator of a well-defined expectation. If I was getting answers like “I’m not really sure” or “well, yes and no,” it meant that we needed to go back to the first question and rework the expectation. Maybe it was too vague or jumbled up a couple of expectations into one sentence. Of course the down side of replacing a discussion question with a yes/no question is that you lose opportunity for conversation. So I tried to compensate for that by modifying the third question.
3. Why did it happen?
It’s obviously important to know why something happened if you hope to derive a lesson from it, but the closer to the root cause you can get the better it will be. This is especially true when trying to capture lessons from good things that happened. There is a tendency to think “We expected this, it happened just as we expected, and the reason was because we planned it that way. End of story.” But there are always new things to be learned by digging a little deeper. So to compensate for reducing the discussion as mentioned above, I added a second “Why” question. What this looks like in practice is that my spreadsheet has two columns that both just say “Why?” This is essentially a short version of the 5 Whys root cause analysis process that Toyota has promoted for many years with two whys instead of five. When you ask the why behind the why, you often find that what look like technical problems on the surface are really just symptoms of deeper communication or business process problems. Solving these kinds of problems gives you a bigger bang for your buck because they can be a single solution to many many surface-level problems. Getting team members to think deeply about why things happened the way they did will result in lessons that can be real game-changers for the whole organization, not just the current team.
4. What will you do next time?
If your organization repeatedly engages in very long projects containing multiple stages or phases (think construction or product development), you will quickly realize that “next time” can mean two different things. Sometimes you want to use what you’ve learned in the next phase of the same project, but sometimes you want to use it in the same phase of the next project. In these situations, simply split this column into two. Instead of using the words “next time,” have a column for “next phase” and one for “next project.” This makes clear when the lesson should be applied and also helps when it comes time to assign responsibility. I sometimes also added a short column just after this question simply labeled I/S. Its purpose is to mark whether the action to be taking is to improve a practice that is not as good as it could be or to sustain a good practice that we want to emphasize and repeat.
Speaking of responsibility, it’s also sometimes good to have a final column where you can put names assigned to take the actions that have been identified. Sometimes it means ensuring that things are done differently next time, and sometimes it means taking a specific new action in between projects to fix a problem at its root cause. Without well-defined assignments, it’s very easy for the whole team to walk out of the AAR nodding their heads in agreement, only to find out next time that nothing had changed because no one individual had the explicit responsibility to change it.
What about a Before Action Review?
If you can have an AAR, why not have a BAR? Although BARs are not as standardized as AARs in terms of the questions to be asked, projects often have a kick-off meeting at the beginning in order to set expectations. So this can be the perfect time to have a standard approach that can feed into the AAR later. I have found that these four discussion questions work pretty well in preparing for a project:
- What are the specific, measurable expectations that will define success? Can they be prioritized (“must have,” “nice to have,” etc.)? A good crisp set of expectations defined up front serves as a guide throughout the project and can also be plugged into the AAR framework as a starting point for discussions at the end.
- Is there anything we must not do in order to be successful? Most folks tend to focus on the tasks that have to be done, forgetting that it’s sometimes equally important to understand actions that need to be avoided.
- What did we or others learn from past similar situations that we can make use of this time? There is no point in doing AARs if the lessons identified in them aren’t learned and applied in the future. It’s good to have a prompt like this at the start of a new project that forces the team to revisit the lessons that have already been captured.
- What challenges or obstacles to we anticipate? What are the back-up or mitigation plans for dealing with them? If you can develop plans for when things go wrong before a crisis while there is time to think, then when the inevitable crisis does occur everyone will just be operating to the plan rather than scrambling around trying to figure out what to do.
In the next post I’ll provide some tips I’ve collected over the years to help when facilitating an AAR.
Next part (part 6): Ten Tips for Facilitating After Action Reviews.
Article source: Lessons Learned Part 5: Capture Using After Action Reviews.
Header image source: Adapted from Review by Nick Youngson on Alpha Stock Images, CC BY-SA 3.0.
References and further reading:
- Lloyd Baird & John Henderson: The Knowledge Engine: How to Create Fast Cycles of Knowledge-to-Performance and Performance-to-Knowledge
- Marilyn Darling & Charles Parry: “From Post-Mortem to Living Practice: An In-Depth Study of the Evolution of the After Action Review”
- Moira Levy: A Holistic Approach to Lessons Learned
