Doing an after-action review (AAR) online

Recently I participated in an innovative portfolio review process which the USAID field mission plans to make regular events. While pretty successful, there’s always room for improvement. I saw this as a perfect time for an after-action review (AAR) with the activity to be reviewed being a clear, definable event which those who designed and implemented plan to replicate.

The problem was how to do an AAR when the facilitator (me) was not in the room. There really wasn’t anyone on site who could facilitate it. We thought about using Adobe Connect, which has a whiteboard function, or maybe a video conference with the camera pointed to sheets which the facilitator would write on. I didn’t want to make the AAR too formal, or too slow and contrived, and I honestly didn’t quite know how to proceed.

So, I put the question to the KM4Dev and Pelican online communities: how could I facilitate a remote AAR?

Report on our online AAR

The KM4Dev and Pelican online communities provided some wonderful ideas and help in walking me through how to do a remote AAR.

Here’s a quick summary of what we tried, and some lessons learned.

To do the AAR, it was decided by those involved with the subject matter to use a “scribe” where the participants were located (Kampala, Uganda), and a video link between Kampala and USAID Washington. We weren’t sure how we’d maintain a dialogue as well as capture all of the interchange (and cover the points about AARs many in the KM4Dev and Pelican online communities had raised), but we were willing to give it a try.

But then … the weather forecast offered heavy snow exactly when we were doing the AAR, making the use of the agency video teleconferencing (VTC) system a non-starter.

So we decided in the evening of the night before to go to Plan B: using AID Connect (USAID’s Adobe Connect license). It was a little daunting (I use Adobe but I’m not an expert at it), but I drew on Adobe Connect and virtual learning guru Dar Maxwell who very nicely came up with a game plan. Plus I used many of the great ideas the KM4Dev and Pelican online communities had offered up; the net result was quite successful. It can be improved, but overall it went well.

The following just talks about the overall technical approach to an online AAR, and does not talk about the Uganda Mission’s innovative Field Based Portfolio Review (which is maybe worth its own article sometime!).

We had someone in Kampala be the primary scribe, and I was the primary facilitator, with another subject matter specialist as the co-facilitator. Per the informal “rule” of AARs, the facilitators were familiar with the activity, but not actually involved in its design or management – I participated in the field based review which had been the focus of the AAR while the co-facilitator had designed a predecessor event in 2014. We also had two others listening in and making sure nothing was missed. Overall, we had 6 people in Kampala (in one location) and then 5 staff in Washington in 5 locations. We also had my Adobe guardian angel (Dar) online just in case something went wrong.

Facilitation/staffing approach

• Facilitators in DC each calling from their respective homes.
• Discussion team in Kampala, Uganda in one conference room at the USAID Mission. Note taker with the discussion team in Kampala.
• Used the basic 5 questions in a generic AAR, although we tried to focus, per Nancy White, on the first question, What.

Adobe Connect (aka AID Connect) setup

We set up Adobe Connect so screens could be shared in a way which would be, mostly, like a normal in-room AAR. We only used video at the beginning during the welcome so the team in Uganda could see me before the discussion. Because the focus was on a free-flowing AAR discussion with simultaneous note-taking we decided to limit any additional video. The bandwidth is not reliable and we didn’t want the technology to get in the way. For the same reason, we decided to set up a separate phone line and not use a computer link. Unfortunately that meant that we would not have a simple integration of audio and the presentation, but given that the written sheets were sufficient we thought it was okay not to have integrated audio in this instance.

We used individual “note pods” devoted to each AAR question, similar to having the question at the top of a flip chart page. We grouped the questions across two “layouts” which we could make visible at will and toggle back and forth. As you can see in the screenshots below, the first three questions were grouped together and the last two were grouped together. We wanted to be able to stay in one “layout” at a time to notate a free flowing discussion. With multiple “note pods” (flip charts) visible, we were able to add content across multiple questions as the topic required. An AAR discussion doesn’t necessarily follow the order of questions the facilitator hopes!

Adobe Connect has the functionality to directly send the content of the note pods via email. We emailed the notes to the facilitators to “clean up: and consolidate within about five minutes after the session was over; once we factor in people’s comments and do some basic editing we will share with the full team. (These steps are actually better than a normal AAR because the flip chart notes have to be transcribed).

Sample screenshots

As you can see below, we left the attendee list and chat pod on screen so we had visual confirmation of team member connectivity from all locations and if we lost the phone line, we could still communicate (backups on backups).

There were three “screens”, or views:

Lessons learned

We’re thinking next time of maybe having three content screens – one with the first two, the second with the second two, and then a third with next steps. While there would need to be some back and forth between screens it would avoid having too much on that second screen. This is still very much a work in progress.

Needless to say, Adobe Connect is not the only software which does this. What is key though is a focus on being able to share a document which both sides can write on (Adobe Connect is good because their emphasis on “pods” which allows for easy swap out of pages to be worked on). Personally I’d steer clear of video, and make sure you have a counterpart in-country, to ensure the content can be put up without the annoying delay that happens even on the phone. It also builds trust (essential to an AAR), which can be an issue with a facilitator “from afar”. One reason it worked in this case is that I was well-known to the field staff and had just been out there. Not sure if it would have been as effective with a “stranger”.

In terms of other uses, a few years back I considered how to do a peer assist remotely. That I think is much harder. We’ve done some in a way using Adobe Connect, with countries presenting and then others from around the world commenting through chat (again, voice can create more problems than it’s worth), but I don’t think we’ve cracked the code – peer assists still seem to be so much better face to face.

Again, thank you so much to the KM4Dev and Pelican online communities for all of their suggestions; they were a huge help.

Dar Maxwell contributed additional technical details for this article.