Learning as We Go with the Power of Retrospectives

By Tristan Kromer

There are few skills in life as valuable as the capacity to learn from our mistakes. In every facet of our daily existence — our education, our relationships, our leisure, our careers — the ability to mentally review our choices, determine what worked and what didn’t, and build a plan to make better decisions the next time around is what separates those who grow in new directions from those who just keep going in circles.

The same is true for business. Companies that don’t learn from history are doomed to repeat it, while companies that do learn from history will drive those who don’t right out of the market. In other words, companies that don’t learn from history will BE history.

This backwards-looking analysis is commonly called a retrospective in the agile, lean, and design-thinking communities. But retrospectives are not a new phenomenon; historically, this general review-in-hindsight has a different name: After Action Report.

The after-action report (or AAR, because I don’t want to keep typing “after-action report”), is one of the most valuable tools a business can have in its kit. In a very broad sense, an AAR goes by many names and can take many forms, largely differentiated by purpose, length, and focus.

There are few skills in life as valuable as the capacity to learn from our mistakes. Click To Tweet

So let’s start by splitting some hairs on the main types of AAR we might encounter in a lean startup environment.

Debriefing: A meeting to share facts and review collected data (from an experiment, sales call, etc.). A debriefing is generally short, focused, and meant to convey information rather than identify and solve problems.

Retrospective: A retrospective might follow a debriefing, but rather than focusing on the data, would focus on the methods and teamwork used to collect that data. In a lean environment, retrospectives are the shortest type of AAR, sometimes lasting only a few minutes — thus they are handy in just about any situation, and don’t necessarily need to follow a debriefing. Retrospectives occur as needed, rather than just at the end of a project.

Root cause analysis: As its name implies, a root cause analysis seeks to solve a specific problem. It could be something that surfaces in a debriefing, or something that is identified in a retrospective and is significant (and unclear enough) that a full diagnosis is needed to avoid just treating the symptom but leaving the problem to fester.

Performance review: A performance review is about people rather than processes, evaluating team members, not the projects themselves. This is more commonly associated with large companies and yearly promotion cycles. It’s not seen so often in the context of innovation teams since we rely so heavily on team performance, not individual performance. Rest assured, if a specific team member is underperforming in a small startup team … everyone knows it.

Post-mortem: Literally meaning “after death,” a post-mortem is a review of an experiment or project after it is complete. Unlike other AARs, the post-mortem isn’t meant to adjust course during a project or between sprints, but to learn lessons that can be applied to future endeavors (some people would call this a project retrospective).

Feet of a dead bodySince we are talking about making adjustments to projects and experiments as we go, we’re going to say goodbye to performance reviews and post-mortems (See ya!), and focus on the relationship between debriefings, retrospectives, and root cause analysis.

But before we get into these details, let’s take a look at the big picture to understand the history and context of AARs.

A REVIEW OF REVIEWING REVIEWS

The simplest way to understand AAR’s is by thinking of them as answers to five simple questions:

What did we want?

What worked?

What didn’t work?

What was learned?

What improvements should be made for future projects?

In what format these questions are answered depends on the size and scope of the organization. For example:

Military

The modern AAR was developed by the U.S. Army in the 1970s, but military AARs go back as far as Julius Caesar’s Commentarii de Bello Gallico (Commentaries on the Gallic War), a 2,000-year-old Latin classic chronicling Caesar’s nine-year exploit fighting the German and Celtic peoples. While the Commentarii is a book-length manuscript, the modern military AAR is often a short script that incorporates qualitative and quantitative feedback reviewing an exercise to “correct deficiencies, sustain strengths, and focus on performance of…training objectives.”

Military units around the world use different forms of AAR — ranging from informal discussions to highly structured documents, such as this modern sample from the U.S. Marine Corps:

Sample of AAR from the U.S. Marine Corps part 1Sample of AAR from the U.S. Marine Corps part 2

Government

After the Army introduced the modern AAR, it quickly became one of the most valuable weapons in its arsenal, and it wasn’t long before other government departments took note.

AAR powerful tool

The government — a bloated bureaucracy by nature — managed to make these AARs larger and more complex. After Hurricane Katrina devastated the Gulf Coast in 2005, independent AAR’s were produced by the White House, the Senate, and the House of Representatives. The House report alone was 364 pages.

Some government agencies now produce AARs to review things other than specific events, such as FEMA’s post-mortem on the entire 2017 hurricane season, landing at a paltry 56 pages. Slackers.

Bureaucracy mountain of papers.

Business

There are as many formats to business AARs out there as there are companies writing them. Ultimately, we all want an AAR template that’s right for our organization, one that addresses our priorities and speaks to our values.

If we need a place to start, a simple template might take this structure:

  1. Overview
  2. Goals and expectations
  3. Outcome
  4. Evaluation
  5. Recommendations

Put into action, we end up with something like this — an actual lean AAR used by a startup for a recent event (the names have been changed to protect the litigious).

Tech chat AAR document part 1.

Tech chat AAR document part 2.
THE LEAN AAR

At Kromatic, we like to keep our AAR process focused on problem-solving, so our review process leans heavy on a quick-and-dirty retrospective. Typically we set a timer for three minutes, and ask everyone to write down on a sticky note:

  1. What went well?
  2. What didn’t go well?

Someone puts down sticky notes on stickies wall.

Then we review and QUICKLY discuss what everyone wrote down, looking for patterns and trying to identify the most prominent solvable issue. These discussions are typically under two minutes.

After this BRIEF discussion (did I mention it should be quick?), we set the timer for another minute and write out actions to address these issues.

Behold, five minutes that could change the course of your experiment:

Backlog grooming retrospective diagram
(You’ll notice Elmo has been joining our retrospectives lately to keep us on task.)

While the example retrospective above took about five minutes to conduct (remotely, thanks to Mural), these babies can be boiled down to sixty seconds (damn that’s lean!) by slimming down the process:

  • Set the timer for 60 seconds.
  • Have everyone write down one thing they would change about the experiment.
  • Timer goes off, team leader collects the feedback and chooses ONE thing to correct in the next-go-round (and assigns ownership).

Is the sixty-second retrospective a pretty low bar for AAR’s?

Yes, definitely.

Is it still useful?

Yes, most definitely.

A flash-retro isn’t likely to foster any big changes, but it won’t waste time, it curtails groupthink, it limits scope creep, and it delivers at least one small win for the team, lending a key ingredient to the execution of an agile experiment: momentum.

Red race car moving fast.But of course retrospectives can be done on any scale, and there are some great resources out there for different formats. And as I mentioned above, while they can be — and often are — used by themselves, they are also very practical when used in combination with a debriefing and a root cause analysis.

For example:

Debrief: What happened? We threw a party for a dozen two-year-olds. Five of them threw tantrums, two families left early, and we had to fire the clown.

Retrospective: What worked? What didn’t? What now? The cake and decorations were lovely. The clown terrified the children. In the future, we need to more closely scrutinize our entertainment options.

Root cause analysis: Why did we err? We asked Uncle Barry to book the entertainment, and he hired the clown from his bachelor party. The clown was not informed he would be entertaining children.

This example is quite simplistic, but it makes our point about the useful nature of AARs: to identify, to analyze, to solve.

Whether it’s a one-page document, a multimedia presentation, or a week-long conference, an organized review of an experiment is the surest path to constant improvement and a powerful way of communicating expectations to our team.

How to fill in the details — through surveys or individual/group meetings — depends on the nature of the company. But it must be a team effort. It’s crucial to the learning process that everyone involved in the experiment contribute to the AAR, and that they can do so without fear of judgement or reprisal.

As the Army’s guide eloquently puts it, “The environment and climate surrounding an AAR must be one in which the soldiers and leaders openly and honestly discuss what actually transpired in sufficient detail and clarity that not only will everyone understand what did and did not occur and why, but most importantly will have a strong desire to seek the opportunity to practice the task again.”

To put it in lean terms, we need to continuously improve our process as we iterate on our product, and the best way to do that is through clear and open communication from everyone on the team.

Lessons Learned:

  • Always, always, always do AARs (did I mention always?).
  • Debriefings, Retrospectives, and Root Cause Analyses serve different purposes; use the correct format for your needs.
  • Don’t try to fix everything. One improvement is better than ten suggestions.
  • AARs are a team effort — egos are incompatible with continuous improvement.

Interested in learning more about innovation? Kromatic is hosting online interactive workshops on Customer Discovery, Lean Experiment Design and Innovation Ecosystems: Mapping Obstacles & Metrics. Grab your ticket today!

Got something to say?

This site uses Akismet to reduce spam. Learn how your comment data is processed.