Effective retrospectives in Agile development

Leveraging everyone’s knowledge and ideas for improvement can give considerable benefit to the business.  However, you need to implement a pathway for change which listens to the individuals and captures their ideas.  Agile development typically uses the term “Retrospective” for this process of looking back at what has happened.  There are other terms used in projectised environments, such as “Post-mortems” or “Lessons Learned”. The different in language generally reflects the difference between viewing work as continuous flow (in Agile) against transient blocks of work (in projects).

As I emphasise through these Plays, continuous improvement is key to supporting a scaling organization.  Retrospectives are key to that improvement.  No process will be perfect.  Equally importantly, the organization will also not be static.  A perfect process is therefore not only unlikely, but not even a desired goal.  The one process we do need, however, is a way to learn and improve.  If we can learn effectively we can scale and improve. Scalability is at the heart of what we are trying to achieve.

Different teams will have different responses to events.  These responses are driven by the expectations set by the organization.  And these expectations go back to the management style.  Is the organization driven by fear and directive management?  Or are the team self-managing and able to direct their own actions?  Is there a culture of continuous improvement?

Fear-based culture

In a fear-based culture, the response to a project failure is to blame others. Since safety is paramount, the team will seek to find explanations which remove their own accountability for failure.  If the focus is to find specific explanations rather than to understand the truth it is unlikely that there will be much learning involved. 

Worse still, since the team are looking for an explanation in which they are not accountable, they will not feel that they should make changes for the future.  A fear or blame culture pushes teams to believe that the problem is outside their accountability.  The result is they believe that any solution is outside their control.

Action-based culture

In an action response, the team rallies around fixing the immediate issue.  Everyone works long hours, the crisis is solved and then the team gets back to normal working.  This is far preferable to just finding someone to blame.  But there is little thought given to how to prevent the recurrence of the crisis.  The lack of learning is because of a belief that crises are inevitable.  Although that sounds strange it is a widely held mind-set.  Managers believe that their key role is to manage the crises.  The focus here is on corrective actionwhich addresses today’s problem, not preventative measures to ensure that the problem does not recur.

Learning based culture

In a learning response the immediate problem still gets fixed.  However, the team continues beyond this point and considers how to avoid a recurrence. 

The next step here is to consider why something went wrong.  The team need some deep analysis to get beyond the symptoms to the underlying cause.  By abstracting the issue to the “root cause” behind it, the team can see how it could be prevented next time.  This takes an investment in time and isn’t easy.  The analysis can need some skill and facilitation.  And it can be painful for a team to admit that errors have been made as they explore how to prevent them in the future.

Event based retrospectives

We should certainly perform event-based retrospectives.  I would advise these whenever there has been a notable event which you need to understand.  This would include a significant escape – a defect in a live production environment. 

Retrospectives should not be triggered only by problems.  It is also very valuable to do a retrospective every significant release.  This lets the team review progress and change over a significant timeframe.  It lets them decide where improvement might occur without being targeted on a specific problem.  And it lets them identify and celebrate where they have been successful.

An escape retrospective looks at how the defect was introduced, and why it was not caught before delivery.  As always, the objective is not to assign blame.  The purpose of the retrospective in this case is to understand the underlying root cause.  What caused the issue?  Why was it not spotted and corrected?  Most importantly, how can we prevent it from recurring?

Root cause analysis

Root cause analysis is a key skill to develop to make retrospectives work. It’s a good reason to have facilitators involved in some cases, as there is some skill involved.

Whenever we discuss past events we tend to focus on symptoms. What did we observe? How did that affect us? However, symptoms do not generally give a clear picture or an actionable solution.

The key lies in asking why these happened. As we ask why we move from symptoms to an underlying technical root cause. What was the event which introduced the problem? What actual error was made?

As we continue to pursue why we can learn why the issue was not found before it had an impact. And finally we will get to the underlying question of how our way of working allowed the error to be made and not found. To really make a difference we need to expose this level of process issue, which allows us to make changes to the way we work.

However, this is challenging. We need to push our understanding with many levels of why and we need to accept that the answers will include our own imperfections. Shigeo Shingo puts it well in one of my favorite quotes.

Sprint retrospectives

Retrospectives fit well into the iteration process and are a key part of the Sprint cycle in Scrum.  The expectation is that each team will regularly hold a retrospective.  This allows the team to consider how they are performing and how they can improve.  This is part of owning their own progression or self-transcendence.

In Scrum, the Sprint Retrospective is held every Sprint.  It makes sense to tie this to the iteration cycle.  The team have planned and delivered a set of work.  The cadence of an iteration ensures a brief pause for reflection.

Not a Sprint Review

There is often considerable confusion between Sprint Retrospective and Sprint Review.  This is not helped by the similar names and both occurring at the end of the Sprint. However, the two are quite different in purpose.

The Sprint Review looks at the product.  It gets the stakeholders (either customers or their proxies) together to inspect and discuss what has been created.  And then it adapts the future direction of travel based on what was created and learned.

By contrast the Sprint Retrospective looks at the team.  It is a collaborative team event.  It does not generally include wider stakeholders.  To maximise trust and psychological safety, keeping the Retrospective to the core team is generally a good decision. 

The Sprint Retrospective looks at how the team feel they are performing. 

• What has their experience been? 
• Whichp has gone well?
• What is getting in their way? 
• What could be changed or improved? 

It could be a time to look at the team dynamics, the processes, tools, definition of done, or technical debt.  It could look at how the work was addressed, what errors were made and how these could be avoided in future.

Good Practices

As an Agile leader you should ensure that the teams are regularly implementing retrospectives. If you are using Scrum, fit them into the Sprint cycle.

You will also need to make sure there are event-based retrospectives. It’s too easy to be action-focussed, fix the immediate problem and move on. But then you are building up technical debt for the future.

Most importantly you will need to ensure a low-blame culture. Nothing destroys retrospectives faster than using them to blame people. I find using the “Prime Directive” to shape the mindset can help.

Finally, there is no point investing time in retrospectives unless you can generate change.  Often a mistake in Sprint Retrospectives is to talk about the problems, but not to solve them.  If you see the same issues recurring in subsequent retrospectives, you may need to encourage the team to add actions to their backlog to take action..