Are your Retrospectives failing?
Most software organisations are probably performing retrospectives. Most Agile approaches (from Scrum to XP to SAFe) highlight retrospectives as a key activity.
“The Sprint Retrospective concludes the Sprint.”
the Scrum Guide
And yet in my experience many teams struggle with effective retrospectives. If retrospectives are so important (and who would deny the importance of self-improvement?), why are they often not successful?
In this article I want to step back a little from the much-discussed topic of how the team run a single retrospective event. I will focus more on what outcomes are important and how, as an agile leader, you should approach contributing to effective retrospectives.
Continuous improvement
If we are to improve retrospectives, we first need to consider why we bother with them. Retrospectives are a key part of continuous improvement. Some organisations try and design a perfect process from the start and then fix it in stone. There are three problems with this approach:
- Creating the process would require extensive research, pushing you into a up-front planning “waterfall” approach. In a complex environment it is questionable whether it is even possible to take this approach, but it is certainly intensely challenging to maintain momentum through the planning.
- Rolling out the process in a single “transformation” is highly disruptive and incredibly difficult to organise. Even if a perfect process could be developed (which is doubtful), the transition of a whole organisation will introduce many issues.
- No organisation is static. Our hypothetical “perfect process” is only perfect for today. As the organisation evolves, the process will need to evolve, and fixing the process is not a viable approach in a fast-moving world.
What we need therefore is a mechanism to evolve process, to learn and to improve. And this is probably more important than the original process itself.
What matters is not your process, it’s your process for improving your process
Henrik Kniberg , “Lean from the Trenches”
Why the retrospective is run
Retrospectives, as the name suggests, are looking backwards at what has happened, in order to learn from the past and do things differently. We could run these triggered by a specific issue such as an escaped defect or a failure, or even a successful delivery. This would be similar to what classical projects often refer to as a “postmortem” – a retrospective as a response to an event.
I would strongly advise event-based retrospectives, especially to understand major failures or successes. These learning points are a key factor in quality systems.
A typical Agile retrospective, however, is not about an event, but about a team. It is a regular occurrence aiming not for correction of a specific error, but continuous, recurring improvement. This approach is a part of continual improvement which is included in most Agile approaches.
At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
Principles of the Agile Manifesto
The end goal of a retrospective is for a team to improve. But also for a team to own their improvement. This ownership is a key capability which a team needs to develop. Takeuchi and Nonaka refer to this as “self-transcendence” and consider a key part of any self-managing team.
Teams appear to be absorbed in a never-ending quest for “the limit.”
“The New New Product Development Game” – Takeuchi & Nonaka
Not only about the “how”…
How a retrospective is run is important. Let’s not underestimate the challenge of getting a team to come together every iteration and create valuable, meaningful change which makes them continuously stronger. An approach which is sustainable over years of improvement.
However, this is a topic which has been much discussed elsewhere. There’s a perennial debate about formats such as whether the retrospective uses the popular “Start, Stop, Continue” format.
- (For those less familiar with retrospectives), “Start, Stop, Continue” is a popular template. A retrospective is about improvement, and this focusses on where we should add something new, where we should take something away, and what we think doesn’t need change for now.
- (For those more familiar with retrospectives), even a good template becomes stale, and retrospectives are often improved by a different split (what made you Mad, Sad or Glad?) or a different approach (Three Little Pigs – what parts of our product are sturdy bricks, what are sticks and what are only straw).
I’d strongly advise anyone who is in a team running retrospectives to read some of the great advice out there from experts such as Aino Vonge Corry , whom I was fortunate to meet at Agile Cambridge a couple of years back.
1. No Retrospective
The most obvious failure point is for a team not to run a retrospective at all. Not infrequently I’ve seen teams who are not using retrospectives. Assuming that you have introduced retrospectives (and agile development) into the wider organisation, the teams will be aware that they exist. As a leader you need to be concerned why a specific team aren’t running them.
One likely possibility is confidence. The team may have no experience of retrospectives and simply not know where to start. This is a great opportunity for you to both teach and coach. Teaching to explain what these are, what the good practices are and what you would expect from the team. But teaching is rarely enough. Agile teams are self-managing, and need to run their retrospectives. As we have noted, it’s not easy to do this. Even with the right knowledge, the team need to be willing to self-manage. And that’s a coaching opportunity.
One of the biggest reasons (in my experience) for teams not running retrospectives is that they are unwilling to take ownership of their own improvement. Confidence is a large part of that, but understanding self-management is also key. As a leader you need to set expectations around what independence teams should have. Over-reliance on a leader to set goals and to make change will hinder a team’s ability to self-manage.
Another frequent reason that teams don’t run retrospectives is that they feel too busy. If they are constantly pressured to deliver features, something will give. Retrospectives may be the most visible item which has been abandoned, but likely if you look closely there will be a raft of other quality failures as a result of this pressure. As a leader you will need to emphasise the importance and relevance of retrospectives and the importance of quality over quantity of deliverables.
It’s also not unusual that teams have tried but abandoned retrospectives. As a leader you will need to re-energise them. Perhaps they never learned to use them effectively, in which case teaching and coaching in how to run them may be needed. Or perhaps they fell into some of the traps which I explore below.
2. Lack of Psychological Safety
Central to any effective retrospective is a blame free environment. If we are to look openly at what has happened, we need to do so through a lens of learning not blaming. I have worked at several organisations where improvement was limited by people’s desire to “look good”. In some cases this has been lack of confidence by individuals, perhaps “imposter syndrome”. Sometimes this has been fear preventing people from speaking openly. In some cases people have learned that deflecting blame and taking credit for others’ successes is the way to advance.
As an Agile leader, your role is to build a culture where people can speak freely and share ideas openly without fear of repercussions. This will need you to embody these ideas in your own behaviour and to reward them in others. Any retrospective should start from the principle identified by Norm Kerth, often referred to as the “Prime Directive”.
Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand.
“Project Retrospectives” – Norm Kerth
There is a challenge specific to being an Agile leader here. You want to ensure your teams do the best they can with retrospectives. But including leaders in team retrospectives will imbalance the discussion. People may not want to discuss openly. They may wait for guidance rather than giving their own ideas.
Once your team have learned what they need to do, you will need to leave them to self-manage. A retrospective should involve all of the team but no-one in authority. That does include any skills represented in the cross-functional team, including Product Owners, who are part of the team, not managers of the team.
3. Accepting the situation
Another frequent problem that is observed in team retrospectives is that they continually return to the same issues. The retrospective becomes a repetition of things that make the team unhappy. The same items are raised every retrospective.
If a retrospective is a mechanism for change, then raising the same issues each time must show that something is broken. There is no change occurring.
Typically this shows the team have accepted the current situation. However painful, they do not believe they can change it, or that it will ever change. This mindset will not lead to effective retrospectives. A retrospective must start from the idea that the team can make change. Otherwise, why hold the meeting?
Kaizen is about changing the way things are. If you assume that things are all right the way they are, you can’t do kaizen. So change something!
Taiichi Ohno
The team will likely need coaching to support their empowerment. They may also need help to develop actionable steps to make change. Once they see their decisions are having an effect, they will adopt the process and start proposing and progressing more change.
4. Symptoms not problems
There is some skill involved in making retrospectives actionable. When discussing what went wrong (or right) most people’s first reaction is to focus on symptoms. What did we observe occurring? Unfortunately it can be difficult to frame actions based just on these symptoms.
For example, the team completed less than expected in a Sprint. This isn’t desirable – it would be great to align expectations and results. However, what action are we to take? “Complete more in the Sprint” is scarcely actionable. If it were that easy, we would have done it.
In any retrospective, it is important to push towards root causes. Why is the symptom occurring? Lean emphasises the importance of asking “Why“. The Toyota “5 Whys” technique was designed to repeatedly ask “Why” to push down to more underlying causes. The idea of the name is that you may typically need to ask “Why” five times to really understand the root cause. If you haven’t used this approach before, try it – it’s slightly uncomfortable to start with but very effective.
A relentless barrage of ‘why’s’ is the best way to prepare your mind to pierce the clouded veil of thinking caused by the status quo. Use it often.
Shigeo Shingo
Why did our team complete less than expected? Perhaps there were a lot of support issues. Maybe these were because of poor feature design in a recent release. Perhaps that was because of poor designer/developer interaction. And perhaps that was due to investing too little time in initial feature design discussion. Now we have something actionable.
5. Unwilling to experiment
A good retrospective will identify problem areas. But identifying the problem is not enough. A retrospective has little or no value without action. Retrospectives follow the Shewhart cycle popularised by Deming which forms the core of all improvement activity – Plan, Do, Study, Act.
The team need to be informed by the problems they have identified and plan specific actions. The actions are then executed and the results of the actions examined. Has the action had an effect on the problem?
Teams often get paralysed by trying to find “the solution” to a problem. Before they start they want a guarantee that the action they take is “the right one”. However, this certainty is rarely available. More normally, the team need to see each action as an experiment. The team believe the action will head in the right direction. They execute the action and look for a positive outcome.
Teams are often very uncomfortable with the level of uncertainty involved in experimentation. They may also be inexperienced in the basic approach. As an Agile Leader you will often need to coach and support teams in taking this approach.
Good Practices
As we have seen, there are many places where Agile retrospectives can be ineffective. What can you, as an Agile leader, do to support your teams to ensure they make the most of this powerful tool?
It is tempting to get involved in retrospectives directly, attending and steering them. This may have some value initially when you are getting the process to run. However, retrospectives are about team autonomy and achieving self-transcendence. They are not a process which you can run directly without distorting them.
Your role as an Agile leader is twofold. You may need to teach the basics of good practices in order to allow the teams to perform effective retrospectives. This is likely to only last a short time. And then you need to coach the teams to ensure their retrospectives are effective, supporting them in empowerment. Your key role is to ensure a culture of psychological safety which allows them to thrive and develop.
Balancing autonomy and direction is always a challenge in Agile leadership. If you’re concerned about progress, ask the team to talk about the problems they have identified, the experiements they have performed and what they have learned. Use this as a coaching and discussion opportunity.
Leave a Reply