Two estimation traps to avoid
When I joined a past organisation I was surprised by the approach the teams were taking for tracking work. They would always look on track up to the expected end date, and then suddenly everything would fall apart.
When they started a new feature, they would estimate the amount of work. Then they would track the amount of work they had done on the feature. They would report “progress” as work done divided by total work. Where’s the problem?
With fixed teams, work done is equal to team size multiplied by elapsed time. Total work is team size multiplied by expected time to complete. So their progress measure was just measuring elapsed time. No wonder it increased smoothly.
Most agile teams aren’t making that mistake. But there are still a couple of common errors which affect how well a team understand if they are likely to deliver what they expected.
Recalculating estimates
Most agile teams will be estimating future work and using velocity to assess what is a realistic message to customers. I include both Story Points and Effort Estimation as classes of “estimating”. As I describe elsewhere, I use velocity in both cases. Alternatives such as NoEstimate approaches are discussed in detail in other articles.
I often find myself discussing with teams whether and when estimates should be updated. Generally the team feel that the most accurate data should be captured, and that therefore estimates should always be updated to the best understanding.
However, there are two possible scenarios. In one of these you should update estimates and in the other, perhaps counter intuitively, you should not.
The first is when you have future work on the backlog. You have estimated that work, but you now have a greater understanding of what is involved. In this case you should re-estimate the future backlog task. This task will prove to be more work than you previously thought. You have a stable velocity and increasing the estimate will correctly show that it will take longer.
The alternative is when you have completed the work. Often the team feel that you should capture the most up to date data about the task. For example if it was estimated as 5 days effort and took 10 days of effort to complete. If you use Story Points, you can’t update after completion as there is no measure of “how many Story Points did it take” so this applies solely to Effort Estimation.
Imagine a team of 5 engineers start work on a set of tasks all estimated as 5 days effort. After two weeks they have completed 5 tasks. That means 25 estimated days completed in 50 working days, or a velocity of 0.5 estimated days per working day. That’s useful information which tells us that the future tasks will be achieved at the same rate.
What happens if you update the estimate after the work is done? After two weeks they have completed 5 tasks, but the estimates have been increased to 10 days to match what really happened. That now means 50 estimated days completed in 50 working days, or a velocity of 1.0 estimated days per working day. A rate which does not match the estimates for future tasks.
The most important outcome is to get a stable and predictive velocity. Changing the parameters so completed work is measured differently from planned work prevents the completed data being used predictively.
Remaining Work
Measuring Remaining Work on tasks is similar in many ways to recalculating estimates. Intuitively it feels like it must be better to collect more data. Surely knowing more must allow us to make better decisions?
Firstly of course, we need to accept that all data collection has a cost. If we have an Estimation field (Effort or Story Points), we populate it when we plan. We may, as discussed above, update it when we replan future work. But it is largely static. By contrast, a Remaining Work field is updated regularly throughout the development. The team are continually thinking about how long the current task will take to complete, and that is an overhead.
In our Lean learnings, logging Remaining Work is Waste. Remember this does not mean it is inherently bad, just that the customer does not pay for it. Skins for the bananas, if you like. So there needs to be value in measuring the data.
The general argument is that by subdividing a task and logging Remaining Work on the task, we get a clearer picture of progress. Instead of saying we have completed 5 tasks out of 10 and are 50% complete, we can now say we have completed 5.4 tasks out of 10 and are now 54% complete. But does that added accuracy help?
Intuitively we tend to consider predictive problems in our examples. If we have a task which we know will take 10 elapsed days, then after 7 elapsed days we know we are 70% of the way through. But like my team above, that is simply measuring the flow of time. If we only think it will take 10 elapsed days, then the extra detail is misleading. We are being more precise but not more accurate.
It is better to be roughly right than precisely wrong.
John Maynard Keynes
We only know we have made progress if we complete work. Working software is the primary measure of progress. If we want to measure at a higher level of accuracy we should subdivide the work. In Lean terms, we want to reduce the batch size.
The problem becomes worse if a task takes longer than expected. Imagine our 5 day task ends up much longer. We have been updating Remaining Work throughout, and we’ve been working on this for 50 days. But that is not 50 days of progress. It has taken 50 days to do a 5 day task. If you redefine it as a 50 day task you are back with my original team just measuring time as it ticks past and calling it progress.
Good practices
As an Agile leader you need to be confident in your balance of planning and reacting to change. While the Manifesto says “Responding to change over following a plan“, planning is still important. Velocity is an important tool to understand and respond to change beyond the most minimal of planning horizons.
Your teams need to understand good practices around estimation and velocity in order to make predictions in the uncertain Agile environments. Some of the good practices are not intuitive, and you will likely find yourself coaching your teams to help them avoid some of the traps.
Leave a Reply