Three cool metrics to augment velocity
What gets measured gets managed. It is a cliché from the days when MS Project reigned supreme.
There's also a lot of truth in it.
Project Managers used to track all kinds of numbers in bespoke Excel sheets that allowed them to show "progress" to the board. Each column in those spreadsheets would track a metric. Planned value, earned value, actual cost... We tried to measure reality and condense it into a weekly report. Since these metrics were inherently flawed, they gave an unreliable picture.
When a task was estimated to take ten days, and we had already spent 8 of those, we marked the task as 80% done. The only thing we could say for sure is that we burned through 80% of the budget for said task. It told us nothing about completeness.
Story Points were launched as an antidote. Instead of estimating in days, we estimated in arcane Fibonacci numbers, making Excel magic nigh impossible.
But because managers need to manage, they started looking at another hard-number metric: the dreaded Velocity.
A big part of a manager's job is to report progress; without valuable metrics, they will latch on to whatever they can get.
Velocity has become the main progress metric in many companies. But just like in the past, it doesn't tell the complete story.
It's very popular these days to push back against metrics-driven management. Managers need to "understand" and "get out of the way". Yet data-driven progress reporting is vital. We are pathological optimists and need hard numbers to confront us with the truth: did initiative X really speed up our delivery?
So, today I would like to discuss three hard-number metrics that are easy to track and can help give a better picture of where the teams are at. They aren't alternatives to Velocity but can complement it to paint a clearer picture. Velocity in Scrum is "how many Story Points did the team deliver last sprint?". In more Kanban-style environments, we might track "How many tickets did the team deliver last month?" These three metrics should also be tracked for the same timebox. This allows us to see progress over time.
Focus
What it tracks:
Doing what we say we would do is a superpower in business. But distractions and multitasking make this rare. We should be working on a roadmap item or sprint goal but get distracted by all other kinds of work. Suddenly, three months passed, and we didn't complete that goal. To do what we say we would do, we need to focus. The Focus metric is a good indicator of how distracted the team is.
How to measure it:
At the end of the timebox, we count the number of delivered tickets that worked towards the roadmap item and divide that by the total number of delivered tickets. This gives us a percentage. If the team delivered 10 tickets and 6 were in support of the sprint goal, we can say that the team is 60% focused.
Focus = focused delivery / total delivery
How to use it:
We don't need to aim for 100% Focus. That's unlikely. But we should try to increase the level. It's also a great way to compare teams. If most teams are in the 40% range and one stands out with 65%, there might be some lessons to learn.
Increase Focus to increase roadmap predictability.
Quality Decay
What it tracks:
A common argument against Velocity is that it doesn't tell us much about the quality of the delivery. Churning out more broken features will make Velocity go up! Product quality is a subjective and notoriously hard-to-measure attribute. But as usual, we don't need to be accurate. We need to be consistent. We'll track a number that gives us enough information to see if our quality initiatives are working. We want to track the rate at which defects are introduced in the system.
How to measure it:
We count the number of bugs reported in production during the timebox. This is a rough metric as it doesn't mention severity or resolution time. It also comes with a delay of at least one iteration. I advise against trying to figure out in which sprint or release a certain bug was introduced. That is a lot of work and doesn't give us much more info. Keep it simple and count the reported production bugs at the end of the timebox.
Quality Decay = reported number of bugs in production during timebox
How to use it:
This number should go down! Aim for zero on this one. If teams have a high level of Quality Decay, we need initiatives to increase quality. Test automation, slower delivery, faster feedback loops... This metric will tell you that you're improving and how much better you're doing over time.
Customer-facing teams often get a higher number of bugs than platform teams. It doesn't make sense to compare this metric with other teams.
Stability
What it tracks:
Plans are easy to make but hard to maintain. Urgent requests, changed priorities, customer support, critical bugs... Life seems to get in the way of sticking to the plan. Teams can do many things to push back against these plan busters, but the first course of action is to measure just how bad the situation is. It's the opposite of the Focus metric: we track how much unplanned work we encounter.
How to measure it:
At the end of the timebox, we count the number of "expedited" or "emergency" tickets that were delivered. If the team delivered 10 tickets and 2 of them were emergencies, we can say the team is 80% stable.
Stability = (total delivery - emergency delivery) / total delivery
How to use it:
No team is 100% stable, but structural improvement is needed if the house is on fire every week. Stability is a great metric for those teams with ever-changing priorities. It can be measured and improved over time by organizational changes, quality improvements, and better stakeholder management. It also allows us to measure exactly how much better we're doing.
Here are a few cool ways to combine these metrics:
Let's say a team is 40% stable. That tells us we need to lower our delivery ambitions and plan them at only 40% of their Velocity. That way, we keep room for the "emergency" tickets that are sure to come.
If our team introduces a Zero Bug policy, this will have an obvious negative impact on Focus. That's OK for a while as long as we see Quality Decay go down as well. We don't want a team introducing loads of bugs and then fixing them. We want to stop the bleeding. The interplay between Focus and Quality Decay will tell us exactly that.
We can expect quite some pushback from stakeholders if we make it harder to launch can-you-quickly-do-this-one-thing requests. Convincing them to work on one problem at a time rather than multitasking can be difficult. These metrics give us the tools to show them the benefits. There's a clear relationship between high Focus, high Stability, and high Velocity. If we can show them these hard numbers every few weeks, they'll be easier convinced.
A team with high Velocity, Focus, and Stability that manages to keep its Quality Decay low, is a hyper-performant team.
These three metrics, combined with Velocity, give us a rich set of tools to steer the teams in more productive directions.