Before the industry-wide adoption of Scrum-style workflows, I was a project manager. We handled IT projects pretty much like every other project in every other industry. We broke the assignment into work items, estimated, made a plan, assembled a team, and went to town.
If you think about it, the typical agile software development workflow is pretty exceptional. It's hard to find another industry that delivers products iteratively like we do. Bakers don't make a cupcake and add layers on top to evolve into a full-blown wedding cake. No, they design the cake and build it. Construction workers don't start with a tent and evolve that into a mansion.
The nature of software development allows us to switch out the foundations after the roof has been built. We can be ridiculously flexible because we're shuffling electrons, not bricks. Gravity doesn't weigh on software. It makes sense that we designed a set of frameworks to make the most of that advantage.
But waiting outside the bakery or construction site, there is a customer who doesn't like surprises. The work can only start after they agree on the scope and price. Customers love projects because they take away uncertainty. They tell us what will be delivered and when. Nobody in their right mind would give their baker a budget and carte blanche for a wedding cake without some guarantees on the outcome. Nobody builds a house like that.
Our teams also serve customers, and they love agile software development's flexibility. Stakeholders can change their minds at no cost. That allows them to act on new insights instantly. But that doesn’t mean they stopped wanting the predictability offered by a project-style workflow.
Where we have a clear expectation of when the wedding cake will be delivered, we can't get the same answer regarding software. We generally don't know when it's done. We often don't know what "it" and "done" even mean. The number one thing stakeholders hate about agile software development is that it makes everything so unpredictable.
This love-hate relationship creates friction and uncertainty. Stakeholders often have a hard time trusting their teams because those teams don't seem to keep their word. They'll promise to build some feature that's of strategic value, and two months later, they didn't even start.
The way most companies fix this is by putting a manager on top of those teams. Someone who’s responsible for delivering the outcome. A leader to keep the team in check. It's a tried and tested solution that works wonders for projects.
Every construction site has a foreman. Every software team used to have a project manager.
The foreman is appointed to keep the team on track and answer When questions. They’re the single point of contact for the stakeholders. If priorities change or a certain work item starts slipping, the foreman assigns construction workers and updates the plan. That’s classic project management and requires a lot of upfront planning.
Most teams these days still have some kind of foreman. The job title of this implicit manager varies wildly between companies, but their assignment remains the same. Some teams have a Product Owner who has to present status to the board. Others have a Scrum Master who updates projections through burn-down charts. There are Engineering Managers, Lead Devs, and Delivery Leads whose job is to give the stakeholders insights into the state of the work.
While these people are assigned to keep the team in check, that's usually not the job they choose to do. Scrum Masters notoriously take pride in claiming they're not managers. Yet, most of the time, they are. Burnout rates are exceptionally high, and job satisfaction is often abysmal.
I often see Product Managers assigned to babysit engineers — a counterproductive and thankless job. They are so busy discussing Jira tickets and preparing User Stories that they can't free up time for their actual work.
It’s just not how software should be built these days. You can’t skip the big planning phase and then assign someone to keep the team on schedule. There is no well-defined schedule! All they can do is short-term firefighting at best.
The irony is that all this pain doesn't result in a better environment for delivering quality software. Engineers generally dislike their appointed micromanager. We all know what happens when a team first gets to see the workload for the coming sprint. They ask questions. They challenge assumptions. They tweak estimates.
Sprint planning meetings often look like a sadistic game show. The manager has prepared some stories and presents those to the team. They then challenge or flat-out reject most of those tickets. At the end of the two-hour slog, the team is left with vaguely defined tickets that still need some refinement. If you're that foreman, you see your work go to waste, and you feel your blood pressure rise.
Of course, you can't plan like that. Of course, you can’t do what you said you would do.
You’re herding cats
The solution to this common problem requires a shift in mindset.
There’s nothing wrong with adding a manager to a team. Someone needs to report progress and be the point of contact for stakeholders. That’s healthy.
The problems start when this manager is the sole responsible for delivering the outcome without the safety net of a good plan. When they feel like they need to prepare and assign the work so the construction workers can be productive. When all they can do is fight short-term fires, yet get grilled about the long-term outcomes.
Rather than appointing a manager to keep the team in check, we need a representative of the stakeholder's voice—someone who defines and prioritizes the problems and leaves the solutioning to the engineers. We need to set up systems to support such a workflow.
I like Iterative Buffer Planning because it works at the level of problems. It leaves the solutioning to the engineers while supporting long-term plans.
Whatever system you set up, avoid adding a manager to keep the team in check.
We’ve tried that, and it’s worse than BDUF.