What's the business value of self-organization?
Out with top-down planning and task assignment! The ideal software delivery team is self-organized. But why?
I started my consultancy career in large enterprises that are slow to adopt new industry trends. So, while Scrum was almost a teenager, those companies were still manager-driven. Analysts would gather requirements and turn those into specifications, and only then would we developers get involved. The manager told you what to work on. That's the industry I got started in.
So, by the time I became a project manager, the agile software development movement was getting ready for her Quinceañera. Yet, I was still taught that it was my job to run the show. But not for long...
I remember sending an invite for a getting-to-know-you meeting to a new team, only to be met with a lecture about maker/manager schedules. Pushing back against a manager would have been unfathomable in the early days of my career! Something had clearly changed. Engineering teams had become self-organized rather than manager-driven.
The job description of a software engineer had changed dramatically. Gone were the days of the people-shy nerds in cubicles turning a tome of requirement specifications into arcane code. The contemporary developer spends only a fraction of their time in their IDE. Ideation, documentation, feedback, planning, communication, QA, and user interaction are all part of the job these days. They are expected to actively push back against wrong decisions, and their input is at least as important as their manager's. Scrum has popularised the notion that the team, not the manager, decides how to operate best. Overall, that's a good thing.
Some people refuse to accept this reality. On the one hand, the just-let-me-code developers pine for the days when their job was 99% coding. While that kind of job still exists, you'll have to look hard for companies that haven't adopted agile software development.
On the other side of the spectrum is a group of business people and managers not on board with the new role developers take up.
To them, many of the responsibilities that have become typical of software developers are just perks—things to keep the troops happy. They begrudgingly accept the market reality, but deep down, they believe more typing is better. To them, self-organization is being inclusive and polite. They don't see it as the competitive advantage it truly is.
It doesn't help that our vocabulary is coated in fluffy HR language.
We're often told that these "perks" exist because a "happy worker is a productive worker." That's only true to an extent. We do a bad job of explaining the tangible benefits of an engineer's new position in the organization.
As an example, the value of a distributed workforce lies not in the happy worker's mantra but in the broader access to talent. Hiring for an office is hiring on Nightmare Mode.
The value of a Guilds-style setup is not that the nerds enjoy the camaraderie of their monthly book club. It's that team-level innovations can percolate throughout a large organization.
So, what's the value of self-organization?
The problem with what we now call Waterfall was that it was expensive, slow, and inefficient. After spending so much time and cash on gathering, analyzing, specifying, designing, developing, and testing, there were always last-minute tweaks to be made.
In the last few weeks, invariably, something would be uncovered that threatened to derail the project. The developers would be called in and had to devise some workaround/hack to prevent that. The nineties saw us asking the question: If, at the end of the ride, we still have to save the day, what's the point of all those previous phases?
The developers were not smarter or more creative than the rest of the organization. So, why was it always them stepping up? Here’s my theory: In a methodology that firmly believed in splitting thinkers from doers, devs turned out to be both. They have read the specs and not only understand what the system does but also what it is supposed to do. We learned that those who can actually build the product are in the best position to solve its problems.
The immense success of the agile software development movement lies in activating that problem-solving power. And that can't be done with engineers who just blindly follow orders.
In fact, it makes sense for those problem solvers to lead. In a manager-driven approach, the team follows a single point of failure that is, by definition, not in the best position to lead. Even a manager with a technical background doesn't understand the product the way the team does.
So rather than telling teams what to do, we give them problems to solve and the mandate to come up with the solution.
Yes, discussing a feature with a self-organized team can be frustrating. You ask them a question and get five back! Why can't they just build what I ask? Having been on the receiving end, it often seems like they debate for the sake of debating. However, those questions and pushback early on would otherwise translate into production errors at the end of the ride. Those are expensive to fix.
Companies that rely on analysts, product owners and solution designers to spell out the specification, try to optimize for maximum amount of coding. They pre-chew the solution in the hope of speeding up the development. In doing that, they make the same expensive mistake as back in the Waterfall days.
Self-organization means putting our best problem-solvers in a position to work their magic. In exchange for delegating the solution details, you get a system of reliable teams that can scale without adding an army of fallible managers, coaches, and masters.
When teams can tackle high-level problems independently, we get better products with less overhead. All we need to do is aim them at the right problems.
That's as much business value as you can squeeze out of a team.