Episode 25 – Projects and agile development
/
RSS Feed
This dialogue explores the fundamental tension between traditional project management, rooted in analysis, and agile methodologies designed for complex VUCA environments.
This podcast is AI-generated based on material from the “Agile Plays” website and book .
Transcript
Welcome to the debate. Today we are opening the hood of the global economy to look at the engine that well for a century has driven it. The business world has been organized around a very specific unit of measure the project. It has a start date, an end date, a fixed budget and uh a defined scope. It is the language of the boardroom, the bank and the balance sheet. But in the world of modern software development, A different philosophy has taken hold. One that claims the very concept of a project is an obsolete relic of the industrial age.
And that is where I have to step in immediately. Calling the fundamental unit of work a relic is well, it's exactly the kind of hyperbolic thinking that leads businesses into chaos. We're debating whether the discipline of traditional project management, which is rooted in scientific management, is being unfairly discarded for are agile methodologies that to many experienced leaders look a lot less like a philosophy and a lot more like a lack of discipline disguised as one.
And that really is the core tension we're exploring today. Is the project with its fixed end points and its, you know, its famous iron triangle of constraints. Is it dead or is agile just the convenient excuse for avoiding hard commitments? I'll be arguing that in our complex modern environment, the project is well, it's the wrong unit of measure entirely. I believe we need to move towards sustainable pace, continuous flow, and managing complexity through adaptability, not prediction.
And I'll be taking the position of, let's say, the expert project manager. I'm arguing for the necessity of clear goals, fixed budgets, and defined schedules. Because without the rigor of traditional planning, what you'd find in the PM book, the project management body of knowledge, business stability is at risk. You simply cannot run a publicly traded company on vibes and sprints. You need to plan the work and then you work the plan.
I see why you rely on that structure. It's it is comforting. But let me offer a different perspective to start us off. The traditional model, it relies on the idea that we can predict the future. And I want to suggest that in software, we're not building a car on an assembly line where every single bolt is predefined. We're exploring a new continent.
Okay, let's ground this in reality before we start wandering off into metaphors about exploration. Business relies on predictability. If you look at the project management inst stitutes definition a project is a temporary endeavor undertaken to create a unique product service or result and the key words there are temporary and unique you have a specific amount of money let's say a million dollars in a specific amount of time 6 months to deliver a specific thing that is the contract
which sounds incredibly responsible on paper
it is responsible it's about control I mean if you don't have a work breakdown structure a WBS where you decompose the total scope into manageable chunks. And if you don't have a Gant chart to track the variance against that plan, you're flying blind. The goal of management, and this goes all the way back to Frederick Winslow Jaylor and the origins of scientific management is to analyze the work, determine the one best way to do it, and then ensure the workers execute that plan. It's a reductionist approach. You take a big complicated goal, you break it down like a clockwork mechanism, and then you assemble the parts. If you do that correctly, you control cost and you control schedule.
You know, I'm I'm not entirely convinced by that because it feels like it assumes we're operating in what we'd call a complicated environment. You mentioned a clockwork mechanism. That's a perfect analogy for a complicated system. It has many parts, but the interactions are predictable. You turn a gear here, a hand moves there. You can create a blueprint for a clock. But software development, it's not complicated. It is complex. That sounds like a semantic distinction to excuse poor planning. What's the practical difference?
Oh, it's a fundamental distinction, not a semantic one. In a complex environment, uh, like the one described by the Stacy matrix, it's a framework for selecting the right management method. Certainty and agreement are low. Cause and effect are only visible in retrospect. You can't pre-plan the solution because the solution it emerges as you build it. When we treat software like manufacturing or construction, we fail. My argument is that The goal isn't a unique product delivered once. It's sustainable development. We're looking for a continuous flow of value. We aren't trying to finish a project. We're trying to keep the software alive and valuable indefinitely.
But organizations don't budget for indefinite. They budget for results. You can't go to a CFO and say, "Give me money forever and I'll give you code forever."
Right? Which is why I would frame agile not as the absence of management, but as something closer to program management than project management. management. A program deals with temporary flexible organizations, teams that coordinate to deliver evolving objectives. We aren't getting rid of structure. We're just changing the structure to match the reality of the work. I want to drill down on this idea that projects simply don't exist in the agile world. I I find that baffling. Look at the Sydney Opera House. That is a classic example of a project. It had a goal, create a new opera house. It had a team and it had a budget. How Can you possibly manage something of that scale without the structure of a project?
That is an interesting point though. I would frame the Sydney Opera House as the perfect counterargument to your position. Do you know the numbers on that project?
I know it went over budget, which happens when controls fail.
It didn't just go over budget. It was estimated at $7 million Australian dollars and expected to take four years. It actually cost $12 million and took 14 years. That is a cost overrun of what 1,400% and a schedule overrun of 350%. And why? Because they treated it as a project with a fixed plan. Even though they were dealing with massive uncertainty, they didn't know how to build the sales structurally. They hadn't done the ground surveys. The design was changing constantly.
So your argument is that because they failed to plan for the unknown, planning itself is the problem.
No, my argument is that because of the high uncertainty, they should have treated it as a learning process, not a rigid execution path. In a traditional project, you bring the team to the work. You form a team, do the job, and then you disband them. That is so inefficient, it loses all that knowledge. In agile, we bring the work to the team. The team is stable, it's permanent, and the work flows to them. If the opera house had been built with an agile mindset, they would have tested the structural integrity of the sales before committing to the full schedule, pivoting the design when the physics didn't work rather than forcing a plan that was already doomed to fail.
See, I come at it from a different way. If you just flow works to a team, you lose the urgency of the deadline. The structure of the project creates the pressure to deliver. Without a deadline, work expands to fill the time available. That's just Parkinson's law.
But is it the right pressure? Look, if you're navigating a known ocean, you need a map. That's project management. But if you're exploring a new continent, a map is useless. You need a compass and a sounding line. You need to probe and sense and respond. That is agile.
Let's talk about the context of this new continent. The agile manifesto was written in 2001. That is a quarter of a century ago. It was what 17 guys in a ski lodge in Utah.
A very distinct demographic. Yes.
Right. And they were reacting to the heavy-handed documentcentric processes of the '90s. But is Is that manifesto still relevant? It feels like a fossil. The world has moved on. We have tools now that didn't exist then. It feels like some people cling to the manifesto like a religious text rather than a practical guide. I see why you think that, but I would argue the shift in technology makes the manifesto more relevant, not less. I mean, think about 2001. Teams were in offices physically located together, often working on slow links to central servers, deploying software took days, sometimes weeks. You had to burn it onto a CD.
Exactly. Now we have remote work, global teams, instant communication. The constraints are gone.
And we have cloud deployment where code can move to production in minutes via CI/CD pipelines, continuous integration and continuous deployment. The speed of delivery has increased exponentially. So the manifesto values responding to change over following a plan. In 2026, when you can deploy a change in 10 minutes. The ability to respond to change is your biggest competitive advantage. If you're stuck in a scientific management cycle where a manager has to approve a plan before a worker can code it, you're dead in the water. The feedback loop needs to be instant. I'm sorry, but I just don't buy the scientific management is dead just because internet speeds are faster. Taylor's core insight was the separation of planning and doing. You have people who specialize in strategy and resource allocation, managers, and people who specialize in execution. That division of labor creates efficiency. You seem to be suggesting that everyone should just do everything
in knowledge work. The doing is the planning. You can't separate them. When a developer is writing code, they're making all these micro decisions about architecture and logic that a manager can't possibly see. The Stacy matrix shows us that traditional management works for high certainty and high agreement. But as soon as you step into low certainty, which is almost all modern software development, that separation of planning and doing just breaks down. You need the people doing the work to be making the decisions because they're the only ones close enough to the problem to understand it.
But that sounds like risk and that is my biggest issue with this entire approach. In traditional project management, risk is a known unknown. I know a supplier might be late, so I have a mitigation plan. I have a contingency budget. I have a risk log that I review weekly. Agile seems to just ignore this formal risk management entirely. It feels like you're just hoping for the best.
That's a compelling argument, but have you considered that agile is designed specifically for unknown unknowns?
How can you design for something you don't even know exists
through feedback loops? In a traditional project, the biggest risk is that you spend 6 months building something that uh isn't what the customer needs or simply doesn't work when you turn it on. You don't find that out until the very end. The big bang integration. Agile manages risk by delivering working software every 2 weeks. We don't mitigate risk with a document. We mitigate it by revealing the risk early through the product itself. If the code is broken, we know on Tuesday, not in 6 months.
But you're only talking about negative risk, things going wrong.
Exactly. That is the shift. Traditional PM focuses on negative risk, avoiding variance from the plan. Agile focuses on positive risk, on opportunity. Because we're releasing constantly, if we see a market shift, we can pivot. We can capture value we didn't plan for. You can't do that if you're rigidly working the plan and treating every single change as a failure of discipline. Okay, let's talk about that pivot because to me that just sounds like scope creep without a change control board. And this leads to the most frustrating concept I hear from the agile camp. It's a hashtag I see on LinkedIn all the time. #nowestimates.
Ah, yeah. Yes, I thought we might get there.
It drives me up the wall. If there is no estimate, there is no commitment. If you hire a contractor to build a deck and they say it'll be done when it's done, and I won't tell you how much it costs, you would fire them on the spot. Stakeholders need dates. They have marketing campaigns to run. They have board meetings. # nostimate sounds like pure chaos. I'm not convinced by that line of reasoning because it misunderstands what an estimate actually is. An estimate is just a probabilistic guess and usually it's a bad guess, we try to turn that guess into a promise and when we miss it because software is complex, we blame the team.
So the solution is to just stop guessing and hope the stakeholders don't ask.
No, the solution is transparency and flow. Let's talk about the backlog. The term actually comes from a medieval analogy. A backlog was a large log put at the back of a fire to smolder continuously, ensuring the fire never went out. It wasn't about starting and stopping. It was about maintaining the heat.
That's a quaint history lesson. But how does it help me tell the CFO when the product launches?
By using velocity. Past performance predicts future performance. We don't guess a schedule. We measure how fast the team is actually delivering value, their velocity, and we project that forward. If the work is right-sized, meaning broken down into small enough chunks to fit in an iteration, strict estimation becomes waste. It's Muda in lean manufacturing terms. It takes time to generate an estimate, but the estimate itself adds zero value to the customer. The customer wants the software, not the Gant chart.
Stakeholders don't care about MUDA. They care about dates. You're asking for a level of trust that most organizations simply do not have.
But trust is earned by delivery. If I deliver working software to you every two weeks, you stop asking me for a Gant chart for 6 months from now because you see the progress. You see the fire burning. You see the deck being built plank by plank.
I maintain that there is a fundamental clash here. You want to execute based on flow. I want to execute based on commitment. And this bleeds into the team dynamic as well. You mentioned earlier that the separation of manager and worker is bad. But I defend the hierarchy. Efficiency comes from clarity of command. Someone has to be the captain of the ship.
And I would counter with Takiuchi and Nanakaga's famous paper, the new new product development. game. This is the paper that actually inspired scrum. They contrasted the relay race with the rugby approach.
The relay race being the traditional functional silos.
Yes. I do my part, say the design. I hand it off to you, the developer, and I wait. It's full of delays, handoffs, and loss of context. In the rugby approach, the whole team moves down the field together, passing the ball back and forth. They're self-managing. They figure out how to get past the opposition in real time.
But rugby is messy. People get hurt. A relay race is clean. It's precise. You can time each leg.
Rugby is adaptable. If the defense shifts, the team shifts. In a relay race, if the person ahead of you drops the baton, the race is over. We promote what we call management as a service. The manager isn't there to direct tasks. They're there to remove blockers. They support the team, not control it.
That sounds nice, but it ignores human nature. People grab toward comfort. Without a manager driving the schedule, the pace slackens. You need that external pressure.
Actually, the research suggests the opposite is true. Autonomy drives motivation far better than control. And that brings me to the surfing analogy. In a traditional project, the backlog of work feels like a weight crashing down on the team, a title wave of obligation. But to an agile team, the backlog is the wave. It's the energy. You ride the wave. If you maintain a sustainable pace. The work drives you forward. It's not about burnout. It's about momentum.
I'm listening to you and I am trying to bridge the gap. I will acknowledge that for truly complex software with high uncertainty, maybe the new continents you mentioned, rigid Gant charts are going to fail. You can't plan for things you can't see.
That is a significant concession. However, I remain skeptical that a lack of fixed projects can satisfy organizational finance and frankly board requirements. The project is how money is allocated. It's how ROI, return on investment, is calculated. You can't just have a forever stream of money flowing to a team without checkpoints. Scientific management brings a necessary order to the chaos of capital allocation. And I accept that the business needs to know where its money is going. Agile is not about the absence of planning. It's about adaptive planning. My final point is simply that when we shift from a project focus to a product and team focus. We allow for repeatability. We stop treating every single initiative like a panic attack. We achieve repeatable delivery at a sustainable pace which prevents the burnout that destroys quality in the long run. I think we can agree on one thing. The agile manifesto is a set of values, not a rulebook. It's not scripture. And simply buying Jira doesn't make you agile.
Absolutely. And the ultimate goal for any scaling organization isn't doing agile just to say you're agile. It's about delivering value.
And whether you do that with a Gant chart or a Kanban board might matter less than whether you actually understand the problem you're trying to solve.
That is the perfect place to leave it. We want to thank you for listening to this rigorous debate. We hope it has sparked some thinking about your own work.
So ask yourself, are you navigating a map or are you exploring a new continent?
Leave a Reply