“It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so. — Mark Twain"
Developing software systems -and we dare to say all systems– implies mastering the arts of uncertainty and complexity.
- Uncertainty , because it is impossible to know everything you need to know at the outset of a project. We have to make decisions even when not all the necessary information is available or when the information available is ambiguous.
- Complexity , because any non-trivial process will involve many stakeholders in constant flux. And the context for those stakeholders will not have an identical precedent. We tend to think that if there is complexity in a project, there’s something wrong with it, that our job is to “simplify” it. In fact, quite the opposite is true, if we don’t acknowledge that complexity is a given and learn to navigate it, we will be making the wrong kind of decisions. (By complexity we mean a situation where there are no obvious cause-effect chains and where the parts interact in ways that don’t allow a reductionist approach).
A fixed scope contract (one where there’s a prescriptive list of well specified concrete deliverables) does not allow dealing with uncertainty and complexity. However, in many of the projects we have worked with, a fixed-scope/fixed-price contract tends to be the preferred way for the customer. That is understandable: “if I know how much I have to pay and what I’ll get in return, there’s no risk.”
The problem is, the three clauses in that assumption are completely false : you don’t know how much you will pay, you don’t know what you will get in return, and there is quite a bit of risk.
Let me try to explain why.
Requirements and specifications
A fixed-scope/fixed-price only works well when what you are buying is a pre-made thing or a commodity. You already know what you are getting, and you are confident that’s what you needed. But that is not true with a software system.
To understand why, let us split what defines a system in two parts: requirements and specifications. There’s a fine line between one and the other and many times requirements end up blurring into specifications. For this analysis, let’s define requirements as the explanation of the problem and specifications as the description of the solution to that problem.
Let’s use an example:
Requirement : Reach a 10-meters-high platform.
Specification : Build a strong ladder 10-meters high.
Now, for any given requirement/problem, there are multiple specification/solutions that could work:
Requirement : Reach a 10-meters-high platform.
Specification : Build a jet-pack that can fly up to 10.5 meters.
And here is where fixed-scope/fixed-price projects run into trouble: to be able to estimate a price for a fixed scope, we are forced to assume certain specifications.
Nobody can estimate a precise budget using solely requirements (unless you imagine the most complicated solution possible for the problem and pad your budget for that). A problem, no matter how well you define it, does not have an associated effort: the cost of a ladder is very different from that of a jet-pack.
When you create a detailed list of specifications upfront –e.g. screen X should allow a user to input this value and then…–you are hiding under the rug the unavoidable reality that you do not know what is needed. You will not know until you do user research, build a prototype, test the prototype, iterate on the concept, give it to users, and see how it works in real life. Creating the illusion of certainty around the specifications is very dangerous.
As a customer, when you ask for a fixed-scope/fixed-price proposal, you are creating an epistemological conundrum; the provider is forced to choose between two options:
- To give you a proposal with an accurate lie –i.e. a fixed cost estimate for the detailed scope.
- To convince you of the truth of the uncertainty–the fact that you don’t know what exactly you will be building together.
The good news is that there are better options depending on your situation:
Situation #1: You have a fixed budget
In this situation, the best possible course of action is to share what that budget is. The traditional reticence to do that goes like this: “If I share the maximum budget I have for this, you will stretch your proposal to charge me for exactly that amount!”. But that is exactly how you should frame the problem internally, i.e. what’s the best solution we can accomplish for my problem, within this budget.
If you are concerned that there might be more cost-efficient ways of solving the problem, then you can ask for two alternatives: what can we do with $20 to address this problem and what can we do with $40? Then compare and decide if the cheaper solution is good enough.
Situation #2: You have to fundraise
In other situations, you might not have available funds for the project and need to go out and fundraise for it. In that case, the best way is to do a small initial engagement that probes into the problem and exposes some of the potential solutions. If you structure it well, you will have two useful outputs for your fund-raising efforts:
- A better understanding of the problem space you are dealing with, meaning you’ll get a better estimate of the effort needed.
- Some form of prototype, mockup, visual material to better communicate the problem you are tackling and the thinking that went into possible solutions.
Situation #3: You have an ongoing funding source
Some business models might mean that you have at your disposal a certain amount of funds you can invest into your problem. This is a great situation for an agile contract: you define the size of the monthly effort you can cover and you iteratively build your system until you are satisfied with it. The most common critic to this is: “but I’ll never know when I will finish!”. That is not true: as soon as you see the result of the first iteration (and that should be soon, less than 1 month if you are working with a good agile organization), you’ll be able to extrapolate very easily how many of those cycles you need until you get the system to where you want it to be.
Some good practices
Regardless of what route you choose to go, or even if you really have no choice than to go with a fixed-scope/fixed-price contract, here are a few practices that we’ve found to be useful.
Accept uncertainty
Your understanding of reality is and will be fuzzy. You can work to increase the definition of the boundaries that are important to you, but if you are trying to be certain about everything, you are doing it wrong.
Roughly right, not precisely wrong
Understand the difference between precision and accuracy: under uncertainty, a higher precision usually implies less accuracy.
The estimate for the duration of a project can be very accurate if it’s allowed to be imprecise: “we will have this ready between 2 and 5 months”. Compare that to a precise but inaccurate estimate: “we will have this ready in 2.5 months”. Had the project lasted 4 months, which estimate would have you preferred in retrospective? If at certain milestones in the project you need more precise numbers to make a decision, invest on reducing the underlying uncertainty.
Estimate under uncertainty
Learn to estimate under uncertainty: everything can be estimated at any stage of knowledge, provided you are willing to accept a certain lack of precision.
Understand your assumptions
For each thing you think would work, there are always many assumptions that you are making. Make a list of them. Stop only when you reach: “laws of physics will remain constant”; that’s a safe assumption.
Challenge your assumptions
Come up with ways to test early on whether those assumptions hold water. It’s much better to change the building design on the drawing board if the ground will not support it, than to wait until it falls when you build it.
Embrace ambiguity
Delay decisions until the last responsible moment. This is a delicate art; you don’t want to block progress by stalling in a decision. But if that decision is not in the critical path, then it’s much better to wait until a better understanding of the problem surfaces.
We hope this contributes with a few ideas both for providers and customers to find better ways of contracting for complex systems. I’d love to hear other ideas, leave your comments!