Our technology capability continues to grow at an exponential rate. From Moore’s Law to Gilder’s law, well-known trends provide ample evidence that there will be no plateau in our use of technology.
The social effects are just as astonishing: ten years ago the job of app developer, big data analyst or cloud services engineer didn’t even exist. Any of today’s children destined to work in the tech industry in twenty years time will be working with languages and tools of which we currently have no concept.
This continual journey into the unknown is what makes working on digital projects so exciting; and what makes estimating digital projects so tricky. For most people, knowing how long it’s likely to take to walk to the nearest bus-stop is easy because it is they’ve done it before. However trying to guess how long it might take to walk 50 miles due North proves far trickier as:
a) it’s not something you’re likely to have done before
b) the margin of error is far greater due to the time and distance involved.
Most software projects of a reasonable size deal continually flirt with the unknown. This makes estimation hard and provokes the question of how to structure a contract when the exact approach is yet to be worked out.
Traditional client/agency models have centred round a fixed price model, whereby the development team are held accountable to their initial estimates despite not yet possessing all of the information they need. While you can add in contingency to mitigate risk there are still some key flaws with this model:
- It’s very hard for client and vendor to avoid an optimism bias at the start of any project and this subsequently places most of the risk on the vendor
- If a client expects fixed price and fixed timescales then the only thing that can shift is quality – leading to a product that nobody wants.
- This type of contract can lead to an ‘us versus them’ scenario where budgetary pressure gives way to hardened attitudes between client and vendor about ‘what was said’ or what constitutes a change request – essentially it can be a collaboration killer
That’s not to say fixed price can’t be done well. Some companies manage this model extremely well. However, it doesn’t push you into exploring the unknown but is more likely to lead you into seeking out safe ground. This can kill off creative ideas before they’ve been given a chance. Lastly, if you speak to any PM about ‘fixed-price projects’ you’ll occasionally see an ‘a thousand yard stare’ come across their face as they remember the hardship such shackles can sometimes impose on running an innovative project successfully.
Time and Materials
At the opposite end of the spectrum is working on a time and materials basis. Many agencies do this extremely well. Their reputations and experience encourage clients that continually fund the team till the project has ended despite their being a slight fuzziness as to what the exact bill will be. However, the flaws with this include:
- The balance of risk is all on the client – They have no guarantees that their budget will be able to deliver them all the features they need to fulfil their business aim.
- T&M can be erratic – it’s hard to actively manage a backlog when one sprint can suck up a huge amount of time but deliver very little in terms of releasable software; while other sprints deliver a lot more for less dev work. How can you assess the value of a feature when you can’t be sure of its final cost.
- T&M doesn’t promote efficiency – Any contract where you get paid more for taking longer isn’t really delivering a positive subconscious message about working lean
- Even if the client has unlimited budget, T&M projects can result in wooly decision making – there’s no fixed end point, so there’s no pressing need to make clear, timely decisions that fit with a lean build.
We’ve had experience working with both sets of models and have found various ways around the problems they can pose. Currently though, we are moving to a new model that works well for us and our clients
We call this the Price Per Point model:
- Functionality is broken down into discrete user stories and we estimate these according to how long/how complex it will be to deliver every aspect of that story: design, UX, development, PM, AM testing, refinements.
- We estimate these stories in story points. Using points gives us a method of estimating stories in relation to one another rather than fixating on exact times (e.g. it’s easier to estimate that the train station is twice as far away from your house that the supermarket than it is to estimate the exact distances in km).
- Before starting development as a team we review the initial estimates after a sprint planning session. This gives us a chance to update our estimates based on the very latest knowledge. It can also give the client a chance to add further acceptance criteria, split or simplify a story.
- Once ready we begin work on the story and progress it all the way through the stages of production until it is at a stage where the client can sign it off before it is pushed to a live site.
- The client pays a fixed cost for each story point and doesn’t pay a thing until the story has met with their approval and has been fully signed off.
The reaction from clients to structuring billing in this way has been overwhelmingly positive. The key reasons being:
- There is a really clear definition of done i.e. Client’s must sign off each story on a staging or live site in accordance with acceptance criteria that they have written
- Clients are in control of their backlog and can actively engage with adjusting the scope of their project to meet their budget
- The risk on project is balanced between client and agency: the client takes ownership for delivering a functioning product within their points budget; the agency takes ownership for delivering each fixed price component.
- It rewards the development team for being efficient but not for cutting corners
- It necessitates regular, involved sign-offs from product owners and facilitates a more collaborative environment
- It makes it easier to quickly switch directions within a project based on user feedback – a client can adapt and pivot when they’re provided with feedback from real users and are empowered to prioritise user feedback over new features if they wish.
Ultimately, working in this method drives adherence by the agency to an agile working methodology. You simply have to meet your definition of ‘done’ or you won’t get paid.
After embarking on this course of billing we found that we weren’t the first who had considered such an approach. To some in the agile community there is a risk that targets based on estimates will skew the accuracy of estimates. However, for White October, this doesn’t feel like such an issue because:
- We work unusually closely with clients, and this closeness and as part of this we expect our clients to question, debate and understand any estimates we provide
- We expect that estimates will often be very inaccurate but over time on an agile project we believe that a greater project understanding, monitoring velocity and improving efficiency with each iteration will rebalance any estimates that may have been subject to an optimism bias at the initial stages.
Having a clear focus on getting vertical slices done and signed off by the client has transformed a number of major projects. Team members and clients have a clear, shared goal now and a mutual vested interest in project efficiency. While it isn’t a magic bullet that solves all problems, we can safely say this model is here to stay as a valuable addition to our project toolkit.
To me this is one of the great things about working at White October, our ability to experiment, adapt, iterate and improve leads to some really remarkable findings and some very happy clients.
Having a project described as “rare and amazing” by a client was the ultimate reward for taking a risk and trying something new.