Estimation can be a tricky part of any new software initiative. Provide an estimate which is too high and your project may be cancelled before it even begins; estimate too low and your project has the risk of failing or being highly stressed due to perceived cost and time over-runs.
If the project in question is a capital outlay on a new warehouse or improved machines in the factory, accurate estimates can often be prepared quite easily, as these are highly repeatable, low complexity initiatives where indicative costs and timescales can easily be stated.
For many software projects, however, the same cannot be said and there are myriad high profile examples of projects which failed due to poor planning such as the £12bn failed “NHS Connecting for Health” system in the United Kingdom or the $6bn failed FAA Air Traffic Control System in the United States of America.
This article takes a look at why software projects often suffer from overly optimistic estimation, how you can manage stakeholder expectations and what techniques you can use to create meaningful and useful estimates tailored to your delivery style.
What is estimation?
To estimate is to “form an approximate notion”. In the context of software delivery, estimates are making predictions about time and cost based on the available knowledge and information at the time the estimate is made. The more information that is available, the more accurate the estimate can become.
To have an entirely accurate estimate, you would require all possible information available about requirements, the pace of delivery, resourcing and world events. Of course, there is no way that the future can be predicted in this manner and so all estimates have a degree of uncertainty in them.
In situations where the task, environment and resources are similar to a previous real-world project, you have a good reference point for estimating a new project. However, software build projects often consist of highly bespoke requirements built on complex technology platforms and so delivery times can vary between different initiatives.
The best estimates will consider a number of factors such as:
What teams will be required in order to complete this endeavour, allowing for developers, testers, platform engineers, project resources, change management, trainers, team management. Will this come from existing internal staff, new hires, contractors or third parties?
FinancesWhat will the software project cost and in what currency? How much of this will be capital expenditure (CAPEX) vs operational expenditure (OPEX)? How quickly will the cost be recouped by the organisation and over what period (otherwise known as ROI or IRR)?
High-level tasks and deliverablesWhat are the tasks that will need to be completed as part of the project? Will there be a large build phase, a large migration, a high number of users?
TimelinesAlthough not possible to be exact, a 3-week project is very different to a 3-year project
Third-party involvementWhat will the significant third party costs be? Will there be licensing costs, hosting costs, will it be on a timeat time and materials basis or a fixed cost?
Why is estimation important?
When considering any large capital outlay, most commercially aware professionals will want to know “How long will it take?”, “How much will it cost?” and “When will I start to make a return on my investment?”. Indeed, in large organisations, the answers to such questions are often mandatory pre-requisites imposed by budget holders before projects can even begin.
The cultural reasons for expecting estimates are understandable. Any publicly listed organisation is expected to provide quarterly forecasts to their shareholders to indicate how they think the company will perform over the coming quarter, accounting for any large capital or operational expenditure appearing on their balance sheets.
Estimates provide a common languageNot creating estimates can cause a rift between software development projects and the rest of the organisation. There is a growing movement known as “#NoEstimates” which reasons that as software projects are so often over time and budget, that it is a waste of time to burn effort making predictions that often prove to be incorrect. Even though many software practitioners are abundantly aware of the unpredictable nature of bespoke software delivery, many organisations would find it unpalatable if their teams did not provide any time or cost estimates.
Estimates allow alignment across the project teamNot only is estimation important as a pre-requisite to project sign-off, but the process of creating estimates has a number of other benefits:
- The process of estimation adds structure to project planning and design work ensuring you consider areas of the project which you might have missed if you weren’t having to consider timelines.
- Estimates provide transparency within the project team and allow assumptions to be questioned or validated by team members. As an example, let’s say some team members thought you would be building an integration to a pre-existing system but this was missed by the project team who thought it would be a one-off data load, the publishing of the estimates would allow this to be made transparent.
For more details on starting a new software project, check out our blog post which covers the subject in more detail: Starting Software Pojects
Inputs required for estimating software projects
In order to estimate a software project, a number of inputs will be required before a meaningful view can be created.
Scope of Work
Scope covers a broad set of topics as follows:
- User requirements (at a high level). This is necessary in order to understand the complexity of the system. Consider the difference in complexity between an app which shows users a different photo every day vs an oil trading system. Even though the high-level requirements may be understood, there could be some requirements that carry significant extra effort and complexity.
- Non-functional requirements. An application that is required to be available 24/7 to a user base of 1 million users will be architected and built in a very different way to an internally facing app that only needs to be available during office hours.
External factors are important to consider. These are often summarised with the mnemonic “PESTLE”, standing for:
- Political
- Economic
- Social
- Technological
- Legal
Environmental
Although these should be covered under the requirements, it is worth looking again to see if any of these factors are at play. Such factors will help identify any high degree of compliance, security, reporting or forthcoming change which might be important to consider. As an example, a system dealing with people’s medical records will have very stringent security requirements, a breach of which could result in significant fines for the organisation at fault.
Project risks
A risk is a potential occurrence that, if it materialises (and then becomes an “issue”) will cause a measurable impact on the project. Generally speaking, project planners tend to only concern themselves with negative impacts, as these are the ones that have the potential to adversely impact delivery timelines and costs.
Some examples of project risks could be “We have a high attrition rate of developers”, or “A key system we need to integrate to is unstable and often unavailable”. A whole article could be written on risk management alone, but a decision will need to be made as to whether the risk will be:
- Prevented
- Reduced
- Accepted or
- Transferred
Tech Stack
A final important input to a software development project estimation process is a consideration of the Tech Stack.
- Will the project be using a known quantity of a modern programming language, or is there a requirement to write in something of a legacy nature (meaning programmers will be hard to find and difficult to replace)?
- Do we need to account for items from the tech stack going end of life during the project, or requiring effort to upgrade to the latest version?
- Are we deploying the project to our own physical data centres where internal lead times and processes could cause delays or to a cloud-hosted arrangement where we will need skilled platform engineers to configure the environment?
- Is the stack aligned to the competencies present in the organisation and if not, how will this be managed once the project transitions to its support and maintenance phase?
Software development estimation techniques
For any project team, a toolkit of estimation techniques is required, with different techniques being suitable at different points in the projects lifecycle and for different stakeholder groups. As more details emerge over the course of a project, these facts can be built in and the estimates revisited.
Let’s take a look at some suitable techniques, how to use them and when they add value during the software development lifecycle.
Top-down estimating software projects
One “catch 22” style issue with estimating software projects is that for some approaches, you will need to have hired your development team in order to put your estimate together, but you can’t get approval to hire any resource until you provide an estimate. Top-down estimating in software projects is a technique that is used early on in a project’s lifecycle that allows you to create a high-level estimate without having the technical team onboarded.
Top-down estimates also allow you to create a high-level view of costs without having a full view of all the business requirements, as to gather these would require the onboarding of a business analyst to conduct extensive interviews and documentation. There are a couple of favoured approaches to top-down estimating in software projects as follows:
- Consensus/Delphi method. This method, originally proposed by the military to forecast the impact of technology advances during the cold war, allows for an anonymised sharing of opinions and experiences from senior and middle managers. It uses a structured round of anonymised interviews and response sharing amongst participants to allow the group to agree and coalesce around a single estimate.
- Apportionment or Analogous Estimating. This is an empirical (fact-based) system that compares the scope of the project to be estimated against other projects of similar size and scope, with the assumption being that the estimates will be comparable between similar endeavours.
Pros and cons of top-down project estimates
- Top-down approaches allow you to gain a high-level estimate of what is needed to complete your project. This is usually sufficient to allow organisations to take a decision regarding the return on investment.
- Accuracy is often low for bespoke builds, but if there is good knowledge of similar initiatives, accuracy can be improved.
Bottom-up estimates
In bottom-up estimating, projects are split into a series of work packages which are all estimated separately both in terms of duration and costs, before being rolled up into an overall estimate for the project.
This approach is perceived as being the more accurate approach of estimating due to the fact that it considers project tasks at a lower granular level and is therefore occurring with more information present than in the top-down approach.
Pros and cons of bottom-up estimating
- Tends to be more accurate than top-down
- Useful for developing a detailed schedule & budget
- Requires more time, effort and better articulation of the project requirements.
Three-Point Estimates
Three-point estimates take a sample of three separate eventualities as follows:
- Best case scenario (a)
- Worst case scenario (b)
- Most likely scenario (m)
From these three set of estimates a final weighted average is derived as per the following formula:
This ensures that overly optimistic or overly pessimistic estimates are given less weight than expected outcomes, but are still factored in to the overall estimate. For added accuracy multiple estimates can be gathered from multiple sources and averaged out.
Agile vs Traditional Estimation
One of the key principles of the agile manifesto is “Responding to change over following a plan”. For that reason, agile estimation techniques tend to deal more with what can be delivered over a short term horizon, rather than what can be delivered over a long period.
There can be a natural friction here as some agile practitioners can be strongly resistant to providing estimates, with the teachings of some frameworks actively encouraging their certificants to refuse to provide estimates longer than the next few sprints.
Agile mindsets also subscribe to the view that the development of software should be done in an incremental and “test and learn” approach, rather than adhering to a long term delivery plan. The belief is that we generally don’t have all the answers upfront and so we should value new information over adhering to a previous list of deliverables as that new information might be likely to yield more value than the original uninformed plan.
Advantages of Agile estimation
As the agile community has taken a pragmatic and fact-led approach to estimation, practitioners are much more likely to be a bit more circumspect with regards to the estimates that they publish.
This means that such estimates are likely to be accompanied by heavy caveats underlining the inherent lack of accuracy in such approaches and more commonly, rather than providing a specific time duration such as “4 weeks” they might provide an estimate of “3-6 weeks” in order to illustrate the potential for delays to creep in.
Agile estimations are often quite lightweight, not requiring pages and pages of documentation to be submitted, so they are often less time consuming than more traditional methods.
Techniques of Agile estimation
- T-Shirt Sizing. Using a range of XS-XL sizes (as in T Shirts) the team assignsassign sizes to higher level deliverables (often known as “Epics”). T Shirt sizes are used because they are deliberately vague, in the same way that a Large T shirt covers a wide variety of body shapes and sizes, so will a Large T shirt size of an epic. Once all epics are T-shirt sized, indicative time can be ascribed to each size, allowing an overall estimate to be created.
- Planning poker is a technique that is used primarily once a delivery initiative is in flight. It is similar in some respects to the Delphi estimation mentioned earlier in this article, in that estimates are delivered “blind”, except done in a much more lightweight way. Someone will talk through a deliverable of a suitably small size (usually referred to as a “User Story”) and the team, once they understand it well enough, use poker cards with values written on them to size the item. They all reveal their cards at once so as not to be influenced by each other, then the cycle repeats with the next item.
- Person hours. (Formerly known as “Man hours”). One approach which may be used is to assign days/hours of effort to delivery items. This is often done at the “Task” level, (the lowest level of deliverable). Using person-hours to estimate has fallen out of favour as it is so often inaccurate due to the “planning fallacy” phenomenon where humans are likely to be over-optimistic when estimating.
- #NoEstimates As mentioned earlier, some agile delivery teams have made the argument that estimation should not take place at all, as it is wasted time that could be spent on delivery. Instead, they try to make sure all deliverables are split down to similarly-sized subtasks and achieve a constant flow of delivery. This approach is best used in ongoing product development in organisations that have a culture which is receptive to this divergent approach.
Conclusion
Estimation in software projects is a complicated and broad subject. One thing that can be guaranteed is that if you deliver significantly under time and budget, you will be less likely to suffer negative attention than if you are significantly over time and budget. Having said this, estimations that are too high could result in the project never commencing, or a loss of trust from wise executives who realise the estimates have been “padded out”.
There is a broad range of estimation techniques that should be used at the correct stage in the project’s lifecycle and in a tailored way depending on the nature of your project and the industry you are delivering in.
Although estimates can often be controversial, they are an important tool in opening a dialogue about the complexities of software development with your stakeholders and project team. If you can get these people on board early and embed a culture of pragmatism, experimentation and transparency when delivery dates move, then you have a promising foundation for your project.