How to Realistically Estimate Software Development Projects?

Introduction

It is not a secret that many software development teams occasionally or even regularly end up missing deadlines or exceeding project budgets. Though it is a well-known fact that project timelines and man hours must be properly estimated before a project begins, developers still commonly fail to stay within the restraints they set at the beginning. So what gives? How do costs creep up over time and projects take longer to complete than initially planned?

The following factors often play a role:

1. Because new requirements and ideas are added later on
2. Because certain aspects of a requirement were not taken into account
3. Because of unforeseen technical difficulties during development

In order to complete a project in accordance with the stipulated deadline and budget, all requirements must be listed, clarified, and confirmed with the client first. We believe that the foundation for a successful project can and should be laid by first writing technical specifications, then preparing and confirming mockups with the client, and finally, creating the interface prototype. Once these steps have been completed, a qualified analyst will be much more likely to accurately estimate the project efforts and can therefore plan the deadlines more precisely.

As common sense as this strategy may be, it seems that it is rarely put into practice. But why?

The following factors might play a role:

1. The client wants to know the estimated cost of the project before coming to a detailed agreement about the requirements.
2. It can be rather costly to take all of the above mentioned steps in preparation for each new RFP without compensation.
3. Technical analysts may feel a sense of disillusionment when faced with too many additional requests after already investing a lot of effort into preparing the proposal initially. Too many repeat requests may be emotionally draining and may cause the technical analyst to no longer invest 100% into analyzing each added request in detail and providing accurate estimations accordingly.
4. One of the most common problems: the person performing the estimation doesn't know to how to systematically approach project analysis and estimation. On the other hand, the sales manager often doesn't know how to accurately fill out the commercial portion of the proposal. This scenario is particularly common in small companies, where one person handles both the technical and the business aspects of the proposal without necessarily having the knowledge and experience required to do so. As a result, this takes away from the overall accuracy when estimating man hours and budget.

To solve this problem, many companies prepare a preliminary estimation, and only if the client shows a serious interest in continuing with them, do they provide a more complex and in-depth analysis.

It is not rare that the sales manager is unable to discuss the budget with the client and when the project changes several times, the transaction closes with a price that was stated after only a quick preliminary estimation. This puts the burden of completion under these conditions on the developers.

None of this information is new for those who are in the business and for whom estimations are a daily task. However, the problem still remains unsolved for many development teams.

This issue can be tackled with the help of a solid workflow that is based on a systematic approach for new project requests and new clients. Needless to say, the sales manager and technical analyst should also have the necessary knowledge and experience to work with clients and to make software projects estimations. A systematic approach includes formalizing the process, dividing projects into the individual stages and into assignments within each stage.

What have we come up with to improve this situation?

Several years ago, we already understood that the majority of all mistakes were made during the preliminary project estimation stage. It was obvious that using Excel spreadsheets or Google docs to estimate man hours and then putting this information into template files of commercial proposals was too time-consuming. Not only that, it also resulted in small mistakes that were often overlooked by employees. In addition to that, it also became clear that employees do not always know what needs to be written in which order to accurately estimate a project and to create a commercial proposal.

Upon realizing this, we decided to develop our own internal application for project estimation to use in combination with commercial proposals. We decided to set mandatory rules that required the technical analyst to go through a specific set of pages and to fill in the required fields. In addition, the technical analyst would have to divide each project into different stages of completion and modules (larger sets of requirements) as well as tasks and smaller subtasks. We then chose to display the project estimation data on a separate page in the form of a table, similar to an Excel spreadsheet and to include the option to utilize many of the same hot keys users were familiar with.

This looked as follows:

• Module 1
  • Primary task 1 - 5 hrs
    • Subtask 1.1 – 3 hrs
    • Subtask 1.2 – 2 hrs
  • Primary task 2 - 9 hrs
    • Subtask 2.1 – 1 hrs
    • Subtask 2.2 – 3 hrs
    • Subtask 2.3 – 5 hrs
  • Primary task 3 - 5 hrs
    • Subtask 3.1 – 2 hrs
    • Subtask 3.2 – 2 hrs
    • Subtask 3.3 – 1 hrs
• Module 2
  • Primary task 4 - 7 hrs
    • Subtask 4.1 – 3 hrs
    • Subtask 4.2 – 4 hrs
  • Primary task 5 - 13 hrs
    • Subtask 5.1 – 3 hrs
    • Subtask 5.2 – 7 hrs
    • Subtask 5.3 – 3 hrs
  • Primary task 6 - 6 hrs
    • Subtask 6.1 – 1 hrs
    • Subtask 6.2 – 1 hrs
    • Subtask 6.3 – 4 hrs

Project estimation thus included filling in specific fields and systematically dividing work scope into stages (milestones), modules, and subtasks.

Furthermore, for each "primary task," we not only planned time for coding but also for automatic testing and even manual testing.

Depending on the specific project, work on one module (for example development of the Dashboard in a web application) could also be divided into different stages. In that case, we distributed the tasks as follows:

• Stage 1
  • Module 2
    • Primary task 4 - 7 hrs
    • Subtask 4.1 – 3 hrs
    • Subtask 4.2 – 4 hrs
    • Primary task 5 - 13 hrs
    • Subtask 5.1 – 3 hrs
    • Subtask 5.2 – 7 hrs
    • Subtask 5.3 – 3 hrs
• Stage 2
  • Module 2
    • Primary task 6 - 6 hrs
    • Subtask 6.1 – 1 hrs
    • Subtask 6.2 – 1 hrs
    • Subtask 6.3 – 4 hrs

As a result, estimation of man hours and project timelines in general has become much more accurate. In addition, the initial proposal preparation stage has lead to many very specific questions about particular details, which in turn has made estimation even more precise and accurate as answers to those questions needed to be found. The overall process also changed in that the sales manager only had to fill in a few pages, which included adding the budget (either based on a per-hour cost or a package rate), payment conditions, and certain marketing data.

Another important step was that we decided to split up the responsibilities between the sales manager and the technical analyst. This meant that certain pages could be filled in by the technical analyst only, while other pages could only be filled in by the employee responsible for sales and project costs. Therefore, the technical analyst saw only the technical portion of the proposal, and while the sales managers were able to review the data, they did not have the option to modify anything in the technical section of the proposal.

Over the course of time, we often encountered similar projects and decided to add templates (the option to generate a proposal on the basis of previously created proposals) as well as the CRM module, which would allow client data to be stored and negotiation progress to be tracked.

As the above points clearly show, we have achieved a dramatic improvement in the accuracy of our project estimation process. All of the steps involved, from obtaining the initial requirements to the start of work on the project, have become significantly more structured and formalized. Serious mistakes in project estimation have been eliminated.

There are several similar applications on the market (like www.swproposal.com, www.evenflowpro.com or www.proposable.com) to facilitate the process of project analysis and estimation. They can be found by searching for "Software project proposal creation" or similar queries. However, it is important to emphasize that choosing one software over another isn't the ultimate solution in itself. It is much more important that the processes around project analysis and estimation are well-structured and thought through. This includes clearly defining steps for the desired intermediate and final results as well as splitting responsibilities among employees.

As this post shows, correctly analyzing and estimating a project from the start can substantially increase your chances of completing it successfully, on time and within budget.