How to estimate a software project

Introduction

Let us get one thing straight. The only estimation I have done earlier is during my engineering days where I worked out quantities and rates for construction work. The key advantage then was that most of the design and specifications were finalized before the estimation began. If not, one could quote a vague figure called the 'accepted market rate', and then go on changing that figure [provided one had the skill set ;) to do so] as and when the design and specs were finalized.

For others with my kind of background, estimating a software project based upon a few discussions with the customer and stakeholders is nonsense. Estimating a software project is like estimating what it would take to invent something without knowing what that something is. Function point analysis and cocomo(?) is for geeks. Quite often, software estimates are way off the mark, and the end result being some geeks are then given pink slips.

And so, I did a sensible thing. I stayed well clear of estimating software projects. But situations change and it is getting more difficult to avoid this dumb task. The inability to estimate projects, resources, risks etc. also means inability to get perks, broaden our horizons, and rise up the geek ladder. Meanwhile, there are more mouths to feed, more financial commitments, more taxes to handle. And anyways, there is no point doing low level coding all your life. One should strive to reach a level where one can really screw things up.

Brass Tacks

Right then, enough of cribbing. It doesn’t pay. And my superiors are better at it. What I aim to do is to figure out a way to make estimation of a regular software project similar to estimating a regular engineering project. A typical engineering project estimate contains several parts as follows:

• Identifying the items of work.
• Fixing a unit for each item of work.
• Calculating the quantity of work.
• Calculating the rate for each item of work.
• Fine tuning the final cost.

The rate is the cost of resources involved per unit of work. The resources are usually:

• Man-hours.
• Materials.
• Tools and plants.
• Overheads.
• Profit margin.

The overheads involve:

• The establishment charges.
• Depreciation of tools and plants.
• Interest factor etc.
• Licenses, permits, taxes.

The same concept can be extended for estimating software projects. We start by identifying and describing items of work, then the units for the item, the quantity, and so on. Describing items of work can be fairly irritating form a techie point of view, thanks to its verbose nature. The rate part should look normal enough. Usually, overheads and profit margin [decided by authorized persons only ;)] are taken as a percentage. By now, many would be smirking and/or condemning this kind of an approach. The main defense in favor of this approach is that most engineering works have to last for years, the project durations range from months to years, and there are many non technical people who believe they know things better than engineers (in some case, it's true). Many engineering projects have critics ranging from politicians, journalists, lawyers, social workers, citizens, idiots, and even engineers. Engineering projects include the Egyptian pyramids, space exploration, or manufacturing paper clips. So, why can't engineering techniques be applied to software development - a field that includes engineers of software, hardware, computers, and networking, and where reliability, robustness, and scalability are required.

Identifying items of work

Okay, now for the first part. What does an item of work consist of? In business applications, this could be a workflow, sub workflow, or development of components or sub components. We also need to classify the items of work (i.e., identify types of work).

In the following example, I will create some categories of work in software projects. A typical classification tree could be as shown in the figure below:

We can continue working out more categories and subcategories, but for now, we will attempt to estimate within the categories shown above. All the items under hardware category and sub categories are easier to estimate as we can get the costing from their purchase/market value. The same applies to the operating systems, run time software, and tools. The workstation cost includes establishment charges like office space, electricity, water, air conditioning etc. Other costs like renting, licenses, work permits, and taxes for acquiring each item should be added as applicable. Some overheads and hidden costs are not reflected in the tree. These are usually estimated and added as either a lump sum cost or as a percentage of the overall cost.

The actual software development costing starts from the category 'Application'. Most business applications have several modules. A module is a feature in an application. Now, we start to identify each item of work for each of its categories.

The category 'Framework' provides the backbone on which all the other components in the application link to or depend upon to do common tasks.

The framework development could include the following items of work:

• Data access components
• Exception handling
• Logging
• Email/Fax
• Printing
• Data Export
• Services (web services, remoting, DCOM, COM+, application server etc.)
• Security (authorization, authentication, code access, encryption)
• Resource Files
• Deployment
• Miscellaneous (utilities, constants, enumerations)

The category 'Presentation' could include the following items of work:

• GUI (Forms and controls)
• Custom controls
• Validation
• Animation and Graphics

The category 'Business' could include the following items of work:

• Coding (for work flows and business requirements)
• Validating data
• Formulas and calculations
• Data manipulations
• Implementing business rules

The category 'Data formatter' is used to convert data into acceptable formats before sending and after receiving. The formats can be XML, value objects (an object that is used to carry data from tier to tier), and encryption.

The category 'Data storage' could have the following items of work:

• Database design
• Writing SQLs, procedures, triggers
• Indexing
• Performance tuning
• Flat Files (including text, XML, graphics, multimedia, etc.)

The Application itself is a category. In the category Application, an item of work is a work flow. A work flow itself may consists of other workflows and processes. The most common features when fixing a rate for an item of work are the expected amount of work to be done (GUI objects, lines of code, non private methods,) and expected complexity. This depends upon the experience of the bidding company, technology being used, and support available. The amount of work increases with the size of the workflow. If the workflow has a lot of rules, branches, sub workflows, and processes, it indicates complexity. Complex workflows should be broken down into simpler workflows. Therefore, the basis of estimation would be to develop as many workflows as possible and to put in the maximum number of details into the workflow. In case, developing all the workflows is not possible (often the case), then (the business analyst should be able to) map expected workflows to existing workflows so that estimation can be done with better accuracy. Trying to estimate a project without doing the workflows is practically impossible except in the case of static web pages.

Now that we have identified some of the items of work, we need to explain them in detail. Determine what exactly is going to be provided (and maybe even what is not being provided), the corresponding specifications, the cost of manpower, hardware, documents and other deliverables, overheads up to completion (or delivery). In software, this would also include the cost of testing, quality assurance, user acceptance testing, and warranties.

After describing an item of work, fix the unit for each item. For form development, the unit could be done as numbers (no.s). Note, the forms maybe further divided into categories depending upon the complexity and content in it. So, do not put all the forms into one single item of work unless all forms are similar to develop. In the Business category, we should identify the main business objects, components, or features, and categorize them depending upon the content and complexity. Again the unit for business category works could be no.s.

The database could be estimated based upon the number of database objects expected, and the complexity of the tables, views, SQLs, procedures, normalization, and level of data integrity required.

Once a workflow is done, it should be studied carefully to identify the items of work involved. Below are a set of workflows to request and provide a resource.

Categorization

Let us study the resource allotment workflow for different categories.

Category Data Storage

This requires resource details to be stored so that resources can be easily stored and retrieved. This involves a relation database management system. The database should contain tables to store resources based upon certain characteristics, say human resources, items, and documents. To retrieve these faster, they can be further categorized and have keywords associated with them. All these categories would themselves be stored in another table(s) and mapped to their parent categories.

The request for resource should be mapped to the resources found. Therefore, all requests are stored in a table and can be categorized based upon priority, impact, department/person placing the request, and availability of resource.

Finally, the different criteria for accepting or rejecting a resource should be recorded along with the department/person who accepted or rejected the resource, along with an explanation. The criteria for accepting/rejecting a resource could be stored in another table and also could be categorized.

To place requests, search and allocate for resources would require writing SQLs. By viewing the functionality of the screens (if prototypes are available) as well as the workflow itself gives a fair idea of the complexity of the SQLs to be developed.

So right away, we get an idea of what is required from the data storage point of view.

Category Presentation

The persons in charge of resource allocation would need to view the requests for resources. They should also be able to sort resources based upon the status of the request, date of the request, priority, impact, department, costing etc. This would help them decide how to handle each request. The resource in charge persons should be able to search for resources based upon categories and sub categories, keywords, status, location etc. Finally, a resource should be mapped to a request. Screens should be developed to display requests details, resource details, resources and requests mapping, search for requests and resources, etc. The display can be on WinForms, browser or console.

Category Business

The Business category usually contains the interfaces and business objects. The business object may perform calculations (say, time period can be calculated from date(s) provided), implementing business rules (like whether the particular department can be given a certain resource), sequence of calling the business methods, and interactions with the data tier.

In the resource allocation workflow, we could have business objects such as requester, resource manager, resource, and search engine. Each object required can be considered while estimating the cost of the workflow.

Category Data Formatter

The Data formatter ensures that the format of the data is correct. If a particular data were of type float, then the data formatter would convert the data passed to it into a float before assigning it. Else it would not allow the data to be assigned at all. In case of XML, it would ensure that the XML document is built as per some predefined format. The data formatter could also be part of the framework. A data formatter can also be thought of as an object to carry data from one tier to another.

Category Framework

The framework components required by the resource allotment workflow could consist of tier to tier communication, security features, custom components, libraries, resource files, data tier access, error handling, and possibly logging, email, fax, and third party interaction. The extent to which each of these would be developed should be considered after looking at the entire application. But for our item of work, we should take a fraction of the framework development cost into account. This is because the framework is a development requirement and not a business requirement. So, only consider the features of the framework that would be required for the flow and add an approximate cost to the rate.

Item of Work

Let us call the item of work as 'resource allotment'. Now that we have looked at the item of work with respect to different categories, it will be easier to describe it. A critical aspect is the labor or man-hours involved to develop the workflow. Most companies have their own metrics on the development cost based upon efforts put into previous projects. Based upon the previous project metrics, they calculate the number of man-hours.

Now we put the item of work into an estimation table as shown below:

For an inexperienced estimator, it would be difficult to fix the cost for the item of work shown in the table above. The best way is to describe the item of work with as many details as possible, and then try to arrive at the cost of each detail. Take the help of experienced persons (architects, managers, developers, testers, technical writers. etc.) to identify more details (including hardware, software, man hours, overheads) and their corresponding costs. Then simply add it up and compare the final figure against the cost of similar work done earlier.

Parting Note

Unlike engineering projects where many works are repetitive and the average output of a machine or person is easier to work out, in software projects, there are usually wi(l)der fluctuations. The risk factor, impact factor, changes, even miscommunications are more, and affect the costing greatly. All these issues have to be factored into the cost along with the expected profit margin.

J