Tuesday, March 10, 2015

Embracing Inaccuracy

Perhaps nothing in our industry is more debated than time-estimates. Engineers hate time-estimates because they are never accurate, but everyone else desperately needs them. This tension has spawned a glut of software-development methodologies. At their core, most methodologies are all trying to tackle the same problem: time-estimates. Methodologies either claim to improve the accuracy of your estimates or they claim to lessen your dependency on estimates.

I'm not an expert on all of the various methodologies, but I do know one thing--everyone desperately wants to solve this problem. Everyone is willing to bend over backwards to achieve estimate-accuracy. If someone were able to legitimately solve this problem, the entire industry would immediately adopt the new methodology and its creator would become wildly rich. The fact that such a thing has not happened, yet, tells me all that I need to know. There is no solution, nor will there ever be one. I'm not saying that methodologies are worthless; I'm just saying that you shouldn't expect them to be a silver bullet.

So, should we cease making estimates altogether, as some have suggested? Of course not. That's absurd. Time-estimates are vital to the success of any software shop. So what is the solution? Realism. Don't waste precious energy trying to solve the unsolvable. Embrace the inaccuracy of time-estimates and learn to deal with it.

Much has already been written about why estimates are notoriously inaccurate, so I won't belabor the reasons here. Suffice it to say, there are too many unknowns. Obviously, the more accurate your estimates are, the better off you will be, but you will never come close to perfection, and the law-of-diminishing-returns most definitely applies. There are some simple steps that you can take to significantly improve the accuracy of your estimates, but, once you go beyond those simple things, you will expend far too much effort to make it worth the slight gain in accuracy that you will realize. Here are some practical examples of simple steps that you can take to improve your time-estimates, without taking it too far:

Breakdown Tasks
For any project that's going to take more than a few man-days, do not just ask your engineers to give you an estimate off the top of their head. You want them to give it a little thought first. Even if they only consider it for five minutes, the estimate will be many times more accurate. The best way to ensure this happens is to ask them to give you a breakdown of the major tasks involved with a separate estimate for each task. Ask them to give a man-hour range for any of the tasks that involve unknowns. It should not be a long list--maybe fifteen items or less. It shouldn't take too long to prepare--maybe 5 to 30 minutes. For instance, here's an example of a perfectly good time estimate:
Task 1 (4)
Task 2 (12 - 15)
Task 3
   Task 3a (2)
   Task 3b (20 - 30)
   Task 3c (20 - 25)
Task 4 (6)
Total = 64 - 82 hours
Not only does asking for a written breakdown ensure that a little thought will go into it, but it will also help you to prepare for the most likely upcoming problems. The tasks with the largest ranges are the ones with the biggest unknowns. If some of the ranges are too large, you may want to request that more research be done in those areas before continuing any further. 

Fight the temptation to break it down any further. You will be tempted to gain more accuracy by breaking it down into smaller sub-tasks with shorter time-frames per task. Unfortunately, if you do go down that path, you will accomplish very little. You will invest much more time, but the estimate won't be significantly more accurate.

Consult Your Checklist
When you ask a developer to make an estimate, they usually only concern themselves with the programming tasks. They quite often overlook all of the other necessary work that needs to be done. Depending on the project, these non-programming tasks can be just as critical and involve just as much work as the actual programming tasks. Therefore, when you ask one of your developers to provide you with a time-estimate for any project, it's best to start by providing them with a list of non-programming tasks that you want them to include. Since these things are usually very similar from one project to the next, you should have your own personal checklist of tasks from which you can cull ideas. For instance, here is my checklist of non-programming tasks that commonly apply to most projects:
  • Research
  • Collaboration with third-parties
  • Design
  • Documentation
  • Testing
  • Bug fixing
  • Code review
  • Training
  • Branching and merging

Know Your Average Man-Hours Per Day
Man-hours are not the same as clock-hours. This may seem obvious, but some people don't realize that there is a major difference. If they do realize it, they must regularly forget it because they certainly don't act like it. You need to remember that just because there are 40 clock-hours in a typical work week, that does not mean that one engineer can complete a 40 man-hour project in one work-week.

The primary reason for getting man-hour estimates is so that you can use those estimates to calculate the projected completion-date. The most important figure to know when estimating end-dates is the average number of man-hours that each of your engineers accomplishes per day. Don't focus on precision--a rough estimate will do. It will be different for each person (the value typically decreases with seniority). For most people, only about 30% - 60% of their time is devoted to their assigned projects. That means that you should only count on most developers for somewhere between 2.5 and 5 man-hours per day. If you don't have a good handle on this value, assume 4 man-hours per day until you get a better idea.

I know what some of you are thinking. You think that's ridiculous! You think that I'm yet another engineer just trying to make excuses. Fine. If you want to waste your time, I challenge you. I challenge you to have your engineers track every hour of every day and average it out over the course of a few months. I'm confident that, after wasting all of that energy and goodwill, you'll reluctantly come to the same conclusion as I. By the time you factor in vacations, personal days, doctor appointments, meetings, bathroom breaks, tech support, metaphorical fires, helping other developers, IT problems, and a little down time between tasks, you too will face the same cold-hard-facts. It's your choice; you can swing against the wind or you can accept reality.

Don't Forget the Fudge-Factor
Once you know which developers you are going to assign to a project, you can sum their average man-hours-per-day figures to arrive at the total man-hours-per-day for the team as a whole. You can then take the total time-estimate and divide it by that number to arrive at the total calendar work-days, like this:
Engineer 1 (3.5)
Engineer 2 (4)
Engineer 3 (3)
Total = 10.5 man-hours per work-day
Project Estimate = 64 - 82 man-hours
Best case = 64 / 10.5 = 6 work days
Worst case = 82 / 10.5 = 8 work days
In the above example, the project completion date would be projected to be somewhere between six and eight work-days after the start-date, right? Wrong. You first have to multiply the number of work-days by your fudge-factor. Why do you need a fudge-factor? You need a fudge-factor because developers almost always underestimate. First of all, most developers want to tell you what you want to hear; they don't want to constantly be the bearers of bad news any more than you do. They'll tend to focus on the best-case scenario. In addition to that, there are usually some tasks that get completely overlooked or are unforeseen. Lastly, any project of significance will require a certain amount of rewrite while it's being programmed. If you don't already have a good idea of what your fudge-factor is, assume a factor of two. For instance:
Best case = 64 / 10.5 * 2 = 12 work days
Worst case = 82 / 10.5 * 2 = 16 work days
Notice how the original estimate, in this example, was 64 - 82 man-hours.  If you had asked the developer for an estimate without the task-breakdown, I'm willing to bet that you would have gotten a more rosy, round number like 50 or 60 man-hours.  If you had just taken the quick estimate of 60 man-hours and divided it by three, you would have come to a total of only 20 man-hours per developer.  If you had then assumed that they would work on it for a solid eight hours per day, you would have come to an estimate of only two and a half work-days.  Imagine how foolish you would have looked if you had promised that the project would be complete in under three days and in reality, it took sixteen days.

Understand What It Is
Now that you have a reasonable estimate, you need to understand what it is. It's just an estimate. It's not going to be entirely accurate. Even when you follow all of the above steps, sometimes the project will run past its end-date and sometimes it will be completed sooner. Don't try to fix that. Just accept it. Understand, from the outset, that what you have is just an estimate which may very well be incorrect. Plan ahead for how you intend to handle it if it is wrong. 

On larger projects, you may need to periodically ask for re-estimates based on the work that's left to be done. As the list of tasks shrinks, you can add further breakdown under each one. Wide range estimates should narrow as unknowns are tackled. Because of this, the estimates will usually become more accurate as the end of the project gets closer.

Base Your Estimates on People
Man-hour estimates are not constants.  They may change dramatically depending on who will be doing the work.  Everyone has different skill sets, competencies, and personalities.  As such, you need to know who is going to be on the team before you ask for an estimate.  If you change the members on the team, you need to ask for an updated estimate, or at least ignore the previous one.  As The Mythical Man-Month has taught us, when you add a new team member, you can't just add their average man-hours into the calculation and deduct it from the end-date.  People and tasks are not just numbers.

No comments:

Post a Comment