Summary
People tend to start planning with the best case scenario in mind. People also tend to think that they have better than average capabilities and can consistently out-perform the norm. Taken together, these two well-documented cognitive biases result in overly-optimistic plans. Starting with an external average task duration is much more realistic.
Your plan is probably wrong
The typical planning process, using a step-by-step approach, leads to unrealistic plans
Best-case scenarios with a fudge factor
Typically, when we plan a task, we start to internalize
all the steps required and how long the steps will take.
Again typically, we assume almost a best case scenario –
all the resources will be available, I’ll be mentally sharp
that day, the equipment will work, I won’t get distracted
on other things, etc. With this rosy picture of how everything
is going to work out well, we plan our timelines and costs.
Sometimes, we add a fudge factor to account for one or two
steps going wrong. If we tend to be pessimistic, we add
a few fudge factors. The end result is a forecast of how
the project should work, given a few mishaps. Yet
in actuality, more things will go wrong with a project than
will go right. This is why we often cite Murphy’s Law. Unfortunately,
we usually think of Murphy in hindsight, not during planning.
Even when presented with evidence of how past examples took
longer, we find exceptions in the history that shouldn’t
apply to the future. For instance, it should only take me
3-4 hours to clean out the garage this weekend even though
it took me two days the last time I did it. The reasons
it took me longer last time was because I found a box of
albums from my college days and reminisced over those for
a while, and then I found a box of the kids’ old toys, and,
again, I reminisced. After that, I found some of the neighbor’s
tools and returned those to him, and we started chatting,
and he ended up showing me his new home theatre, and I stayed
and watched a movie. I’m sure nothing like that will happen
this weekend when I clean out the garage.
The reason why so many home improvement do-it-yourself projects
get abandoned half-way through is because we end up encountering
so many unexpected difficulties. In the do-it-yourself guide,
replacing the toilet valve looks so easy, but when we
try it, we find the pipes are rotted and can't turn off
the water. Then, we can't remove the old hardware because
it is corroded. When we finally remove the old, rotted hardware
(having replacef the pipe and fittings in the process) we
find that we actually have a non-standard sized toilet,
and our new parts don't fit correctly. At this point, we
abandon the project and tell everyone in the house not to
use the toilet.
This is actually what happens at work, too. We plan a software
upgrade and find that our old operating system doesn't support
the new version so we have to upgrade that first or we develop
a new streamlined process and discover we have to rewrite
all of SOPs and then train everyone on the new SOPs or we
buy a new piece of lab equipment and realize that we have
to test it rigorously before we can integrate it with our
systems. Nothing is ever as easy as we first planned.
Internal focus (where everyone is above average)
In addition to adopting the best case scenario as our baseline,
we also tend to think that our project or situation is unique.
Somehow we possess the knowledge or the skills or the luck
that everyone else lacks. Garrison Keillor nailed
it when he described Lake Wobegon as a place where all the
children are above average. In studies of how accurately
people can judge themselves, most people rate themselves
as having above average intelligence and above average athletic
ability. This is why, when we plan, we think "It may take
you three weeks to accomplish that task, but I can do it
two."
In study after study, psychologists have documented the tendency of people
to underestimate how long it will take to complete a task,
even given a history of their performance in the past. Thus,
telling people to use better planning techniques and to
err on the conservative side has little effect. People will
still create an overly-optimistic forecast despite history
to the contrary. This common phenomenon is called planning
fallacy. Planning fallacy is a fancy term for "the tendency
to hold a confident belief that one’s own project will proceed
as planned, even while knowing that the vast majority of
similar projects have run late.” (Buehler, R., Griffin,
D. and Ross, M. 1994.
Exploring the "planning fallacy": Why people underestimate
their task completion times. Journal of Personality
and Social Psychology, 67: 366-381. ).
Psychological studies also show that people do a much better
job predicting how long it will take others to perform a
task than predicting their own performance. The difference
in the thinking is that when we judge others’ performance,
we use “outside” information like how long similar tasks
take and how they have performed in the past. When we judge
our own performance, we use internal narratives that discount
our histories with exceptional situations (otherwise known
as making excuses), build an optimistic step-by step picture
of how we think things “should” go. Finally, we are
felled by the “better than average” bias into thinking that
we can do a much better job than those other people in their
projects, kind of like thinking that, “It won’t happen to
me.”
The very simple way of eliminating the planning fallacy
is to base predictions on the knowledge of how similar projects
fared and use that as the starting point rather than the
internal step-by-step of how it “should’ proceed. Once you
have that scenario, plan for pretty much everything to go
wrong, a la Murphy’s Law. After you get that estimate, multiple
it by 3 and now you have a workable plan.