The Fallacy of Planning

The Fallacy of Planning says we are terrible at planning how long something will take and how much it will cost. Restated another way, the fallacy of planning is people’s tendency to underestimate what it will take to get something done. The phenomenon of the fallacy of planning ought not be a big surprise to any CIO or project management professional given the attention it has received over the years. What may be a surprise though is the pervasiveness of the fallacy of planning in our organizations and the cumulative impact it has on IT and the CIO’s reputation for delivering results.

Fallacy of Planning

The fallacy of planning is just one of the many forms of cognitive bias that affect decision-making in every aspect of our personal and professional lives. Cognitive bias is when a persons’ judgment is affected by a lack of mental abilities or from the misapplication of one. Most often this is a decision-making shortcut resulting from insufficient information, social pressure, or personal motivations. Cognitive bias is a very powerful form of irrationality, where people act in opposition to what is expected under the rationale choice theory which requires consideration in every risk management plan.

Innumeracy

Cognitive bias includes innumeracy which is a particular form of irrationality related to a person’s fundamental inability to conduct basic reasoning with numbers. It is a form of mathematic s illiteracy often manifesting itself when dealing with really large (and small) numbers, probabilities, estimating, and even basic arithmetic.

What makes innumeracy so insidious is how pervasive it is in adults including highly educated professionals and how much we tolerate it because “math is hard”. Just consider the knit picking and tirades you have been witness to over innocent spelling or grammatical errors, yet hardly a peep when a gross error occurs in someone’s math or misapplication of statistics.

IT Performance

For CIO’s and IT governance committees the fallacy of planning introduces several forms of risk into IT decision-making that must be mitigated and documented in the enterprise risk management plan and every project risk management plan . The accuracy and reliability of a project’s total cost of ownership (TCO), return on investment (ROI), and internal rate of return (IRR) are all areas where gross errors can occur as a result of innumeracy. By themselves, an error in any one of these could be enough to make a bad project look good supporting approval ahead of others. When combined with errors in the estimates of financial and human resources required, project duration, or the risk management plan estimates things can really go bad.

Unfortunately, innumeracy affects other aspects of IT governance and decision making when it comes to effective portfolio management of the initiatives coming out developing IT strategic plans in support of business strategy. This includes any of the plan’s component parts like a focused technology plan or application portfolio management plan where importance of statistics and probabilities cannot be overstated. Similarly, the security plan and risk management plan depend heavily on developing defensible likelihood, probabilities and statistics.

IT Project Failure Rates from Standish Chaos ReportPerhaps you find this interesting but since you were great at math and you’re one of those people whose resume says something like “….completes projects on time and on budget…” you think this doesn’t apply to you. Well, industry data collected over many years on IT project failures by the Standish Group , McKinsey and others demonstrates only 34% of projects are successful while 15% are failures leaving the remaining 51% as challenged where success is defined as on time, on budget and the expected outcome is delivered.

That is quite a disconnect from the on time and on budget crowd whose portfolio management system reveals a different story. This is the core evidence of the planning fallacy which must be addressed by every CIO’s quality control plan. We think we are better than we are – another cognitive bias.

It is because of the work of the Standish Group CHAOS reports the growth in project management profession began and we created project management offices in IT departments. But many have found the benefits of project management and portfolio management (PPM) have peaked. This is believed to be the result of placing the PMO in IT where the gains cannot reach the maximum potential until the PPM role is move to an enterprise function outside of IT. This notion is further supported by those believing there are no IT projects only business projects requiring IT resources. And so just as governance, risk and compliance (GRC) are enterprise functions so too should PPM.

CIO Job

CIO’s willing to tackle the fallacy of planning have plenty of things they can do to make improvements.

  • Implementing Agile project management offers many benefits from iterative and incremental development regardless of which flavor of Agile project management used.
  • Implementing a portfolio management model (PPM) even if it is paper based so you can begin measuring and managing project and resource performance and improving project estimating accuracy.
  • Conduct refresher training on project estimating, determining risks and calculating statistics and probabilities. Consider Six Sigma training to support your continuous improvements.
  • Formalize the financial models for TCO, ROI and IRR to simplify the calculations and to ensure uniformity between competing projects.
  • Develop a quality control plan to review portfolio management practices, risk management plan,
  • Train IT and non-IT managers on effective project management practices and ways to guard against innumeracy and other forms of cognitive bias.

But perhaps the most effective thing CIO’s can do to minimize the effects of the fallacy of planning is to create a safe environment of trust for making planning decisions. This requires building a culture of sensibility where pragmatism is recognized for its benefits and ego driven plans are discouraged. It also means creating a culture where pulling the plug on a bad decision doesn’t come at the expense of anyone’s career.

Avoiding the fallacy of planning also requires an ability to detect its origins and effects and a commitment to extinguish them through awareness and education.

This entry was posted in CIO Job, IT Performance Management, IT Risk Management, IT Strategy and tagged , , . Bookmark the permalink.

12 Responses to The Fallacy of Planning

  1. PM Hut says:

    I have published an article that looks at the topic of planning from a different perspective. You can find it here.

    I would like to republish your article on PM Hut, please either email me or contact me through the “Contact Us” form on the PM Hut website in case you’re OK with this.

  2. In response to a few comments on the influence of people being just overly optimistic in their estimates, let me offer this addition.

    The cultural factors affecting project failures in the US are legendary. We Americans, and other cultures, do have an overly optimistic view that works hand-in-hand with an inability to grasp large numbers or complex problems with multiple variables. Some think it is more arrogance than optimism originating from over inflated sense of sense and one’s ability to execute – think id versus ego. Perhaps I am making it more complicated than it needs to be.

    I would also offer that there are many organizational cultures where a safer status quo mindset dominates. Here optimism gives way to low risk guarantees for fear of failure. In these organizations, the importance of accurately assessing project risk, resource requirements and expected benefits are critical so that people don’t feel like they are sticking their necks out.

  3. Glen Alleman says:

    Jerry,
    While nearly everything in this post has some value – and I’ve subscribed to the Blog – there is a core fallacy to the Fallacy of Planning.
    Where we work, in the aerospace and defense business as well as heavy construction, planning is a profession. We use estimating tools, planning models, parametric assessments of past performance for forecasting future performance, and mandated probabilistic models for cost, schedule, and technical performance.
    These processes are rarely found in the commercial IT domain – with the obvious results.
    Instead of restating the problems with estimating – cognitive bias is but one – here in the aerospace and defense, nuclear power, nuclear weapons, and large construction we have moved from identifying the sources of the problem to installing processes that provide solutions.
    Until commercial IT does the same, they are bound to the repeated failures of the past.

    • Engineers, including IT types, are notorious for under estimating risks in part from their over confidence in their ability and their systems. This has lead to colossal failures which are made worse by inaction or actions that compounded the event. This is also the case with laypeople who often incorrectly associate risks to technology without considering the risks without it.

      I have worked in the military (active duty), nuclear power (military and commercial), public utilities, large aerospace and defense companies, petro-chemical industry, and for the manufacturers of some of the largest rolling machinery in the world in one capacity or another including IT audit and technology risk management. Every one of them were run by people who were human who made normal human judgements. Sure some had more mature processes or better tools or modelling software, but in the end people make judgements.

      The point of my post is to awaken people to the reality of our results so we examine our processes in a more honest way and avoid over committing and under delivering.

      Indulge me with some illustrations:
      >Yesterdays news of the Pentagon Protected From ‘Open-Ended’ F-35 Overruns or
      >Today’s story of the Ford Carrier Facing Cost Overruns
      >The GAO’s recent assessment of NASA’s performance on long term projects isn’t exactly positive.
      >The GAO 2010 report on DOD Financial Improvement efforts although promising on the whole are not flattering with regard to the ERP portion of the strategy.
      And many more could be shared regarding the critical infrastructure and its command and controls systems or from the nuclear (commercial, military and weapons) industry.

      I am not trying to be heavy handed with this reply. I am trying to demonstrate that results speak for themselves regardless of the sophistication of the tools or models.

  4. Glen Alleman says:

    You may be confusing “over runs” from Off Baseline changes with poor initial estimates.
    Specifically:
    1. JSF overruns are not from estimates, they are from baseline changes to software capabilities, engine controls redesigns, and numerous other “uncontrolled” changes driving the program OTB. We work the avionics suite for JSF.
    2. Don’t know about, but http://bloom.bg/pOo9K2 provides reasons. Extended tested of radar,. The 7% overage of course is a huge number on $56B, but some facts here may help understand that the contract award estimate is still intack for all non-changed elements.
    3. For ERP I can agree, hence my point about using the estimating processes of A&D, DOE, and some NASA centers (JPL for one).
    The Standish numbers are bogus, http://bit.ly/oRRHZB is a place to start, then follow the Federal IT portfolio.dashboard.
    I know of many NASA deep space programs on budget and on schedule, the GAO report itself (http://1.usa.gov/nd1uSU) does not distinguish between initial estimation and uncontrolled baseline changes. GP-B is one I’ve had hands on experience with and engineering failures drove the cost.
    I manage PP&C groups in all those domains from manned spaceflight, DOE D&D, NAVAIR avionics, USAF infrastructure.
    Care needs to be taken to distinguish between estimating gaps and control of the baseline for that estimate.

    So if you include engineering failures (low technology readiness levels that applied poor cost escalation judgement), then you may have a point. But over generalizing is one issue in many of the reports.

    It may be better to distinguish the poor initial estimating from the poor management of the baseline.

    • Glen – I will admit to loving this dialogue as it is helping me realize where I should have been more tight in my language as the need to simplify a post often makes for problems.

      I think most shops use at least a two step approach to project estimating. The initial estimates are generated in support of a proposal submission. For most, this is often just a swag done out of efficiency in order to support some form of request ranking and portfolio sizing before approvals begin. Hopefully the approvals rely on other data such as IRR or at least ROI. The second estimating step is done during project initiation when the team develops a WBS and so on.

      The irrationality is often greatest in the initial estimate where it is the most problematic since it is part of the basis for the approval. Most PPM baseline off of the second estimate to track variance. This is important from a project management practice but from a business management perspective also tracking variance against the initial estimate is essential since it is from their budgets and capital plans are set and initiatives approved. It also creates visibility into the accuracy of the project requestor’s ability to estimate. IN a way creating a reputation score for each requestor’s thoroughness.

      Well great discussion and I appreciate your perspective.

      • Glen Alleman says:

        Jerry,

        I think the term “most shops in my experience” would resonant well. Our “most shops in the enterprise IT domain.”
        But in my experience in heavy construction, DOE D&D (Deactivation & Decommissioning), manned space flight avionics and controls, pulp and paper and petro-chem plant turns, and software development of embedded controls estimating is a professional role that produces relatively high confidence (80%) cost and schedule numbers during the proposal and improves those during execution. This is especially important on the emerging firm fixed price contracts from the US DoD.

        There are of course numerous counter examples. But my experience in ERP and other enterprise IT is that the estimating processes, tools, and skills are sorely lacking. If that your point I agree whole heartily. But it’s the the “fallacy of estimating,” because there are numerous examples of good estimates – even for enterprise IT – that hold up over the life cycle of the project.

        So when you say “most shops,” maybe some specific examples of a domain and a context would help. IN our domain, the proposal grade estimate is awarded and we live with that. That estimate is validated by a 3rd party and if found wanting our proposal is scored lower. “Cost credibility” is an evaluation criteria.

        What you seem to be describing is immature estimating capabilities. But that is not a fallacy that you can estimate – a popular myth in immature organizations. Increasing the maturity of cost and schedule estimating, along with the estimates of technical performance is hard work and many immature organizations simply don’t want to invest upfront in this effort.

        What agile does is provide a mean for incremental commitment to the estimate – in our domain through rolling waves. Definitized estimates for the current rolling waves must be on baseline before authorization to proceed (ATP) is provided. Planning waves with their Planning Packages have budget boundaries but are not definitized to the Task level. But Agile itself provides no improvement in estimating in a mature organization. The Planning Game process popular these days is standard Wide Band Delphi (WBD) with Monte Carlo Simulation (MCS) in use since the 70′s. The Agilest have rediscovered what we did at TRW Redondo Beach 1978. Same for 1,2,5 boxing of estimates for future work. Our own estimating process uses WBD as a start, the a MCS for macro level processes, then a Subject Matter Expert for the current Rolling Wave. Then the MCS is updated (weekly) from actuals in the Earned Value system to produce an Independent Estimate at Completion (IEAC) weekly as well.

        Success require “ruthless” adherence to the plan for the Rolling Wave (like Scrum should do for the iteration), and corrective actions for any “out of band” variances at the Work Package level. This works of enterprise IT in the same way it works for D&D’ing a $7B DOE weapons plants.

        But as you mention not all shops have this capability, but it’s there for the taking once they figure out that estimating and program controls is an equivalent profession just like writing software is. No Fallacy, just failure to execute.

        Look at the AACEI (www.aacei.org) approach to estimating, our any large construction or A&D firms estimating process. Those GAO reports have the root cause, but rarely is it the original estimate. There are some – SIBIRS (a local program) is one, MOX (Mixed Oxide Fuel) is another, and the few satellite programs make head lines. And of course a load of failed ERP programs. But search for the root cause and I’d suggest its much more than bad estimating.

  5. Glen Alleman says:

    Jerry,

    John Goodpasture – an agile program management colleague – just published an observation on the use if Schedule Risk Analysis mandated by DID 81650 for DoD Earned Value program. SRA uses Monte Carlo Simulation to reveal te needed schedule and cost margins and the probability of completing “on or before” a planned date and “at or under” a planned cost. John’s post can be found at http://bit.ly/nESbJt

    The use of proper estimating and modeling of these estimates is mandated by 81650 and used for proposal down-select in several agencies: NAVAIR, DOE NSSA and DOE EM, some centers in NASA (Houston, Marshall, JPL from are our experience), some USAF Commands, USCG, and some Fed Civ agencies.

    So when I mention that the “fallacy of planning,” is an indicator of immature program and project management, here’s the counter example. Meaning that procurement agencies are no longer accepting proposals without a credible Basis of Estimate.

    This of course doesn’t mean others aren’t completely naive about the process of credible cost and schedule estimates, but there are working examples of how to do it.

  6. Pingback: Why Strategic Planning Fails | The Higher ED CIO

  7. Glen Alleman says:

    Jerry,

    Maybe some insight into estimating tools in practice may aide those unable to estimate.
    http://www.sceaonline.org/awards/papers/2011_software.pdf.
    http://www.sceaonline.org/ is a place where cost estimators go to find methods to put into practice to avoid the fallacy of planning.

  8. Glen Alleman says:

    Jerry,

    Another paper on the same topic that references Kahneman and Tversky http://www.dau.mil/pubscats/PubsCats/AR%20Journal/arj59/Oneil_ARJ59.pdf

  9. Pingback: IT Project Failures and the Black Swans | The Higher ED CIO

Leave a Reply

Your email address will not be published. Required fields are marked *


3 + = 7

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>