Evaluating IT Investments Using Objective IT Project Rankings

Evaluating IT investments for funding is one process where using a simpler approach is not always better. That is because the process of evaluating IT investments should involve a two step process for each project under consideration in order to support an objective IT project ranking of all proposals and ultimately, the IT project selection decision.

The first step involves developing the business case for the IT investment directly on the merits of the proposed initiative. This first step is often done as part of the project proposal (re: project request) for submission. Then the second step occurs within the IT governance process set up for evaluating IT investments in a portfolio context. The IT governance process does this by comparing the business cases of all proposed IT investments to determine which ones get financing.

Evaluating IT Investments – Methods

Champions Approach

Realistically, many organizations’ process for evaluating IT investments for approval is done based on who is championing the initiative. This is sort of a my dad is bigger than your dad approach to IT governance whereby the senior executive with the most clout or political influence gets their pet IT projects approved over those of their peers, regardless of the merits of the initiative or its business case. Organizations supporting a Champion’s Approach neutralize their IT governance committee’s effectiveness by negating any objective IT project rankings which ultimately leads to making bad IT investment decisions.

Squeaky Wheel

This is pretty straightforward. The squeaky wheel in the organization gets their IT projects approved over other project proposals regardless of the business case in order to appease the person. Put plainly, organizations approve IT projects to make the person go away hoping to eliminate the energy drain caused by the squeaky wheel. This rarely works as it only reinforces their behavior and creates more squeaky wheels. Interestingly, the Squeaky Wheel approach is often a variant of the Champions Approach because the person most affected by the squeaky wheel is the one who champions the approval to make their personal pain go away.

Hi, Med, Lo

Organizations attempting to develop the maturity of their IT governance model often create an IT project selection process for evaluating IT investments which relies on rating the importance of each proposal with a simple Hi, Med, or Lo rating. This approach is certainly sensible enough as a starting point. However, it is a very subjective approach not significantly better than the Champions Approach. Although it does provide a basis for comparison of unrelated IT investments, it still leads to bad IT investment decisions.

Risk reward matrix for combining Hi Med Lo risk with rewardOrganizations can build upon this model with the simple addition of a second Hi, Med, Lo rating for the project risk of failure. The addition of a risk rating to the importance rating creates a way to gauge the likely downside (risk) of the proposal along with the upside (reward) in order to reduce an overly optimistic view. The two ratings can also be combined for an aggregate rating to support the IT project selection decision.

TCO and ROI

Total Cost of Ownership (TCO) and Return on Investment (ROI) are two methods for evaluating IT investments which begin to take some of the subjectivity out of the process. Notice I said some, since there is still plenty of room for manipulation of both TCO and ROI calculations. Additionally, TCO and ROI can create an illusion of precision if the financial calculations are over simplified. When TCO and ROI are used for IT project rankings where projects have non-financial benefits which might be difficult to quantify additional considerations are needed. Despite the possible weaknesses in using TCO or ROI as part of the IT governance model for IT project ranking, they do represent a more mature approach to evaluating IT investments for funding.

Weighted Likelihoods

It is advisable to consider including a Hi, Med, Lo rating with TCO and ROI in order to add a weighting to the TCO or ROI calculations and to estimate risk. You can use a weighting in one of two ways, as a positive weight or a negative. A positive weighting would involve an ROI calculation to arrive at some number and a rating for the likelihood of achieving the ROI. The ROI is then combined with the rating to produce an IT project ranking for approvals. The negative approach would be to rate the likelihood of not achieving the proposed benefits or simply the likelihood of failure.

Time Value of Money

Whimpy cartoon with quote "I will gladly pay you Tuesday for a hamburger today"Evaluating IT investments really cannot be meaningful unless the analysis factors for the time value of money. The time value of money principle says that a dollar expected today is worth more than a dollar expected tomorrow. The application of the time value of money is made by evaluating IT investments projected future cash flows in the form of increased revenue or in hard dollar savings. Because they are projected future returns, their value must be discounted to their present value.

Examples of the discounted cash flow model are internal rate of return (IRR) and net present value (NPV). The use of discounted cash flow calculations in evaluating IT investments is really one of the best approaches for IT project selection. That being said, some caution is needed when using IRR or NPV for IT project ranking. That is because just like TCO and ROI, NPV and IRR are mechanical calculations, standard spreadsheet formulas, and not a consistent principle. That means IRR and NPV calculations can still produce misleading information especially when comparing mutually exclusive projects leading to bad investment decisions.

Probabilities

Just as with weighted TCO or ROI calculations incorporating a probability factor with an IRR or NPV calculation gives you a way to account for errors and faulty assumptions. These are the factors which create black swan projects and serve as the basis for the fallacy of planning. Instead of using a simplified weighting, it is advisable to use a probability for the likelihood of success or failure. In doing so you can set the probability based on the whole range of risk factors from requester bias, project complexity, or other factors that might reduce the rewards and inflate the risks.

Evaluating IT Investments – Final Thoughts

The goal of any IT governance model ought to be a uniform standard for evaluating IT investments and IT project selection from a portfolio of requests. Achieving those goals requires an assessment of the financial performance of a proposed investment that factors for the risk of being wrong.

In the case of for-profit businesses the evaluation is more easily focused on how well the IT investment produces returns in the form of profits and shareholder dividends. This makes TCO, ROI, IRR and NPV calculations a lot easier. More importantly, it makes validating the calculations of approved projects very easy which allows IT governance to ensure the promised returns are in fact delivered.

For non-profits and governments the evaluation is still focused on producing returns but in this case the returns often take on a different form. Here the returns might be increased public safety, new schools, reduced patient mortality, or better educated students. These kinds of outcomes are much more difficult to quantify in a spreadsheet. So, IT governance models might still struggle to account for the intangibles.

This entry was posted in CIO Job, IT Financial Management, IT Governance and tagged , , , . Bookmark the permalink.

7 Responses to Evaluating IT Investments Using Objective IT Project Rankings

  1. Peter Karandeniya says:

    This is very good view of IT value project evaluation and agree with what it´s exposed in such a clear manner.
    thks

  2. Al Forte says:

    Regarding ROI – wow how does one come up with a value for time and productivity savings of both IT staff, regular staff, faculty, and students who benefit from technology, for instance going to virtual desktops.

  3. The Higher Ed CIO says:

    The most straight forward way is to develop an average hourly rate for a given position or resource type. Be sure to use a fully loaded rate (salary + benefits + overhead). Then just multiply the estimated hours for each resource type needed for the project in your project estimating process using the example or your own approach and voila. So that how you get it for the cost side to do a project. To get the savings side follow the same method but estimate the workload eliminated for your calculation.

    CAUTION: Let’s say you automate a process that saves 100 hours of labor each quarter or 400 hours per year of staff time which costs you $65 per hour fully loaded. Most people then claim the $26,000 in their ROI as savings. But unless you are actually eliminating $26K in real costs, the savings are not actually occurring and we call them soft savings. If you actually reduce costs and improve the bottom line then you can claim hard dollar savings. Using soft dollar savings in ROI’s is the single biggest abuse in building project justifications. Another form of abuse is in claiming a cost avoidance as savings (re: returns) in the ROI. This happens when you budget for $30,000 and it only cost $26,000 and conclude you “saved” $4000 when in fact you really have a cost avoidance. If you do in fact return the money or simply just don’t spend it, then you do have a savings. But if you just spend it on something else you didn’t actually save any money.

    So talk it over with your CFO to see how these issues should be best handled. And, I will post an sample worksheet you can use in a day or so.

  4. If the culture is one of Champion or Squeaky Wheel, that’s what will tend to happen regardless of the amount of apparent objectivity. It just takes longer and generates more meetings and paper.

    Because early (project-decision stage) estimates of cost and benefit are inherently uncertain (4x or more), they’re easily gamed. What’s worse, the approved projects are then saddled with overcommitments, triggering a negative “haste makes waste” feedback loop in implementation.

    It’s especially bad in functional organizations, where the value streams are orthogonal to the org chart. Hard costs tend to land one place and benefits another. That pits senior leaders against one another.

    I think the best thing you can do to make better IT decisions (or any other) is to break up the functional silos, organize along value streams, and give every value-stream leader control of the backlog in front of a dedicated IT implementation team. That still requires some cross-stream governance controls. Federal input, Duopoly decision, in Weill and Ross’ terminology in IT governance.

    This is just my synthesis of others’ ideas. See Guidance on Agile Project Set-Up from the Agile Expert for PMI’s LEAD CoP. That’s my summary of Sally Elatta’s talk, plus a direct link to it.

  5. The Higher Ed CIO says:

    Robert, I appreciate you taking the time to comment and for your great thoughts. I am glad you included the note on costs landing one place and benefits another as a factor.

  6. Pingback: IT Governance and the LMS - Principles of IT Governance

  7. Pingback: ROI Calculator Template | The Higher Ed CIO

Comments are closed.