IT stress test scenarios using lessons from submarines will determine the IT department’s minimum operating levels and what is truly critical. Using some analogies from my time on submarines seemed like an interesting and relevant way to further illustrate the idea behind conducting an IT stress test. I realized I hadn’t spent enough time on these points in the previous post IT Stress Test for Colleges and Universities, so here goes.
Rig for Silent Running
Anyone that has watched Hunt for Red October or the old classic Run Silent Run Deep would understand that submarines go to great lengths to remain undetected. To remain undetected requires the submarine to have a very quiet acoustic signature all the time.
During sensitive and special operations submarine captains will order the crew to “rig for silent running”. To rig for silent running the crew shift systems into slower speeds, secure loose objects, avoid unnecessary activity, and avoid sudden changes in ship speed to avoid creating noises from air bubbles collapsing (cavitation) behind the screw (propeller).
If a CIO ordered the IT department to rig for silent running it would also mean dropping to a lower speed or pace of activity, avoiding any sudden changes in operations, and only doing the essential activities.
Silent running might mean you would process standard service requests but not enhancements or new functionality. Patch management would continue but upgrades would have to wait. And, all project work would be suspended unless it is vital to the business.
What would your fixed costs and variable costs look like if you ordered silent running, the minimum operating level, for a year affect? This is the IT stress test.
- What systems could you decommission?
- What staff could you let go or reassign out of IT?
- What contracts could you cancel or not renew?
- What contracts could you drop to a lower level of support?
- Do you have enough inventory or ready spares?
- How much budget would you need?
Rig for Ultra Quiet
When the order is given to “rig for ultra quiet” it means somebody is near by and you absolutely must avoid detection. Rig for ultra quiet results in practically everything coming to a stop if it might cause the slightest noise to transmit out into the ocean.
No maintenance, opening doors or cabinets, entertainment, cooking, or walking around. If you are not involved in something mission critical you just go lay in your rack (bed). During a rig for ultra quiet only the most critical systems and functions are carried out.
During ultra quiet the submarine crew uses internal sound and vibration analyzers to track down anything making the slightest noise that might be heard outside of the ship like a bad bearing, loose fastener, or coffee cup rattling on a counter.
For an IT department to rig for ultra quiet you would effectively suspend all operational support services and projects unless they were vital to the continuation of the business. If it wasn’t required to sustain life, fire your weapons, or track down sources of noise you just wouldn’t do it.
If a CIO were to rig for ultra quiet for a year, like going on life support, just how much could be cut from the total cost of IT? This is another IT stress test.
Test Depth and Crush Depth
Another interesting IT stress test analogy from my time on submarines relates to how deep you can safely operate the submarine. Each submarine has a specific test depth and crush depth based on the design strength of its hull.
Test depth is a design rating of how deep a submarine can be safely operated based on actual submergence tests. Test depth is based in part on being able to surface in an emergency. Crush depth is the design depth at which the ocean pressure with collapse the hull of the submarine.
An IT stress test should in many ways attempt to determine the test depth of your IT department. That would be how far can you cut the IT department and still be assured you can get back to the surface if you needed to.
An IT stress test should attempt to determine the level at which your IT department and the IT functions of the organization will be crushed if cuts are taken too deep.
Remember, the purpose of conducting an IT stress test is no different that testing your DR plan. It assures you have a plan to safely make cutbacks in degrees with known consequences and risks.
PS – If you’re a bubble head and want to share a favorite old sea story feel free to use the comments below.