The Coming Data Crisis

Guest post by Eric Greenwood, editor for

The Coming Data Crisis

For the past decade or so, our technology has been doing a decent job with keeping up with our computing needs. While storage space once came at a premium – storing just under a megabyte and a half required a floppy disk – today the average home user can add an external hard drive that holds a terabyte of data for less than $150.

Things are changing, however. Our cameras are becoming much higher quality, able to capture clearer, crisper images that take up considerable amounts of storage space. High-quality videos

In short, we’re creating more data today than we currently have capacity to store.– even HD quality videos – can be downloaded to your computer.

How fast is data growing?

On a consumer level, you can probably look at your photo and video files and see just how much more data you’re using today than you were a decade ago.

In the business world, it’s even more obvious. For example, the databases that handle all of Wal-Mart’s customer transactions use around 2.5 petabytes. Facebook holds over 40 billion photos currently.

According to IDC, it’s expected that the amount of information created on a yearly basis will be twice what our available storage capacity is within a year.

There is no doubt in anyone’s mind that we’re undergoing a serious data explosion; the question is how to deal with it.

Entering the cloud

Fortunately, there are those in the marketplace that feel the winds of change, and are taking steps to meet our storage needs. This kind of opportunity creates a positive economic pressure on the online storage industry to grow and become even more profitable than it is today.

Distributed cloud storage – like that you have with online storage providers – is the natural solution to the coming data crisis.

Converging needs

It’s interesting to note that, at the same time that we’ve seen this great data explosion, we’ve also seen an explosion in Internet access. No longer are we tied to desktops if we want to access the web, email, or online applications; now we can get all of that data from our smartphones and tablets.

(Some would argue that the advent of the smartphone and tablet have actually helped to cause the data explosion; certainly, in the realm of photo storage for consumers, this could be the case.)

Up to this point, online storage has met that need for mobility posed by the advent of tablets and smartphones. Indeed, over the past year online storage services have boomed. Even Apple has finally seen at least some marginal success with its cloud storage medium, iCloud.

Moving forward

As time goes on, more eyes will turn to the online storage providers to handle greater amounts of data, creating fierce competition and great potential profit.

What will be interesting to see is whether the online storage marketplace becomes a battleground for some of the bigger players (i.e. Apple and Google) rather than for startups like Drop

Box that are specifically designed to meet online storage needs.

Eric Greenwood

Eric Greenwood is a technophile whose interests have lead him to study all things related to the cloud computing movement from software as a service to online storage. Get more tips and advice on the blog Online Storage!

This entry was posted in Cloud Computing and tagged , . Bookmark the permalink.

6 Responses to The Coming Data Crisis

  1. The Higher Ed CIO says:

    I appreciate the guest post and wanted to pose an odd thought I had. I am wondering just how much data duplication is playing into the ever expanding growth. So the photo taken by someones phone gets loaded flickr and shared to Facebook and Twitter then pulled into a blog post or two all while being saved off onto people’s own devices mashed up and reposted or pinned on Pinterest. Oh don’t forget the cache copies at Google, Bing and everywhere else. Seems the data duplication resulting from the consumerization side will be way worse than what we have lived through in the enterprise.

  2. Eric says:

    I definitely agree!

    I know of people who, instead of collecting the source links to a picture they find online, will actually save the picture to their hard drives – Fortunately there are sites like flickr and imgur who make it a lot easier to represent a picture through the tag without having to host the media yourself!

    But I definitely agree, that has got to be a great proportion of the data growth!

    Thanks again J!

  3. TJ says:

    Hi Eric,
    Nice article. In your piece, you state: “According to IDC, it’s expected that the amount of information created on a yearly basis will be twice what our available storage capacity is within a year”

    I’m writing a piece on big storage, and I’d like to use your statement… but I need to back it up with the original source. Could you help me out? Where exactly can I find that info from IDC? I’ve been to their site, but can’t seem to locate it.


  4. Eric says:

    Hey TJ,

    Thanks for your question – Unfortunately I can’t find the source on their site either, but there are some remnants left of it in this article: <- If you check out the graph part way down the page you'll see a graphic based on the IDC info.

    Hope that helps!

  5. The Higher Ed CIO says:

    I did a little digging today and found a IDC citation in a June Mashable aritcle on Big Data which projected 1200 exabytes of data will be generated this year alone. ON another site I found the IDC Worldwide Quarterly Disk Storage Systems Tracker data for 2011 which placed the amount of storage shipped in 2011 at 6,279 petabytes (6.279 exabytes). So it seems data is growing at 191 times storage – or there abouts.

  6. TJ says:

    Thanks so much. That helps a lot.

Comments are closed.