Getting Past the Big Data Hype—and Backlash

Why arguments at both ends of the spectrum are missing the point

Solution CTO, IBM

I’ve also recorded a podcast on this topic—download it here. There’s a commercial for a new TV making the rounds right now—one where images explode off the screen and envelop the people watching. Suppose you actually went out and bought this TV, but then you realized that the images didn’t literally jump off the screen the way they did in the commercial. Would you return your new TV? Of course not. Watching the commercial, you knew it was both marketing and an overly dramatic illustration of reality intended to make a point. Chances are, the TV works just fine for your reasonable expectations. But what does that have to do with big data? Like your response to the commercial, starting a big data project depends on having reasonable expectations and not diving headlong into hype. This is worth pointing out since we are now at the evitable point in the adoption cycle where we are seeing some churlish, defensive behavior from vendors that feel threatened by the trajectory of these new technologies. A good example of this came up on one of the big data forums I moderate on LinkedIn. An individual practitioner asked if the space was overhyped (of course it is). This question was answered by Microsoft business partners, who ranted that there was no value in big data and that no firm should be considering it. They argued that big data is only useful with social media and that big data projects are a “bet the whole company on the outcome” sort of thing. These arguments are myopic, self-serving, or both. Public discussion about big data has polarized people into two camps: one focused on overly positive hype, and—at the opposite end of the spectrum—excessively pessimistic naysayers. Is there too much hype about big data right now? Sure. But does it contribute to the discussion to throw around obviously wrong and uninformed arguments, either? No. Time for some perspective, I’d say. What’s happening in big data right now is a classic example of a well-established technology adoption pattern. The usual pundits and media sources that need something compelling to write about are over-hyping certain aspects of the big data space. But remember: they did the same thing with Java, SOA, application servers, and business intelligence when these technologies were emerging. Last time I checked, those technologies all seem to have stuck—and their adopters did it by focusing on real problems and not trying to shoot the moon. Myth-based arguments against those technologies didn’t stand the test of time, and neither will the ones against big data technologies. So how do you balance the discussion and look beyond the hype? As usual, it comes down to pragmatism. Get inspired and think big thoughts, but start by implementing modest projects. (Boil a bathtub, not the ocean.) If you can’t sketch a plan for ROI, stop. Reconsider your approach, and don’t do anything until the path to success—with the metrics to back it up—is clear. Ask for references, and demand experienced guidance from the ecosystem you decide to tap into—not just for big data, but for the overall system flows as well. As I have written elsewhere, anyone who implements a significant big data investment based solely on media hype is making a major mistake—and anyone who rejects these technologies based on ill-informed backlash is equally misguided. What do you think? Let me know in the comments.   [followbutton username='thomasdeutsch' count='false' lang='en' theme='light']

 <table cellpadding="0" cellspacing="0" valign="top" width="15%>

  [followbutton username='IBMdatamag' count='false' lang='en' theme='light']