Recapping 2014: Significant Trends for Big Data

The months ahead represent the year that was when it comes to prognosticating big data, analytics, and more

Solution CTO, IBM

Normally, a calendar year elapses before evaluating how the predictions for that year have panned out. How conventional. Because big data represents a disruptive space, I thought I’d be correspondingly disruptive by reviewing what happened in 2014—even as only the year’s first quarter draws to a close. Reviewing all the predictions for 2014 made by all the different pundits would be a monumental task. Evidently, if you can fog a mirror, you were obligated to make a prediction. Instead, let’s look at a few of the most common predictions and see how they will play out.

Big data becomes a mainstream thought

Big data is already a mainstream thought, even if it isn’t yet typical in terms of production capability. The idea of accessing all the data sets, handling them flexibly, and using them to drive analytic insight is in my experience an ordinary notion in most big enterprises today as far as thinking about what has to happen. Still, many enterprises continue to struggle to turn opportunity into realized gains. Skill sets and analytics know-how continue to obstruct the move from idea to production. This obstacle is the single, primary focal point in most C-level executive conversations I have had with client organizations so far this year. I think when we look back, the number-one big data story of 2014 will be how big data became normal but skills remained an issue.

People will continue to give up privacy in return for enhanced applications

Unfortunately, that privacy will be forsaken for improved applications is not so much a prediction as it is a restatement of historical fact. Many consumers are very ready to give up their privacy in return for either enhanced or more intimate experiences than they previously had, or to avoid spending money. In fact, the willingness to forgo privacy for these reasons is a bit of a feedback loop in which the return for giving up privacy tends to result in an increasing willingness to continue to do so. So long as customers value free over their privacy, this one is in the bag.

Big data as a service becomes increasingly commonplace

One could argue that there is already enough big data history—even though 2014 is just three months old at the time of this writing—to know that this prediction is already well underway. Client organizations are not only interested in highly agile style approaches for delivering initial projects, they’re also very focused on agile style approaches for running production instances, especially once they are ready to move beyond the initial projects. And both cloud computing and as-a-service models play here quite well. IBM is certainly following the course this prediction has set, as evidenced by its acquisitions of SoftLayer and Cloudant, and of course the whole as-a-service IBM Watson™ ecosystem. And, of course, it isn’t just IBM. The infrastructure-as-a-service (IaaS) and platform-as-a-service (PaaS) trends are not going away anytime soon. As mentioned previously, as big data becomes normal there will be a natural growth in service models as well.

Open source continues to be a key driver of innovation and disruption

There is not much room for debate on the prediction that open source continues to keenly influence innovation and disruption; we already see it taking place. The bigger takeaway is that we can expect the continued evolution of enterprise spending to transpire. Certainly, infrastructure matters, and that requirement is not going to change, but a move toward solutions tightly aligned to specific business outcomes for the enterprise software companies will keep unfolding. This move is expected to be disruptive, and not all current vendors are likely to survive it. 2014 will continue to see the democratization of access to processing and analytics tools, but that does not mean the know-how to use them will be freely available. While there is room in the market for a couple of vendors per major open source project, a shake out between those who are going to reach critical mass and those who will be relegated to a niche players will transpire. Keep in mind that as open source projects mature, the need for firms to backstop gaps in market knowledge of how to use them historically diminishes. Please share any thoughts or questions in the comments.