How Does Big Data Make the World Go Round? – The Answers Revealed

Social Business Manager, IBM

In June, we asked the very important question, how does big data affect our world?

In industries throughout the world, executives recognize the need to learn more about how to exploit big data. In fact, 28 percent of organizations today are already tackling these big data questions. But despite what seems like unrelenting media attention and over-hype, it can be hard to find in-depth information on what organizations are really doing to benefit their businesses.

To help the industry better understand the big data market and differentiate the reality from the hype, the IBM Institute for Business Value partnered with the Saïd Business School at the University of Oxford to conduct the 2012 Big Data @ Work Study.

After surveying 1,144 business and IT professionals in 95 countries, and interviewing more than two dozen academics, subject matter experts and business executives, it was determined that 63 percent – nearly two-thirds – of respondents report that the use of information (including big data) and analytics is creating a competitive advantage for their organizations. This compares to 37 percent of respondents in IBM’s 2010 New Intelligent Enterprise Global Executive Study and Research Collaboration – a 70 percent increase in just two years.

To help gather a full understanding of these findings, we chatted with Rebecca Shockley, Business Analytics and Optimization Global Research Leader for the IBM Institute for Business Value.

1. What did you learn about this current state of big data?

The concept of big data is not new. Many organizations have been tackling the notion of big data for years. However, two important trends make this new era of big data quite unique:

  • The digitization of virtually “everything” now creates new types of large and real-time data across a broad range of industries. Much of this is non-standard data: for example, streaming, geospatial or sensor-generated data that does not fit neatly into traditional, structured, relational warehouses.
  • Turning that big data into action is what matters. Today’s advanced analytics technologies and techniques enable organizations to extract insights from data with previously unachievable levels of sophistication, speed and accuracy.

2. Big data is distinguished by the three V’s – volume, variety and velocity. However, the study suggests that organizations should really be concerned about a fourth, veracity. Why is that?

Yes, perhaps the most interesting revelation from this study is the emergence of Veracity, or data uncertainty. For example: sentiment and truthfulness in humans; GPS sensors bouncing among the skyscrapers of Manhattan; weather conditions; economic factors; and the future. When dealing with these types of data, no amount of data cleansing can correct for it. Yet despite uncertainty, the data still contains valuable information. The need to acknowledge and embrace this uncertainty is a hallmark of big data.

They need to create context around the data. One way to achieve this is through data fusion, where combining multiple less reliable sources creates a more accurate and useful data point, such as social comments appended to geospatial location information. Another way to manage uncertainty is through advanced mathematics that embraces it, such as robust optimization techniques and fuzzy logic approaches.

Humans, by nature, dislike uncertainty, but just ignoring it can create even more problems than the uncertainty itself. In the era of big data, executives will need to approach the dimension of uncertainty differently. They will need to acknowledge it, embrace it and determine how to use it to their advantage; the one certainty about uncertainty is that it is not likely to go away.

3. The study said that almost 30 percent of organizations are in the early stages of big data efforts. In what areas were these organizations focused?

By analyzing the responses of these early adopters, five key study findings show some common and interesting trends and insights, both on the business and technical side:

  • Across industries, the business case for big data is strongly focused on addressing customer-centric objectives, such as improving the customer experience and understanding customer preferences;
  • Two thirds of respondents said their big data journey begins with a scalable, extensible and secure information management foundation;
  • There is still tremendous value locked away in internal systems. Therefore, organizations are beginning their pilots and implementations by using existing and newly accessible internal sources of data. Conversely, there was a very small impact of social media data within these initiatives;
  • Big data by itself does not create value. Advanced analytic capabilities are required, yet often lacking, for organizations to get the most value from big data; and,
  • As organizations’ awareness and involvement in big data grows, we see four stages of big data adoption emerging among organizations – educate, explore, engage and execute.

For more discussion of the big data study and an inside look at new infographics and videos created around these findings, join Rebecca on Friday, November 2 for a day long Facebook chat on the People for a Smarter Planet Facebook page. A complete overview of findings will also be shared on the November 7 web seminar featuring Michael J. Schroeck, IBM’s Partner/Vice President for theGlobal Information Management Foundation and Dr. Janet Smart, Senior Research Fellow at Saïd Business School.