2014 has been hailed as the year when big data adoption goes mainstream. As adoption accelerates, I’m starting to see a change in the thinking about big data. More and more business and government leaders seem to be taking a step back and asking questions like these:
- How can I understand the connection between data from new sources and the data I already have in the enterprise?
- How can I integrate the best of big data with some of my internal systems?
- How can I determine which data is worthy of my confidence?
- How can I protect personally identifiable information when I introduce big data into the mix?
All of these questions point to a new recognition that in order to get the value that the enterprise wants, and expects, from big data, it is important to deploy information integration and governance (IIG). As Forrester analyst Michele Goetz has said: “You can’t do big data without IIG!”
Information integration and governance is not a new concept. Organizations have been integrating diverse data within the enterprise for years, for example, to prepare data from multiple applications to come together in data warehouses. They have been standardizing, matching and de-duplicating data so that the data in the warehouse, or other target environment, is the best it can be. For master entities like customers, they have been further consolidating and enforcing rules to create an authoritative data source. Organizations have been managing data across its lifecycle according to business and compliance rules for data retention. And they have been securing and protecting data from threats within and beyond the enterprise.
But it’s quite a recent phenomenon for organizations to realize that all of these integration and governance capabilities can be applied—in appropriate measures—to big data. As long as big data lives totally in the ether beyond our organizations, chaos may prevail. But when we choose to find the jewels in the big data and use them to help us understand our customers better, to target our marketing with more clarity and to gain more insights into our competitors, we need to apply the best practices of information integration and governance, adapted to suit the new data types and the intended uses for the data. Well understood, well integrated, high-quality data that is secure and protected can deliver big benefits to any organization.
- White paper: Who’s Afraid of the Big (Data) Bad Wolf?
- Book: Harness the Power of Big Data: The IBM Big Data Platform
- White paper: Integrating and Governing Big Data
- Forbes article: 2014: The Year Big Data Adoption Goes Mainstream in the Enterprise