Big data: Handle with care

Director, Product Marketing, InfoSphere, IBM

There have been a number of high-profile missteps with big data in the news. One recent example is the story of OfficeMax and the “Dead Daughter” marketing mailing (read more about it here OfficeMax accidently sent a mailing with “Mike Seay – Daughter Killer in Car Crash” in the envelope window. Mr. Seay’s daughter had in fact been killed in a car accident three years ago.  Naturally this was disturbing to him and his wife, and so they asked some fairly basic questions. Why do you have this information? Where did you get it? And how did this happen? Unfortunately those straight-forward questions are incredible difficult for organizations to answer for one simple reason – they don’t track and govern big data adequately. OfficeMax is hardly alone. This could have happened to any organization utilizing big data, and in fact there are numerous similar stories in the news. Will there be a backlash in the adoption of big data?

Some organizations are concerned about using big data and creating unintended consequences. There are two fears – coming across as ‘big brother’ and data leaks and misuses that become public, such as the OfficeMax story. Both can have a serious effect on customer retention and loyalty. In fact, studies indicate that 66 percent of your customers would leave if you lost or mistreated their data. It turns out that the mysterious asset of ‘brand loyalty’ can be quantified after all. Fearing the negative consequences of big data is a healthy exercise, but abstaining from big data projects is not. For organizations sitting on the sidelines and waiting until all the unknowns are known, you picking the wrong strategy. You’re missing the window of opportunity. And your competitors are seizing it. The best way forward is to begin big data and analytics projects and address those concerns, mitigate those risks, and make risk-aware decisions. 

Big data governance is the best way to mitigate those potential risks. Take the OfficeMax example. Big data governance would have caught the name issue – standardization would very easily recognize that ‘Mike Seay – Daughter Killer in Car Crash’ is not a valid name pattern. Governance would also answer the two questions asked by Mr. Seay – where did you get that information (metadata and data lineage) and what are you using it for (data usage and governance policies). In the era of big data, there will be greater demands for transparency. Big data governance puts organizations in a position where they can answer those questions easily for both internal and external users. In addition to avoiding missteps, big data governance also helps organizations discover and understand big data more rapidly, which gives them a tremendous benefit in acquiring and analyzing the right information. Studies indicate that 80 percent of each big data project is spent finding, understanding and fixing data sources and feeds; big data governance therefore could have a tremendous impact on improving that statistic. 

Big data and analytics projects are experiencing a few teething pains. Does that mean you should stop eating? Absolutely not! It simply means that organizations must invest in big data governance alongside big data and analytics technologies. The quality of the data matters every bit as much as the insight – in fact, the insight is critically dependent upon high quality data. The ability to handle big data ‘with care’ yields financial benefits, avoids the bad headlines and ultimately it can be a differentiator for organizations. Customers will leave organizations that mistreat their data, and flock to companies that safeguard it. Big data governance enables organizations to handle big data with care.

Learn more about big data governance and watch my video chat with David Pittman for more on data gaffes: