Make sure your big data isn’t bad data

Analytics Solutions Social Specialist, IBM

Social media advertising is at an all time high, but how many legitimate eyes on content are advertisers really getting?

Investors have expressed concern that too many registered accounts are fraudulent or spam, or are accessed through third-party systems and so are never exposed to paid advertising. Researchers estimate this number could be as much as 20 million fraudulent users and company estimates report that up to 13.5 million of its registered users may be spam accounts. But even this report is muddled with uncertainty: it notes that the estimation may not be accurate, and may even be a conservative estimation.

With high advertising costs at stake, advertising investors are determined to be certain that they are getting value in return. Yet, considering the sheer volume of user data being generated and handled by companies in the digital age, uncertainty is ubiquitous. 

Data confidence in the news

credit card bw_0.jpgIn recent news, a particular social media giant was forced to confront the debate over just how many eyes its advertisers are actually receiving for their ad dollars. This story addresses “correctness,” one of the seven “essential elements of confidence” outlined by Nathaniel Rowe, a research analyst for Aberdeen Group. This element, and the remaining six, can be seen at work time and time again in companies’ successes and blunders when handling big data.

“Security,” considered the fourth most important element of confidence by surveyed companies, gained attention in the news when a large retailer failed to protect customers’ payment card information around the holidays. This case demonstrated that data confidence is about far more than just gleaning effective insights (though this remains top of mind)—big data confidence demands built-in security measures so that sensitive information contained in the data can be used without fear of loss.

Decision making with confidence

According to a recent Aberdeen study, about 50 percent of companies say that poor data is making their decisions inaccurate. While absolute certainty in big data and analytics may be out of reach, David Corrigan, IBM’s director of product marketing for Infosphere, suggests that certainty is not necessary—only an understanding of how much confidence you have in your data, and how much is required to make a reliable decision using the data.

Before, data confidence was measured according to one or two “essential elements,” or even just intuition. New developments in data confidence measurement have made it possible to assign a standardized, numerical score that takes into account all six established elements of confidence. This allows business decision makers to know precisely how confident they can be in their data, to evaluate what level of confidence is required and to make more effective decisions from more trustworthy big data.

To learn more about developing confidence in your big data, the seven essential elements of confidence and the data confidence scoring system, see Nathaniel Rowe and David Corrigan’s “How to Calculate Data Confidence”.

Related links