Protecting the new elephant in the room: Security, big data and Hadoop

Product Marketing, Information Integration and Governance, IBM

It happened again today. In a release just recently, Bell Canada revealed that the usernames and passwords of at least 22,400 small business customers had been compromised through a third-party supplier.   

Based on the number of business accounts exposed, Bell could probably be considered a small blip as far as recent data breaches go. Still, it provides yet another proof point that the frequency of attacks is growing rapidly in environments rife with sensitive data. 

Our favorite yellow elephant, Hadoop, is quickly proving to be one of those environments. Whether it is analysis of customer data, investigation into healthcare data, analytics on operations or countless other projects, it’s a good bet that there’s something sensitive in our big data and analytics project. 

So, how seriously are we taking the threat when it comes to those companies using Hadoop? Not very, based on some recent data collected by Merv Adrian at Gartner. In fact, according to the research, it looks like only two percent of survey respondents viewed a lack of robust security as a barrier to their Hadoop project.  It’s likely that the causes for this vary by company and project. 

Adrian points to one possible explanation as a misunderstanding of Hadoop’s security features, perhaps giving project teams a false sense of security. We’ve heard this and a number of other reasons from our clients and the broader market. 

lock data.jpgHere are some of the most common reasons that we hear Hadoop projects continue without security protections in place:

  • "We’re not using any sensitive data in our big data project."
  • "This is just a sandbox environment, so we can easily control the environment and the users."
  • "Meeting time-sensitive business demands takes priority over building the appropriate protections."
  • "We have some level of protection, but we’re not too sure what’s needed."

Stay tuned: in a series of upcoming blog posts, I’ll explore these responses and outline a few security considerations to help determine whether your organization is protecting the elephant in the room.