Big Data and a Hierarchy of Data Needs

Don’t overlook initial capability-building stages when creating a solid foundation for analytics processes

Product Marketing Manager, IBM

Ever since the high-profile data breach at a large US retailer toward the end of 2013, the Identity Theft Resource center has recorded a whopping 666 data breaches as of mid-November 2014.1 Given such an alarming number of reported data breaches, taking another look at data security and privacy practices is not only worthwhile, but also presents an opportunity to think about how they can be adapted to a growing number of big data initiatives. Without a comprehensive data security and privacy strategy in place, detecting, responding to, and taking quick action on a data breach can be highly challenging.

A useful framework for organizations getting started with big data projects to consider is the hierarchy of needs.2 In this framework, organizations begin their data management journey by satisfying some basic physiological needs of raw, ungoverned data and progress to a stage where they can exploit data through advanced analytics and real-time feeds. They then can pass through stages that add key capabilities and maturity to the organization and program. Although this framework appears to be primarily applicable to organizations building their data management program from the ground up, the following valuable inferences can be drawn for organizations not taking this path in their journey:

  • In a big data initiative, an organization’s responsibility begins the moment data is accessed and gathered. Data needs to be governed and secured before proceeding to the analytics stage.
  • If an organization has a data management program in place, data security and privacy practices should be reviewed and adjusted to suit big data requirements.
  • To get consistent and reliable end results through analytics, organizations need to pass through each of the stages in the order specified. If data governance, security, and privacy capabilities are not built in during the early stages, organizations may not achieve consistent results from analytics in the long run.
  • Data security and privacy capabilities support analytics initiatives over long-term periods. Therefore, these capabilities need to be sufficiently comprehensive and flexible to adapt to changing security scenarios. Once an organization has the structure in place, it can’t change the structure’s foundation.

Security and privacy for dynamic enterprise environments

The importance of having a comprehensive and flexible data security solution cannot be stressed enough. From human errors to advanced persistent threats (APTs), data security threats are continuously evolving, especially in the dynamic big data ecosystem. The kinds of data breaches organizations face today are the result of the growing stealth and sophistication of criminals.

A proactive and holistic approach is essential in a big data environment to help mitigate risk, demonstrate compliance, and prepare for additional regulatory constraints without interrupting vital business processes or daily operations. Organizations can engage the following stages to develop a proactive approach to data security and privacy (see figure).3

  • Discover and classify sensitive data: Discover sensitive data and create a heat map for protection efforts.
  • Harden the environment: Obfuscate sensitive data through data masking and encryption. Evaluate systems to ensure they enforce the security policies.
  • Secure and continuously monitor data access: Detect unauthorized or suspicious activity, and alert key personnel based on policy-based controls.
  • Protect data and remain vigilant: Facilitate compliance processes through pre-configured reports and other capabilities, and minimize technical and business disruptions.

This approach helps organizations navigate through the foundation stages in the framework based on a hierarchy of needs4 and integrates data security practices in analytics processes. In addition to protecting data, this proactive approach helps organizations tailor policies and processes for different types of data and the various information needs of internal and external stakeholders without compromising business process efficiency.

Big Data and a Hierarchy of Data Needs - figure

Developing a proactive approach to securing and privatizing data

The importance of a foundation for framework implementation

As mentioned previously, organizations in the real world sometimes overlook the initial capability-building stages and directly proceed to integrate big data into their analytics processes. Skipping the stages required to build a solid foundation is akin to a piecemeal approach to big data and analytics. At best, organizations achieve suboptimal results across projects, and at worst they become victims of a data breach. The right approach requires a good data security foundation and regular reevaluation of its capabilities to always remain a step ahead of cybercriminals.

Please share any thoughts or questions in the comments.

1 2014 Data Breaches, Identity Theft Resource Center, August 2014.
2,4A Framework to Map and Grow Data Strategy,” by Theresa Kushner and Maria C. Villar, Information Management, November 2010.
3Four Steps to a Proactive Big Data Security and Privacy Strategy,” IBM Software ebook, June 2014.