Measuring Maturity of Big Data Initiatives

Discover the right way, and the wrong way, to integrate big data projects into an organization

President, Sixth Sense Advisors Inc.

The buzz and hype around big data has begun to settle down, and there is a sense of uneasiness about the topic and how to define the success criteria for big data programs within enterprises. On one end of the spectrum is a departmental- or individual-driven effort to understand and integrate big data, which is akin to a shadow IT-style effort. On the other end of the spectrum is a line-of-business–driven initiative that captures the attention of the enterprise. Both endeavors are steps in the right direction, but they can also be the wrong steps at the same time.

They are the right steps from the standpoint of needing to start somewhere and make a big data initiative happen. However, they are the wrong steps from the perspective that in most cases the attempts fail because of a lack of understanding or planning, setting incorrect expectations, or not implementing the appropriate architecture.

How can a big data program be implemented in the most appropriate manner? And who can guide that journey?


A modeling approach

Using a reference architecture model is one approach to understand a well-suited implementation of big data. The underlying issue is that the newness of the big data space can limit robust reference architectures. The other way to create a well-suited strategy and implementation is to follow a maturity model that provides the inputs necessary to create a road map–driven architecture for implementing a big data program.

A first reaction to hearing the phrase, “big data maturity model,” may very well be, “What?” Yes, a maturity model for big data, and one that is the first of its kind, can be oriented toward each aspect of the big data program.

In any measurement process, several categories exist across which the maturity of the enterprise can be measured, and with big data there is no difference. Think of three components in a triangle, with technology at the bottom, process in the middle, and people at the top. When drilling down into these three core areas, the following subcomponents emerge:

  • Alignment: What is the alignment to the big data effort across the enterprise?
  • Architecture: How advanced is the big data architecture, and to what degree do groups adhere to architectural standards?
  • Data: To what degree does the data provided by the big data environment meet business requirements?
  • Data governance: Does the organization have a data governance model, and how effective is data governance with the big data program?
  • Delivery: How aligned are reporting and analysis capabilities with line-of-business user requirements, and what is the extent of the usage?
  • Development: How effective is the big data team’s approach to managing projects and developing solutions?
  • Measurement: What are the different strategies for measuring success? Who defines these measurements and their associated success criteria?
  • Program governance: Does the organization have a program governance model? How effective is the program governance with the big data program?
  • Scope: To what extent does the big data program support all parts of the organization and all potential users?
  • Skills: Does the organization have the skills needed to support the big data initiative?
  • Sponsorship: To what degree are big data sponsors engaged and committed to the program?
  • Statistical model: What are the requirements for measuring and monitoring the performance of the enterprise? What are the different techniques used today and expected from big data technologies and solutions?
  • Technology: What are the key capabilities available within the organization for big data technologies?
  • Value: How effectively does the big data solution meet business needs and expectations?
  • Visualization: What are the key requirements for data visualization and analytics delivery?

The goal of the big data maturity model is to provide a capability assessment tool that is focused on these key areas as well as additional subcomponents. The maturity model can help guide development milestones and avoid pitfalls.

While the model resembles traditional data warehouse implementations, the context of applying the maturity algorithm and its questionnaire are focused on expanding the data platform for the organization. There are several complex situations that exist today, especially with big data, the platforms, and its applications. In addition, several non-Internet organizations are wondering if there is real value in big data and the applications associated with it. These are all issues that need to be addressed, and the maturity model can provide companies with the data and related opportunities.


A deeper dive into the maturity model

A course at The Data Warehousing Institute (TDWI) Conference has been crafted to encompass the points highlighted here and provide an overview of TDWI’s big data maturity model. The techniques to be discussed in the course are all practical insights learned from big data implementations across the globe over a period of more than eight years.

Please share any thoughts or questions in the comments.

Editor’s note: This article by Krish Krishnan, president and chief executive officer (CEO) at Sixth Sense Advisors, Inc., is offered for publication in association with the Big Data Seminar 2014, September 17–18 in Washington, D.C., sponsored by Data Management Forum.


[followbutton username='IBMdatamag' count='false' lang='en' theme='light']