Darwin meets big data: Evolution of roles and rules
At a recent IBM event focused on building confidence in big data, a highlight was a wide-ranging discussion by a panel with very diverse backgrounds and industries. The panel comprised an industry analyst with an information governance specialty (Michele Goetz of Forrester Research), a chief data officer (Heather Wilson of AIG), a master data management strategist (Rob Harris of Dell) and a senior director of data governance and quality (Al DeCarlo of Express Scripts). Evolution was central to the discussion: the evolving roles of data leaders, the evolution of business users of data, and the evolution of approaches to data governance in the era of big data.
Here are a few noteworthy observations based on the panel’s comments.
- Key data roles need to sit between business and IT. It’s important that information governance is driven by the needs of the business, even though some governance activities are led by IT.
- Power users of data are creating new challenges for those who manage the data. These power users—including data scientists and others across the business—are increasingly knowledgeable about the data and what they want from it. IT is challenged to keep up with accelerating demands for delivering the right data at high speed to enable business operations and analytics.
- Metrics are key to demonstrating a basis for confidence in data. It’s important that data projects have well defined metrics from the start, so that later, the success of the projects can be clearly measured against agreed-upon objectives. When business leaders and power users ask whether or not data can be trusted, there must be evidence to support confidence.
- New approaches are needed to incorporate the best of data from new big data sources with more traditional data. In one company, that meant a clear delineation between a platform for data from trusted sources and an important but separate discovery platform to investigate data that wasn’t yet trusted. In another organization, users were allowed 90 days to work with a new data source and make their own determinations about the value of the data and the importance of incorporating it into their master view. After that, the data would no longer be available unless it could be placed under standard governance practices on a platform for trusted data.
From other sources as well, we’re starting to hear more about the need for increasing flexibility in governance. This translates to tighter controls for some types of data, looser controls for other types, but some sort of governance for all. Have you observed the evolution of data roles and rules in other organizations?