Big Data Relationships

Successful big data initiatives require solid statistical relationships and constituent skill sets

President, Business Over Broadway

The success of big data projects depends on two types of relationships—statistical and personal. Organizations can reveal statistical relationships among variables in the process of building useful algorithms to improve how they operate. And in their quest to glean insight and gain value from data, organizations need to consider the quality of the relationships among their important constituencies that include chief information officers (CIOs), chief marketing officers (CMOs), and data scientists.

Relationships among variables

In big data rhetoric, the value of big data rests in helping organizations find patterns in data. These patterns take the form of statistical relationships between two or more variables. Analytics practitioners want to know how variable A is related to variable B. When there is a change—increase or decrease—to variable A, does variable B change—increase or decrease? Marketing executives, for example, want to know if customer satisfaction—variable A—is related to customer buying behavior—variable B. They then use these patterns, or statistical relationships, to build predictive models that help them optimize an important outcome, such as improving tech support leads to increase cross-selling.

The practice of finding patterns in data relies on the access to different sources of data. Sticking to a single silo as the sole data source limits the ability to uncover important patterns in data. Analytics has always been about finding relationships among variables; with big data technological advancements, the relationships people pursue are limited only by their imagination.1

Similarly, big data projects aimed at improving US hospitals show the value of merging disparate data sources, including patient experience, health outcome metrics, process metrics, and financial metrics. While each of these four data sets alone provides much information to help healthcare consumers,2 integrating them enables understanding the interplay of these different metrics.3 For example, we can test new hypotheses with this broader data set:

  • Is patient experience related to health outcome metrics? Yes, they are related; hospitals that provide an enhanced patient experience also have higher survival rates than those that don't.
  • Is healthcare spending related to the customer experience? No relationship is found between patient experience and Medicare spending (see figure).

Big Data Relationships – figure

Patient experience by Medicare spending per beneficiary

In business, variables are typically housed in disparate databases. Customer feedback data is kept separate from customer buying data. Employee data is isolated from supply chain data. Operational data is not linked to energy expenditure data.

A recent study found that 20 percent of IT decision makers said data silos are the biggest barrier to exploiting data for business advantage.4 Sure, the owner of each data source can still find statistical relationships among variables in his or her respective data silos—that is, identify correlation between employee experiences and employee loyalty. However, the value of big data analytics increases when organizations can link and merge data from different sources. In business, getting access to the right data requires working with the owner(s) of the data.

Relationships among people

Data doesn’t generate and analyze itself. People do. By their very nature, big data projects can require many different constituencies, each providing unique expertise in analytics projects. CMOs and CIOs need to be partners in extracting value from data.5 The CMO defines the business goal of the analytics initiative, and the CIO assesses its feasibility. The resulting analytics project requires each party to work together to clearly articulate and balance its respective needs around the project.

Also, the shortage of people possessing data science skills is pushing companies to build data science teams,6 combining people with disparate yet complementary big data skills such as statistical analysis, computer engineering, and business acumen, to name a few.7 Working together toward a successful outcome of the analytics project requires effective teamwork from many different people.

Researchers have found that the ways employees relate to one another can have a substantive impact on the success of an organization’s big data projects. They found that teamwork quality was related to the success of innovative projects. Teams that were better in six areas—communication, coordination, balance of member contribution, member support, effort, and cohesion—than their counterparts had more successful team project outcomes as well in terms of both effectiveness and efficiency measures of success.8 So, putting practices in place that encourage team behaviors that are conducive to success is highly recommended.

Additionally, IBM researchers found that employee trust plays a significant part in how industry leaders optimize analytics. They found that industry leaders—those who outperform industry peers—demonstrated a high degree of trust among individual employees—60 percent among executives and 53 percent among business and IT executives.9

While effective teamwork is essential in a big data project, recent research paints a pretty bleak picture for many organizations. For example, IBM found that 45 percent of CMOs said that the alignment of their department with IT is lacking.10 Additionally, CMOs and CIOs have very different, competing ideas about each other’s role in big data. In a report by, researchers found that a significant number of CIOs

think IT has the primary responsibility for investments in big data and segment analytics. And CMOs

generally think marketing has the primary responsibility.11

Thriving outcomes

Successful big data projects depend on relationships among variables and among the people who support the analytics initiatives. Big data analytics initiatives become interesting and potentially more useful when organizations look to integrate data from different silos and explore relationships among a wider set of variables. In addition, the people who extract value from the data and work together toward that end drive the success of big data initiatives. Big data projects can be more valuable and deliver greater return on investment than ever before when organizations ensure the constituencies involved work together toward a mutually agreeable business outcome.

Please share any thoughts or questions in the comments.

1 For information on how open data illustrates the value of merging and mashing disparate data sources, check out this Technology, Entertainment, and Design (TED) talk, “The Year Open Data Went Worldwide,” by Tim Berners-Lee.
2Map of US Hospitals and Their Patient Experience Ratings,” by Bob Hayes, Business Over Broadway, July 2012.
3Big Data Provides Big Insights for US Hospitals,” by Bob Hayes, Business Over Broadway, September 2012.
4Making Better Use of Data: Why It’s An Elusive Goal for Many Businesses,” a survey of 100 UK senior business-decision makers, Bull UK and Ireland, 2014. Download a report of survey results—registration is required.
5CMOs and CIOs Need to Get Along to Make Big Data Work,” by Matt Ariker and Jesko Perrey, Harvard Business Review blogs, February 2014.
6It Takes Teams to Solve the Data Scientist Shortage,” CIO Journal blog, The Wall Street Journal blog, February 2014. Subscription required.
7The One Hidden Skill You Need to Unlock the Value of Your Data,” by Bob Hayes, Business Over Broadway, February 2014.
8Teamwork Quality and the Success of Innovative Projects: A Theoretical Concept and Empirical Evidence,” by Martin Hoegl and Hans Georg Gemuenden, Organization Science, July 2001.
9Analytics: A Blueprint for Value,” Business Analytics and Optimization Executive Summary, IBM Global Business Services, IBM Institute for Business Value, October 2013. For a detailed discussion of the report, see “Creating Value from Analytics: The Nine Levers of Business Success,” by Bob Hayes, Business Over Broadway, October 2013.
10Strengthening the CMO-CIO Relationship,” IBM Data magazine NewsByte, June 2014.
11CIO and CMO Partnership Survey, executive summary,, April 2013.

[followbutton username='bobehayes' count='false' lang='en' theme='light']
[followbutton username='IBMdatamag' count='false' lang='en' theme='light']