Data Scientists: Bridge the Cultural Divide with BI Practitioners

Big Data Evangelist, IBM

One of the myths I deflated a few blogs ago was this prevailing notion that, as I expressed then, "data scientists are just an elite bunch of precious eggheads."

Well, maybe I was slightly exaggerating the myth for comic effect, but that statement, in spirit, sums up how traditional business intelligence (BI) professionals sometimes view data science and its practitioners. If you do analytics for a living, you can't help but notice the cultural divide between data scientists and BI practitioners. It's too soon to say whether the rift is growing, shrinking, or remaining the same over time. However, it's very real, and stems from many factors.

For starters, data science is in vogue, and it's increasingly the ticket to career advancement in analytics. If your specialization is only in any of the traditional BI disciplines – including reporting, dashboarding, and online analytical processing – you might have a stable job but won't necessarily be considered for the coolest new big data projects. Clearly, jealously might set in when you feel that your BI career is losing steam while the upstarts pass you by.

Also, data scientist skills are at the heart of big data, and organizations are growing their investments in those technologies, tools, and skillsets as a key internal competency. By contrast, traditional BI is increasingly either being outsourced to cloud/SaaS providers and/or being managed directly by business analysts and subject matter experts through the new generation of self-service visual exploration tools. Longtime BI pros might feel marginalized, wondering whether there's a future for them with their current employer in their current job.

Furthermore, data scientists build a more specialized type of analytic application – often dubbed "advanced" – than do traditional BI developers. This state of affairs can easily feed into a bogus "we're smarter and more skilled than you" climate of cultural snobbishness and resentment. The steep learning curve associated with the more arcane branches of statistical, mathematical, computational linguistics, and artificial intelligence may operate as a cultural barrier, keeping old-school BI people from joining a seemingly elite new fraternity of data-science brainiacs.

Yet another issue dividing these two camps is open source. The new generation of big data data scientists is far more inclined to favor Hadoop, NoSQL, R, and other open-source platforms and tools than traditional BI professionals (though open-source has certainly made inroads in that world as well). No, open vs. closed source is not a holy war in the analytics world, but it's definitely an orientation that separates newbies from old-schoolers. As a result, data-scientist upstarts and BI stalwarts tend to have different frames of reference, speak different lingos, subscribe to different newsletters, acquire different skillsets and certifications, attend different conferences, engage with different vendors, and so on.

There's another issue at work here: divergent orientations toward development vs. operations. Data scientists tend to work special projects, focused on seeking answers to complex business questions with ten zillion variables. BI professionals, by contrast, focus their efforts on ongoing functions such as production reporting and data governance. In almost any organization, a special projects person might come off looking like a preening fancypants to the operations people who endure the daily grind. Through the other end of the telescope, the projects people may regard operations folks as little more than clock-punching plumbers.

I'm going to close out this particular post by extending a thought I put forward in yet another prior blog. To the extent that you're establishing a data science center of excellence (CoE), it should align seamlessly with your BI CoE. To bridge the cultural gap, you should be encouraging data scientists and BI pros to engage with each other through common collaboration tools, communities, and events. For example, regular lunch-and-learn sessions can be a great forum for getting both groups up to speed on what's going on in each others' world. These events might also be a recruiting opportunity for data scientists seeking the operational experience and discipline that a longtime BI professional offers. This will become even more critical as advanced analytics become embedded in operational applications in marketing campaign optimization, next best action, and experience optimization.

Be sure to nip this cultural polarization in the bud. Don't let data scientists and BI veterans drift into mutually suspicious, turf-protecting camps. You'll be sabotaging your big data initiatives if you allow that to happen.