It's time to invite data scientists to the board room

GM, Analytic Platform, IBM

Hadoop as a data platform has reached a new level of acceptance and overall maturity. It’s time to open the full potential of data.

When I saw T-shirts, jeans and hoodies (sweatshirts for the uninitiated) on display at the 5th Annual Strata Conference in San Jose, I knew right away that I was in for an interesting conference. Most of the conversations were focused on machine learning, Apache Spark and the Open Data Platform Initiative for Apache Hadoop.

This was clearly a data science crowd consisting of domain experts, data engineers, statisticians and application developers—all looking to find out what’s next to power their analytics projects.

When the President of the United States made a video appearance, the audience took notice. Why does POTUS care about Hadoop, Spark and APIs Because, as he said: “Understanding and innovating with data has the potential to change the way we do almost anything for the better.” So true!

We at IBM see this change with clients around the world. Data, and the insights derived from it, is the basis of competitive advantage, and we see more and more examples of Hadoop playing a central role for data scientists to enable business transformation. In fact, as Rob Thomas and I discussed in a recent interview, Hadoop is no longer a technology only topic. CEOs are asking about Hadoop. It’s now a boardroom discussion.

As Hadoop moves beyond niche use cases in engineering and into industry-specific applications, it becomes imperative to harden the platform and invite more data scientists to uncover patterns that are valuable to business. For this reason, we were a founding member of the Open Data Platform (ODP) initiative to standardize the core Apache Hadoop components as a solid foundation for the future. Every vendor is talking about ODP, whether they joined it or not.

Key to uncovering deeper insights is the ability to include more people, with varying diverse backgrounds, to work with data the way they want to. Whether it’s through visual interactions or a programmatic environment like R, the key to success is applying domain knowledge thru analytics on the data to create value. This week, we introduced IBM BigInsights Data Scientist to make it easier to analyze the data, not the data platform.

It is clear to me that the community is moving beyond the parts and pieces conversation and into data science to create sustainable value for society. It is now time for us to invite the data scientists, hoodies and all, to the boardroom to make data make sense.