Big Data & Analytics Heroes

Fern Halper

Research Director for Advanced Analytics at TDWI
Post Comment

Fern Halper

It’s not simply about cutting costs or becoming more efficient or understanding behavior. It’s about taking that next step to think about what they can do with that data that will drive top- and bottom-line impact.

Fern Halper, research director for advanced analytics at TDWI and this week’s big data and analytics hero, discusses her thoughts on the biggest challenges for companies when they decide to get started with big data and analytics and explains why she believes tomorrow’s generation should acquire analytics skills, no matter the degree—because it’s going to be hard to escape the data deluge.

What are the biggest challenges with getting started with big data and analytics?

I recently completed my latest TDWI Best Practices Report that discusses next-generation analytics. In that study, we asked respondents to name their top analytics challenges; they could select up to three challenges. The top challenge was the lack of skilled personnel. Over half of survey respondents cited this lack as a challenge. I’ve seen this result in previous TDWI research as well. People are concerned about building out skills in big data technologies such as Apache Hadoop, newer kinds of databases, and big data and advanced analytics. They are concerned that they don’t understand the technologies and that they aren’t quants. There are other challenges as well, and they tend to be organizational in nature. They include lack of budget, lack of executive support and overcoming cultural issues—to mention a few. Often the cultural issues are harder to overcome than the technology issues.

How are big data and analytics changing business strategy?

It’s really interesting—when you talk to people who are already thinking about or actually doing something with big data and more advanced analytics such as predictive analytics, one of their top drivers is to try to monetize their data. I’m not necessarily talking about providing new services that companies sell externally. However, the thought process is more innovative. It’s not simply about cutting costs or becoming more efficient or understanding behavior. It’s about taking that next step to think about what they can do with that data that will drive top- and bottom-line impact. That said, many organizations are still quite early in their big data and big data analytics efforts, which often means that they don’t have a strategy in place.

What is the market still missing for big data and ­analytics to really deliver ROI?

I’m not sure that the market is missing something, per se, at this moment to deliver ROI. The problem is that maturity in big data and big data analytics isn’t something that happens overnight. We’ve been assessing a great many companies using our big data maturity model that IBM is sponsoring. Our stages of maturity go from nascent to pre-adoption to early adoption to corporate adoption to visionary. We look at how organizations fare across five dimensions—organization, infrastructure, data management, analytics and governance.

The reality is that on average, many organizations are still in the pre-adoption stage. This stage means that they may have some proofs of concept in operation, but they typically don’t have a good funding stream in place. Analytics is happening in pockets across the organization, and these organizations are not using more than a few data sources. If they’re doing advanced analytics—such as predictive analytics—they may be doing it across large volumes of structured data.

What’s holding these organizations back? First, they are not organized to execute; there are no teams, for example. Some are struggling to get key people in place. Many don’t have anything resembling a center of excellence. Many feel overwhelmed by their data, and they don’t have a solid data management plan approved. They’re still using their data warehouse for most of their data management. Data warehouses are good when you understand what reports and dashboards you want to produce. They’re not the best when it comes to iterative analytics. These organizations need to make some new investments.

Should tomorrow’s generation acquire analytics skills no matter the degree? Why, or why not?

I’ll admit that I’m biased, but I think if you’re in a data-intensive field, which many fields will be in the future, then you should have some idea of how to analyze data. Think about it. Customers generate data; patients generate data; machine components generate data. The list goes on. It’s going to be hard to escape the data deluge. College theoretically provides you with critical-thinking skills, but this skill set is different than actual data analysis know-how. Even if you’re the recipient of analytics, having some analytics skillsit is useful. Tomorrow’s generation should acquire analytics skills, no matter what the degree.

Do you think big data and analytics will handle the data growth in 10 to 15 years, or would we need another shift in technology? Why?

If you look at how data has grown over the past few years, I’d have to say that technology will shift. EMC, for example, estimates that the digital universe is growing 40 percent a year. It estimates that in the next five years—by 2020—there will be 44 trillion gigabytes of data. Tens of billions of connected devices are estimated to be in existence in the next five years. Imagine what it will be like in 10 to 15 years. Think of all of the data that will be generated that companies may want to manage and possibly analyze. Can our current systems handle it? I think that shift is why there is so much research going on about new kinds of materials, technologies and approaches to support this increasing amount of data.


Data scientists everywhere are getting up to speed on Apache Spark. To beef up your Spark skills, enroll in the course, IBM Big Data University on Spark Fundamentals. You can also peruse other Big Data University online courses. And to hone your skills even further, sign up for the forthcoming IBM for Apache Spark as a Service on IBM Bluemix. To learn more about Hadoop, in which Spark is an integral component, check out this recent Forrester Research study, in which the independent analyst firm stated:

IBM assembles an impressive set of capabilities, putting predictive at the center. No matter how an organization wants to get started with predictive analytics, IBM has an option for them. The solution offers one of the most comprehensive set of capabilities to build models, conduct analysis, and deploy predictive applications: both on premises and in the cloud. With customers deriving insights from data sets with scores of thousands of features, IBM’s predictive analytics has the power to take on truly big data and emerge with critical insights.