The perils and promise of fast data

A new Forrester study, Part 1 of 2

Worldwide Offering Manager for Hybrid Data Management Platform, IBM

A new Forrester Consulting study commissioned by IBM from October 2018 shows that fast data with fast analytics is an enormous, rapidly growing resource that’s not being used to its full potential. 

Fortunately, there are concrete steps and general strategies businesses can adopt to get more value from fast data. This two-part series will dive into some of the findings from the new Forrester study, “Don’t Get Caught Waiting on Fast Data”, and how you can apply them. 

Data is changing fast 

The Forrester study found that the data landscape is changing so rapidly that most organizations are struggling to adapt. About 85 percent of organizations say that the volume of data, velocity of data, and number of data sources have “increased dramatically” over the past three years. 

But that’s not the whole picture. Fast data is not just more data per second. A key part of the study’s definition of fast data notes that it’s “any type of data that originates in applications and devices and is streamed, stored, and immediately analyzed”. 

Fast data and analytics impacts:

  • The use of data from a growing number of sources
  • Time-sensitive business needs and opportunities
  • Automated data and data management

All three factors make stark demands on data management systems to work efficiently. The variety of data types and formats places a premium on flexibility and analytic power. The need for real-time insight means that fast data solutions cannot wait on slow data ingestion; traditional extract, transform, load (ETL) processes that delay applying analytics; or moving data around to faster databases for analytics. Enterprises need efficient, well-automated systems to both speed the process and allow developers to avoid spending time on workarounds. In other words, fast data is challenging data at its very core. 

"77% of the organizations surveyed are already using fast data for some or all of their applications"

Nonetheless, 77 percent of the organizations surveyed are already using fast data for all or some of their applications, precisely because it offers such valuable insight into real-time operations, opportunities and challenges. Roughly three-quarters of enterprises say that mobile, Internet of Things (IoT) and internal data is either a “high” or “critical” priority for their business, and 67 percent say the same about external data. Enterprises clearly know that fast data is important, but they also know it’s not being used to its full potential. 

Missed opportunities

Despite the prevalent use of fast data, those same organizations struggle to meet the needs of fast data challenges with their current solutions.

This is due in part to complexity and lack of integration between analytic components. Only 15 percent of organizations believe their data and analysis tools are completely integrated today.

Fast data solutions working with high volumes of data must ingest all the data that is generated, analyze the data in real time or near-real time, and the ability to use results in applications. Yet a majority of organizations surveyed think their data solutions aren’t supporting these capabilities.

"More than half of enterprises who use fast data solutions say there is room for improvement." - Forrester

The stumbles of those fast data solutions are leading to real detriments. For example, a significant proportion of firms surveyed described each of the following problems:

  • Missed opportunities to act on perishable insights (61 percent)
  • Lower productivity of data and analytics teams (52 percent)
  • Slower time-to-market (45 percent)
  • Lower customer satisfaction (39 percent)

Promise and peril

In the first part of the Forrester study, a decidedly mixed picture of fast data analytics appears. On the one hand, the raw material — a variety of real-time data — is omnipresent, valuable and arriving in ever-increasing volumes. More than three-fourths of the firms surveyed are making use of it already. 

While the big picture is encouraging, the details still need some work. Most of the firms surveyed say their fast data analytic efforts aren’t going as well as they’d like. Opportunities are being missed. Advantages are being squandered. The poor level of integration between different data tools and the limitations of those tools are having real consequences. 

How should firms address these shortcomings and prepare themselves for fast data success? We answered those questions in part two of this series. If you want to get a head start, read the full study today, or if you are ready to take action, check out IBM fast data solutions including IBM Db2 Event Store and IBM Fast Data Platform.