Embracing big data practitioners in 2015

Manager of Portfolio Strategy, IBM

A variety of technologies are available to perform real-time queries and event processing with speed; these include in-memory databases, operational intelligence platforms and business process management. These technologies optimize queries, deliver business rules faster and support flexible and dynamic responses. 

The next revolution in event processing is stream computing: delivering sophisticated analytics at lightning speed. Stream computing exploded in 2014 because it allows organizations to predict previously unseen situations without requiring predefined business rules to respond. The inaugural Forrester Wave on Big Data Streaming Analytics Platforms calls this a “windshield view.”

High velocity flows of data from real-time sources such as market data, Internet of Things (IoT), mobile, sensors, clickstream and even transactions remain largely unnavigated. These data sources are growing and the sweet spot for stream computing.

The top use cases for stream computing in 2014:

  • The Internet of Things (IoT): Optimize availability, performance, capacity and resource utilization
  • Enhanced security intelligence: Predict, prevent and act on security threats and real-time fraud detection; increase situational awareness
  • Next best action: Act on up-to-the-second observations, while the event/transaction is still happening
  • Real-time sentiment analysis of social media: Effectively respond to improve the client experience

What’s amazing about these use cases is they go beyond the IT professional (developer, DBA, IT architect) and data scientist—a subject matter expert is required to make them work. To be effective, big data solutions need to provide information in context to the subject matter expert at the right time to generate the appropriate action. Deep context can be delivered in real-time derived using sophisticated techniques such as machine learning, text analysis, geospatial analytics, entity analytics, image recognition and more.  

I predict that 2015 will be the year of the practitioner

KimMadia_Blog.jpgThe next step for the big data industry is to embrace the non-IT person. Data scientists are in demand, but they need insight from practitioners with unique subject knowledge to advance. Stream computing platforms and Hadoop systems are revolutionary and the technology of choice for many in IT, but how do these platforms help the coal miner, the police officer, the tractor operator, the foreman of a high tech wafer manufacturing plant and the small business owner? 

The answer lies with the roll-out of more user-based big data offerings, such as stream computing solutions. Unlocking streaming data sources and making them consumable for each subject matter expert will be a focus for 2015 and beyond. This insight will be delivered in context so users can understand each new data point and observation as it relates to their situation.

Also key for 2015 will be the high availability of flexible models to consume these services. It will be exciting to watch an entire new ecosystem of offerings unfold and grow to support user requirements.

Now is the time to design big data solutions around the users. The marketing is moving beyond pure management of big data and the consumers of big data will have their voice heard.