Big Data in Motion Where? Everywhere

Real-world examples demonstrate how real-time analysis of data in motion inspires a smarter planet

Executive IT Specialist, Competitive and Product Strategy, IBM Analytics, IBM

Last month, I made the case that the big data discussion has to include both data at rest and data in motion. This time, I want to go over some examples in which the data-in-motion approach is needed.

When we consider a smarter planet, thinking about things such as sensor-equipped buoys in the ocean measuring currents, temperatures, and so on is easy, but let’s bring it closer to home. An intensive care unit (ICU) uses a lot of different machines to keep track of the various aspects of the health of patients. One problem is that each machine is independent from the others and has a narrow range of acceptable values. If a machine gets out of that range, it sounds an alarm.

Some ICUs average 350 alarms per patient per day,* and they found that a vast majority of them do not require an intervention. This scenario leads to what is called alarm fatigue. Still, missing an actionable alarm can be disastrous. Models are being developed that can correlate all the data and determine which alarm is actionable. A quick real-time analysis of the correlated events can lead to a quicker response time than without this analysis, which makes solving the problem less challenging and helps save lives.

If you take the idea a little further, you can see some similarities between running an ICU and running a factory. Doing real-time analysis on the different sensors from all the pieces of equipment could improve preventive maintenance, which would help reduce the overall maintenance cost by reducing or even eliminating unexpected failures.

Machine data can come in at a very high rate, in some cases at the microsecond interval. Even collecting data for the creation of a model can be challenging. A data-in-motion solution could pre-process the data by either aggregating or summarizing it at an appropriate level of granularity to put into the data warehouse that is used to create the model.

Merely pouncing at the right moment

What about sentiment and buzz analysis? There is a lot of information about this subject. Is it a real-time problem? It can be. If there were a buzz in law enforcement that developed around gang activities, for example, intervening after the fact would not be helpful. The problem is a matter of how fast the buzz develops and how quickly someone can identify it and take action.

Then there is monitoring network traffic. What about the usual activities of all the computers in an organization? Keeping track of network access for all end users is one option. Any deviation in normal usage could indicate a machine is infected with a virus. How quickly should an organization want to find out? I’m sure a few large retailers now have definite ideas on this issue.

The IBM® InfoSphere® Streams platform is well suited for addressing these types of problems because it provides a programming platform for distributed real-time analytics. It was designed from the ground up for enhanced performance and scalability. It comes with an integrated development environment that includes a visual editor, wizards for many tasks, and sets of operators grouped into toolkits.

InfoSphere Streams can also be extended using languages such as C++ and Java. Plus, the runtime environment is covered with web- and command-line–based tooling for monitoring and administration. InfoSphere Streams does not live in a vacuum. It is fully integrated within the IBM big data architecture. It integrates nicely with other software such as the IBM InfoSphere BigInsights™ solution, the IBM MessageSight messaging appliance for the Message Queuing Telemetry Transport (MQTT) protocol, and IBM SPSS® predictive analytics. Look for a lot more information about InfoSphere Streams in upcoming articles.

Just scratching the surface

These examples offer just a few areas where data-in-motion analytics helps solve problems. What about your environment? Can you come up with reasons why you would apply data in motion to your business problems? Please let me know in the comments.

*Joint Commission Warns of Alarm Fatigue,” by Mike Mitka, The Journal of the American Medical Association (JAMA), June 2013.

[followbutton username='IBMdatamag' count='false' lang='en' theme='light']