Rage against the machine data? More like race to embrace it

Big Data Product Marketing Manager, IBM

In a connected world where, for everything, “there’s an app for that,” machine data is stealing the spotlight across industries.

  • “By 2015 more than half a billion people worldwide will use medical apps on their smartphones”—Global Mobile Health Trends and Figures  Market Report 2013-2017
  • “The number of cars connected to the Internet worldwide will grow more than six-fold to 152 million in 2020 from 23 million now.”—IHS Automotive.
  • “In 2014, the number of cell phones will be greater than the number of humans, over 7 Billion.”—International Telecommunications Union

With all this machine data generated internally and externally organizations are challenged to access and search large volumes of machine and operational data to combine it with enterprise data for a more comprehensive view of customers, transactions, operations and business as a whole. Many unfortunately are disheartened by their lack of sophisticated search capabilities and the right technology to effectively analyze their machine data, or even understand what data they have.

In an age where decisions are fast and information is vast, it may not seem like a big deal to say, “You know what? We’re making pretty good decisions using our enterprise data. It’s not imperative that we use all this machine data, too.” That’s true, unless you like the unpleasantness and costliness of unplanned outages…or you are not interested in identifying and investigating anomalies…or you have no time for real-time data analysis and visualization.


But for those who want to gain real-time visibility into operations, customer experience, transactions and behavior, machine data analysis is an imperative. It is equally important for those looking to be more proactive and prevent outages, or those wanting to identify or investigate anomalies.

Some people think of machine data with too narrow a focus, like IT reporting on log data, for example. But applications for machine data analysis are much broader than that and span multiple industries.                

  • In telecommunications data from smart devices and social networks can prevent network issues and optimize network capacity. When combined with transactional customer data, machine data contributes to tailored offerings that prevent customer churn.
  • In health sciences, real-time analytics of physical monitoring data from newborns can help provide alerts if complications arise. Combining this data with existing patient records can lead to customized health treatment plans that improve health and patient satisfaction.
  • In the automotive industry, real-time data from vehicles can provide driving assistance and positioning data. Sensors can detect abnormal wear and tear on parts while checking inventory at different auto shops and arrange for repairs.
  • In security, real-time analytics of streaming video and audio data can increase surveillance measures.
  • In the public sector, real-time performance data from public transportation can lead to smarter traffic and optimized transportation routes.

Machine data analysis solutions

There’s no question that machine data is complex, but the solutions don’t have to be. The key is being able to combine machine data with your enterprise data to gain a full view and improve business results and decision-making.

InfoSphere BigInsights, IBM’s Hadoop distribution, comes with a Machine Data Accelerator built right in, which allows you to store and do deep analysis on large, complex machine data. InfoSphere Streams processes machine data in-motion and performs real-time and time series analysis. Using these types of solutions allows you to process your machine data, then correlate it with other enterprise data (customer, product information, and so on) which helps put deeper insight in the hands of operational decision-makers. These decision-makers can visualize data across many systems and get the most informed view with Watson Explorer. Business decisions are now more informed and can happen in a fraction of a second.

Are you facing an abundance of machine data in your industry? How are you managing it? What are the biggest challenges you’re finding? I’d love to hear from you in the comments section.

Related Links: