Big Data and the Internet of Everything

A brave new world of possibilities

Given all the discussion on the big data information technology megatrend, it is important to recognize key factors that are driving massive growth in volume, velocity, and variety. One of those trends is the “Internet of Things” or the “Internet of Everything” (as Cisco’s Chief Technology and Strategy officer, Padmasree Warrior, recently named it).

The Internet of Everything describes a world of billions of interconnected devices providing continuous streams of real-time and discrete data. With the advent of IP-6, we now have a globally addressable world that provides an unimaginable amount of information that will continue to power various decisions. In some instances, human will remain in the loop, but as the ongoing debate continues regarding autonomous systems, the stream of interconnected devices portends a new information order.

Within the Internet of Things, an interconnected Sensor Web is growing as well. Consider standard Android- or IOS-enabled smart phones; each one contains an average of 15 sensors to measure everything from gravity fields to acceleration to location. Ponder the millions of video cameras recording virtually everything, everywhere, all the time. Think about the swarms of microsatellites being launched into space while nanobots are developed and released into micro-scale environments to measure and manipulate events.

Machine-to-machine (MtoM) communication protocols are skyrocketing along with the continued growth of sensor-enabled industrial control systems. And it isn’t just the growth of device-deployed sensors that we have to think about—it’s also the accessibility, addressability, and networkability issues related to IP-6. As Cisco’s Warrior states, “We expect to have 50 billion things connecting to the network by 2020, going up from 10 billion now.”

In response to all this growth, companies are working feverishly to imagine, build, and deliver the network that will support hundreds of billions of interconnected devices, each producing data and communicating continuously. What are the implications for the bandwidth necessary to support this connectivity? Where does the operating system live in the future? How does the Sensor Web fit into cloud infrastructures? Does Sensor-as-a-Service (SaaS) fit with Modeling-as-a-Service (MaaS) layered into Infrastructure-as-a-Service (IaaS)?

Back in 2007, I attended a conference hosted by the Defense Advanced Research Project Agency (DARPA). The room was full of people who were thinking about the implications of the billions of devices—some smart, some dumb—that were projected to exist by 2015. The main speaker stepped onstage with a bucket of sand and grabbed a handful, letting it slip through his fingers. In twenty years, he said, it’s possible that each one of these grains of sand will be a machine (either carbon or silicon). They will be powered, communicative, connected, autonomous, quasi-imperceptible, everywhere, and doing things. He stated the key questions in dramatic fashion: How do we manage this beach—and what is it doing for us?

Six years later, the implications are clearer. Advances in transportation, aerospace, defense, medicine, financial systems, telecommunications, energy resource management, and more are driving this future—a smart future with smart data and smart choices. We are creating an information storm of epic proportions that is the quintessence of big data.

The challenge we all face, regardless of specialization, expertise, capability, interest, or technology preference, is to imagine the cyber infrastructure capable of successfully managing, harvesting, and leveraging this Internet of Everything into something useful for our global civilization. As we continue down the big data road, one thing is certain: The system of systems required to support billions of smart, addressable things doing lots of stuff does not yet exist. It is being created organically, one piece at time, and assembled imperfectly. Let’s work together in an interdisciplinary way to achieve the future we want—rather than a future we simply inherit. With great challenges come great opportunities. And clearly, the next ten years will be full of both.

What do you think? Let me know in the comments.