Big Data is Rocket Fuel

VP Strategy, Silicon Valley Data Science, Editor-in-Chief, Big Data Journal

Larry Page, CEO of Google, believes in “moonshots.” Not just incremental thinking, but breakthrough progress that makes an order of magnitude difference in a field. At his company's developer conference in San Francisco, Page urged others to do the same: “I'd encourage more companies to do things that are a little bit outside of their comfort zone, because I think it gives them more scalability in what they can get done. Almost every time we have tried to do something crazy we have made progress, not all the time, but much of the time.”[i]

rocket-fuel_Page_1.pngAside from Page's belief, what has been Google's secret? Quite simply: data. Google's order-of-magnitude advantage in the search stakes was maintained by algorithms that gave positive returns on scale. The more people used the search engine, the better it got, thanks to deriving quality signals from user behavior—if you used the first result from a search, it's a good result, if you have to use the seventh, then the result can be improved.

So far, so obvious. Yet the connection can also be drawn to Google's more explicitly adventurous innovations, the autonomous car, or the wearable computer Glass. The car could not drive without the masses of data accumulated from the company's maps product, and Glass would be useless without the social data collected by Google Plus.


In each instance, Google's bold moves have been fueled by the vast potential of data. For this reason, as a community we should welcome and endorse moves to open up public data. Not just from a perspective of democratic accountability, but for the potential of growth in research and commerce that this data provides. We saw a commendable step forward in the United States this May, as President Obama signed an executive order focusing on using government open data for entrepreneurship, innovation, and scientific discovery.[ii]

The advent of big data technologies has caused wild excitement for its potential and also the inevitable backlash due to media overstatement. Big data stands on the legs of two important pillars: open source software and commodity computing hardware. These things have lowered the bar for accessing supercomputer-level power and vastly increased data processing throughput.

To focus on these two legs alone, though, misses an important component. Critics of big data observe that most companies don't have big data,[iii] and the IT industry's marketing of big data tools does little to serve them. This is unsurprisingly true. Without data, tools have little to do: In our journey to the moon, it helps if we put fuel in our ship. Hence, the third and equally important leg of the big data platform is the data itself. It is the most valuable asset, being the image of the world in which we are attempting our endeavors. Prizing data is something Google understood from the start.

As the field of big data progresses, this growing maturity will be shown in the focus of enthusiasm moving away from the initial excitement that we can process large data and toward understanding the importance of acquiring, stewarding, and sharing our data.

If, like Larry Page, you're making a moonshot, then having big data as rocket fuel sure helps.

Listen to Edd talk more about Big Data as Rocket Fuel in this podcast.


[i] Copeland MV and Wohslen M. May 16, 2013. With I/O speech, Larry Page reminds us why Google rules tech.

[ii] Howard AB. May 15, 2013. The best thing Obama's done this month.