Debating the hot topics in weather data analytics

Big Data Evangelist, IBM

Everybody on this planet needs to adjust to the vicissitudes of weather and climate. Whether that adjustment is the mundane matter of ensuring you’re dressed appropriately to venture outside today or a looming threat such as global warming, we need to predict and mitigate these risks as best we can.

People talk about the weather, and, in recent years, they’ve been discussing long-term climate change. But climate change is much more than a contentious political debate and a free-floating sense of global anxiety around the collapse of the ice caps and inundation of coastlines. The subject is about people taking action, and people are taking action.

The world community is pursuing diverse initiatives to reduce carbon emissions and take other measures that, hopefully, will be able to reverse, slow or at least cushion the impacts of this trend on humanity in the 21st century and beyond. At the heart of this global push is a renewed worldwide focus on the need for advances in the atmospheric sciences to deliver practical insights necessary to address this problem.

Empowering data science

Data scientists are pivotal players in these efforts. Working with top researchers and using the most advanced high-performance computing assets, data scientists are using vast troves of weather data and building extraordinarily complex predictive models. And they are tapping into real-time sensor feeds from the Internet of Things to find the causal factors driving global warming, El Niño, tropical cyclones and other impactful atmospheric events. the need to empower the next generation of data scientists to tackle these challenges, IBM recently announced it was acquiring digital assets from The Weather Company’s digital asset repository. IBM is incorporating these assets into on-demand analytics being brought to market by its new Insight Cloud Services group.

With The Weather Company acquisition, IBM has gained a high-volume cloud platform that ingests, processes, analyzes and distributes enormous sets of sensor data at scale in real time. The Weather Company’s sophisticated models analyze data from 3 billion weather forecast reference points, more than 40 million smartphones and 50,000 airplane flights per day. This immense network of Internet of Things data resources enables IBM to offer a broad range of data-driven products and services to more than 5,000 clients in the aviation, energy, insurance, government and media industries.

Contributing supersmart innovation

Clearly, basic science is just one of many uses for sophisticated weather data analytics. In that respect, IBM kicked off another pivotal initiative around the same time it announced The Weather Company acquisition: the Sparkathon: It’s Raining Data online hack. The challenge, which is still ongoing and will take submissions until February 3, 2016, offers cash prizes for data scientists who can build supersmart weather apps that harness the power of the IBM Bluemix platform and IBM Analytics for Apache Spark. The contest is designed to encourage innovative approaches to using big data, machine learning, graph computation and stream processing to discover significant patterns in weather data sets. Just as important, it is intended to drive that intelligence in practical applications of all sorts.

If you want a sense for the creativity that comes from giving data scientists a challenge such as the Sparkathon, check out Monica Fox’s discussion of the semifinalists from the Hack the Weather event that IBM held in fall 2015, or see my blog posting that covered the finals.

Be sure to submit your supersmart weather apps in the Sparkathon: It’s Raining Data challenge. Registration takes place through February 3, 2016, and the competition offers $30,000 in prizes. And be sure to visit the onramp to IBM Insight Cloud Services to learn more about how Spark, Bluemix data tools and weather data can empower your smart data scientists to tackle these global challenges.