Big Weather, Big Data

Client Technical Manager, Analytics Solution Center, IBM

Derechos, Droughts, Hottest July on Record, Shattered High Temp Records, Greenland Ice Sheet Melts

Just what is going on with the weather these days?  Is this weather really abnormal or does it just seem to be that way? Is this part of a trend?  Does global climate change mean we’ll have more of these extreme weather events?  Being a data and analytics person, I started looking to see what data analysis had been done on this subject.  

The US Climate Extremes Index[i][i] provides a measure to track the occurrence of extreme data (although it doesn’t take into account Derechos and other severe wind events).  The trend of the index (smoothed) has been on the rise since 1970 and now is at an all time high, as shown below. The Index was at a record high 46% during the January-July period, over twice the average value, and surpassing the previous record large CEI of 42% percent which occurred in 1934.  Extremes in warm daytime temperatures (83 percent) and warm nighttime temperatures (74 percent) both covered record large areas of the nation, contributing to the record high year-to-date USCEI value.

This index is compiled by combining measurements throughout the country (1,218-station US Historical Climatology Network) that show the percentage of the country impacted by extreme weather in terms of maximum temperatures much above or below normal, minimum temperatures above/below normal, percentage of country in severe drought/severe moisture surplus, percentage of the country with a much greater than normal proportion of precipitation derived from extreme 1 day events, and the percentage of the country with a much greater than normal number of days with precipitation/without precipitation.

The U.S. Global Change Research Program in 2009 published a study which documented the changing climate and its impact on the United States[ii][ii].   The study uses 3 standard forms of data analysis: 1) reports on observations, 2) predictions based on the observed trends, and 3) modeling to better predict future climate changes based on various assumptions about the amount of heat-trapping gases in the atmosphere.  While the first two types are based on large quantities of collected data, they use only U.S. observations.  The modeling, however, must be done on a global basis which substantially increases the amount of data that must be crunched.

Here are some of the findings as they relate to extreme weather:

Overall Warming of the Climate

Temperatures, on average, in the1993-2008 period are 1-2ºF higher than in the 1961-79 baseline.  By the end of the century, the average U.S. temperature is projected to increase by approximately 7-11ºF under a high emissions model and by approximately 4-6.5ºF under a lower emissions scenario.   The temperature observations show that there has been an increase in warmer and more frequent warm days and warm nights, and warmer and less frequent cold days and cold nights in most areas.

More intense, more frequent, and longer-lasting heat waves

In the past several decades, there has been an increasing trend in high-humidity heat waves, characterized by extremely high nighttime temperatures.  Parts of the South that currently have about 60 days per year with temperatures over 90ºF are projected to experience 150 or more days a year above 90ºF under a higher emissions scenario.  In addition to occurring more frequently, at the end of this century these very hot days are projected to be about 10ºF hotter than they are today.

Increased extremes of summer dryness and winter wetness with a generally greater risk of droughts and floods.

Trends in drought have strong regional variations.  Over the past 50 years, with increasing temperatures, the frequency of drought in many parts of the West and Southeast has increased significantly.  Models show that the Southwest, in particular, is expected to experience increasing drought as the dry zone just outside of the tropics expands northward with global warming.

Precipitation coming in heavier downpours, with longer dry periods in between

While average precipitation over the nation as a whole increased by about 7% over the past century, the amount of precipitation falling in the heaviest 1% of rain events increased nearly 20%.  One of the outputs of the climate modeling is to project the probability of certain events.  For example, heavy downpours that are now a “1 in 20 year occurrence” are projected to occur about “once every 4-15 years” by the end of the century.   These heavy downpours are expected to be 10-25% heavier by the end of the century than they are now.   This will likely cause more flooding events (flooding depends both upon the weather and the susceptibility of the area to being flooded).

More intense but fewer severe storms

Reports of severe weather such as tornadoes and severe thunderstorms have increased during the past 50 years. However the climate study indicates that much of this may be due to better monitoring technologies, changes in population areas, and increasing public awareness. Climate models do project an increase in the frequency of environmental conditions favorable to severe thunderstorms.  But the report notes, “the inability to adequately model the small-scale conditions involved in thunderstorm development remains a limiting factor in projecting the future character of severe thunderstorms and other small-scale weather phenomena.[iii][iii]”  Advances in modeling and big data analytics, as well as improved monitoring networks are likely to reduce this limitation in the future.

The June Derecho that hit the Washington metropolitan area shows an example of the current state of the art in forecasting a severe storm.  The Storm Prediction Center of NOAA was able to provide approximately 4 hours advance warning of the storm.  Longer term predictions would require additional data about the atmospheric instability that propelled the Derecho from Iowa to the Washington Metro area, as well as better real time modeling.

Shift of storm tracks towards the poles

Cold season storm tracks are shifting northward over the last 50 years, with a decrease in the frequency of storms in mid-latitude areas.  The northward shift is projected to continue, and strong cold season storms are likely to become stronger and more frequent, with greater wind speeds and more extreme wave heights.

The climate changes will have an interesting effect on the so called “lake-effect”.  Over the past 50 years, there is a record of increased lake-effect snowfall near the Great Lakes.   As the climate has warmed there is less ice on the Great Lakes which has allowed greater evaporation from the surface resulting in heavier snowstorms.   Eventually, the temperatures are expected to rise sufficiently that much of the precipitation will end up falling as rain, reducing the snow totals. 

Mutual Interactions

While trending of individual elements such as temperatures is useful, accurate predictions require consideration of the interaction between the climate elements.   For example, there is mutual enhancement effect between droughts and heat waves.  Heat waves enhance soil drying, and drier soil heats the air above more since no energy goes into evaporating the soil moisture.  Big data modeling can show the results of this escalating cycle of warming on the future climate.

The New Normal

So it seems that all this abnormal weather we are seeing will become the new normal.  Forewarned is forearmed!


[i][ii] Global Climate Change Impacts in the United States, Thomas R. Karl, Jerry M. Melillo, and Thomas C. Peterson, (eds.) Cambridge University Press, 2009

[i][iii] Ibid, page 38