Broadband boosts power of national climate center in Asheville

Last year, NOAA said it was the warmest year on record in the lower 48 states.

In 2012, the average temperature was 55.3°F, which eclipsed 1998, the previous record holder, by 1°F. That difference from 1998 is an unusually large margin since annual temperature records are typically broken by just tenths of a degree.

This report is just one example of the important work happening at NOAA’s National Climatic Data Center (NCDC) in Asheville.

However, since most of the data for this particular announcement is station-based throughout the U.S., this is relatively a small volume of data compared to what the facility ingests on a day-to-day basis with satellites and radar.

And, without high-speed broadband connectivity, none of this would be possible to tackle in a timely fashion.

NCDC maintains the world’s largest climate data archive and provides climatological services and data to every sector of the United States economy and to users worldwide. The center’s mission is to preserve this data and make it available to the public, business, industry, government, and researchers.

NCDC recently initiated a satellite Climate Data Record (CDR) program to continuously provide objective climate information derived from weather satellite data that NOAA has collected for more than 30 years. This data comprises the longest record of global satellite mapping measurements in the world, and is complemented by data from other sources including NASA and U.S. Department of Defense satellites as well as foreign satellites.

For the first time, NOAA is applying modern data analysis methods, which have advanced significantly in the last decade, to these historical global satellite data. This process will unravel the underlying climate trend and variability information and return new economic and scientific value from the records. In parallel, NCDC will maintain and extend these Climate Data Records by applying the same methods to present-day and future satellite measurements.

In fall 2011, NCDC received two 10G broadband connections as part of the build-out through the first phase of the of Golden LEAF Rural Broadband Initiative administered through RTP-based non-profit MCNC. These connections were a significant upgrade from the two 1G connections previously used at NCDC.

Most of the specific uses of these 10G connections are classified, but one use mentioned is for the Suomi National Polar-orbiting Partnership (NPP), which represents a critical first step in building next-generation Earth-observing satellite systems. The NPP is the result of a partnership between NASA, NOAA, and the U.S. Department of Defense.

Since Oct. 2011, 1.3 petabytes of data has streamed into NCDC for consumption and storage from the NPP. In that process, a copy also is made of all data and is sent as a backup to Colorado. This means since this project went active, more than 2 petabytes of information has traveled over broadband-based pipes to researchers all over the world.

“Broadband is absolutely critical to what we do now,” said Alan Hall, IT project manager at NCDC. “If we didn’t have broadband, we wouldn’t be able to move all this data in a timely manner and get it to researchers who need it … it is absolutely critical and broadband is a must to be able to do what we do.”

“As more data comes in and out every day, we need high-speed connections to realize all these data sets,” added Preston Carter, an IT specialist at NCDC who works on network operations and storage. “We have better download rates now and as others get more speeds soon, we will be ready as we continue to future-proof our infrastructure.”

NCDC has seen astronomical spikes in data in recent years with new technology and higher-speed connections. On a typical day, about a terabyte of data comes in to be stored and archived – most in real-time. During major weather events, like Hurricane Sandy last fall, that would jump three-to-six times that amount.

NCDC was incorporated with all civil weather entities as part of NOAA in 1970. Twelve years later, the organization was renamed the National Climatic Data Center and has remained housed at the Veach-Baley Federal Building since 1995.

Today, data comes to NCDC from not only land-based stations but also from ships, buoys, weather balloons, radars, satellites, and even sophisticated weather and climate models. In the past 10 years, NCDC’s digital archive experienced a six-fold increase from 1 petabyte to 6 petabytes. With increasing sophistication of data collection equipment, data is expected to exceed 15 petabytes by 2020.

The United States has made tremendous investments in Earth-observing satellites over the past five decades. Despite remarkable success, great potential remains in the nation’s archived measurements for climate change applications.

NOAA’s new Climate Data Record Project promises to unleash the potential of this data to address critical climate questions. But again, doing this type of work today would not be possible without high-speed, low latency broadband.