Thursday, 2 August 2012

Do you want to help with data discovery?

Reposted from the blog of the International Surface Temperature Initiative

As was alluded to in an earlier posting here, NOAA's National Climatic Data Center has recently endeavored on an effort to discover and rescue a plethora of international holdings in hard copy in its basement and make them usable by the international science community. The resulting images of the records from the first chunk of these efforts have just been made available online. Sadly, it is not realistic at the present time to key these data so they remain stuck in a half-way house, available, tantalizingly so, but not yet truly usable.

So, if you want to undertake some climate sleuthing now is your moment to shine ...! The data have all been placed at ftp://ftp.ncdc.noaa.gov/pub/data/globaldatabank/daily/stage0/FDL/ . These consist of images at both daily and monthly resolution - don't be fooled by the daily in the ftp site address. If you find a monthly resolution data source you could digitize years worth of records in an evening.

Whether you wish to start with Angola ...



or Zanzibar ...

There is data for you to discover. As the Readme file says ...

The following are documents recently imaged through an effort at the  National Climatic Data Center (NCDC). NCDC holds over 2000 cubic feet of foreign records in paper format of in-situ weather observations in the on-site physical archives. The data files included are digital photographs of records in this collection. NCDC federal and contract employees captured the images using NCDC-owned digital cameras. All photos are in JPEG format. The images contain in-situ observations from all areas of the globe other than the United States. African observations were the initial geographic area of concentration in the imaging effort but have grown to include other continents and regions. The collection consists of weather observations that were taken almost exclusively between 1885 and 1975. The text within the images are in various languages. A worldwide community of scientists will benefit from this effort, which is part of a global effort to discover, scan and key missing in-situ data. The data from the images will eventually be added to integrated global datasets, including baseline datasets at NCDC. This directory contains the following:
  • 45 tar files containing images of the data
  • An inventory file describing the files that were recently examined at NCDC (the ones highlighted in yellow have been imaged)
  • An example .csv file describing the preferable format of the data if these images were to be digitized

So, if you want to do some discovery and recovery of data the opportunity is now there to do so. Any data submitted to the databank using the submission guidelines detailed here will be shared without restriction for use by anyone for any purpose. We would strongly encourage keying of all recorded meteorological parameters although clearly temperatures are essential.

Regardless of one's viewpoint there cannot be a downside to improving data availability if one wants to be able to make informed analyses and decisions, particularly so when that data has an unbroken chain of provenance back to the raw paper record. So, this really is an opportunity to provide something uniquely useful to scientists and the public around the world and to 'own' a chunk of the global climate record.

Any help gratefully received.
Author: PeterThorne

2 comments:

Marion Delgado said...

How will the recipients check the data?

Victor Venema said...

I have reposted your question at the original blog.

User friendly crowd sourcing initiatives are:
OldWeather for maritime records:
http://www.oldweather.org

Data rescue at home has currently two projects online: German radiosonde data from the Second World War and meteorological station data from Tulagi (Solomon Islands) for the first half of the 20th century:
http://www.data-rescue-at-home.org