Beta version of a new global temperature database released

Today, a first version of the global temperature dataset of the International Surface Temperature Initiative (ISTI) with 39 thousand stations has been released. The aim of the initiative is to provide an open and transparent temperature dataset for climate research.

The database is designed as a climate "sceptic" wet dream: the entire processing of the data will be performed with automatic open software. This includes every processing step from conversion to standard units, to merging stations to longer series, to quality control, homogenisation, gridding and computation of regional and global means. There will thus be no opportunity for evil climate scientists to fudge the data and create an artificially strong temperature trend.

It is planned that in many cases, you can go back to the digital images of the books or cards on which the observer noted down the temperature measurements. This will not be possible for all data. Many records have been keyed directly in the past, without making digital images. Sometimes the original data is lost, for instance in case of Austria, where the original daily observation have been lost in the Second World War and only the monthly means are still available from annual reports.

The ISTS also has a group devoted to data rescue to encourage people to go into the archives, image and key in the observations and upload this information to the database.

Future research in homogenisation of climate data – EMS2012 in Poland

By Enric Aguilar and Victor Venema

The future of research and training in homogenisation of climate data was discussed at the European Meteorological Society in Lodz by 21 experts. Homogenisation of monthly temperature data has improved much in the last years, as seen in the results of the COST-HOME project. On the other hand the homogenization of daily and subdaily data is still in its infancy and this data is used frequently to analyse changes in extreme weather. It is expected that inhomogeneities in the tails of the distribution are stronger than in the means. To make such analyses on extremes more reliable, more work on daily homogenisation is urgently needed. This does not mean than homogenisation at the monthly scale is already optimal, much can still be improved.

Parallel measurements

Parallel measurements with multiple measurement set-ups were seen as an important way to study the nature of inhomogeneities in daily and sub-daily data. It would be good to have a large international database with such measurements. The regional climate centres (RCC) could host such a dataset. Numerous groups are working on this topic, but more collaboration is needed. Also more experiments would be valuable.

When gathering parallel measurements the metadata is very important. INSPIRE (an EU Directive) has a standard format for metadata, which could be used.

It may be difficult to produce an open database with parallel measurements as European national meteorological and hydrological services are often forced to sell their data for profit.(Ironically, in the Land the Free (markets), climate data is available freely, the public already paid for it with their tax money after all.) Political pressure to free climate data is needed. Finland is setting a good example and will free its data in 2013.

The paleo culture

A volunteer of the Ancestral Health Symposium 2012 has criticized the culture of the paleo movement. Richard Nikoley apparently felt attacked and as a prolific blogger immediately wrote a hot tempered post in defence. (In the meantime, the blog with the criticism has been deleted due to the personal attacks and threats.) Richards defensive post focused on the few lines that went over the top.

The demographic at this event was almost all white, child bearing age, healthy, wealthy, highly educated, libertarian, racist, sexist and bigoted.

I presume these lines were more provoked by a life of discrimination as by a single symposium.

It is normal to be defensive while receiving criticism. The day after, one often notices that honest feedback is actually very valuable, that it gives rare and precious insight into how one is seen from the outside. The valuable points of the criticism were (i) that she did not feel welcome, as a not wealthy person and also as an older woman. Furthermore, there were (ii) many crackpots at the symposium.

Demographics

I must admit that I also sometimes find the paleo culture to be rather off putting. The reason I stay is because many good ideas from the paleo community have helped improve my health enormously. The main bloggers are friendly and many focus just on science, which is neutral, but you are often just one click away from the National Rife Association. The community has a strong focus on the health effects of nature, but I never saw a link to a nature conservation group. Paleo is inspired by the life style of hunter-gatherers, but I had to hear about Survival International, an organisation that helps indigenous peoples protect themselves, on the German radio. There is lots of talk about expensive food, supplements and gear, but not about anti-hierarchical strategies used by hunter-gather groups to keep their band egalitarian and strong. Much of the advice is focused on males and it may, for example, well be that the standard routines for intermittent fasting are too heavy for woman.

Statistical homogenisation for dummies

The self-proclaimed climate sceptics keep on spreading fairy tales that homogenisation is smoothing climate data and leads to adjustments of good stations to make them into bad stations. Quite some controversy for such an innocent method to reduce non-climatic influences from the climate record.

In this post, I will explain how homogenisation really works using a simple example with only three stations. Figure 1 shows these three nearby stations. Statistical homogenisation exploits the fact that these three time series are very similar (are highly correlated) as they measure almost the same regional climate. Changes that happen at only one of the stations are assumed to be non-climatic. The aim of homogenisation is to remove such non-climatic changes in the data.

Figure 1. The annual mean temperature data of three hypothetical stations in one climate region.

(In case colleagues of mine are reading this and are wondering about my craftsmanship: I do know who to operate scientific plotting software, but some “sceptics” make fun of people who have no experience with Excel. I just wanted to show off with being able to use a spreadsheet.)

For the example, I have added a break inhomogeneity in the middle with a typical size of 0.8 °C (1.5 °F) to the data for station A; see Figure 2.

Do you want to help with data discovery?

Reposted from the blog of the International Surface Temperature Initiative

As was alluded to in an earlier posting here, NOAA's National Climatic Data Center has recently endeavored on an effort to discover and rescue a plethora of international holdings in hard copy in its basement and make them usable by the international science community. The resulting images of the records from the first chunk of these efforts have just been made available online. Sadly, it is not realistic at the present time to key these data so they remain stuck in a half-way house, available, tantalizingly so, but not yet truly usable.

So, if you want to undertake some climate sleuthing now is your moment to shine ...! The data have all been placed at ftp://ftp.ncdc.noaa.gov/pub/data/globaldatabank/daily/stage0/FDL/ . These consist of images at both daily and monthly resolution - don't be fooled by the daily in the ftp site address. If you find a monthly resolution data source you could digitize years worth of records in an evening.

Whether you wish to start with Angola ...

A short introduction to the time of observation bias and its correction

Figure 1. A thermo-hygrograph, measures and records temperature and humidity.

Due to recent events, the time of observation bias in climatological temperature measurements has become a hot topic. What is it, why is it important, why should we and how can we correct for it? A short introduction.

Mean temperature

The mean daily temperature can be determined in multiple ways. Nowadays, it is easy to measure the temperature frequently, store it in a digital memory and compute the daily average. Also in the past something similar was possible using a thermograph; see Figure 1. However, such an instrument was expensive and fragile.

Thus normally other ways were used for standard measurements, using minimum and maximum thermometers and by computing a weighted average over observations at 3 or 4 fixed times. Another good approximation for many climate regions is to average over the minimum and maximum temperature. Special minimum and maximum thermometers were invented in 1782 for this task.

Blog review of the Watts et al. (2012) manuscript on surface temperature trends

[UPDATE: Skeptical Science has written an extensive review of the Watts et al. manuscript: "As it currently stands, the issues we discuss below appear to entirely compromise the conclusions of the paper." They mention all the important issues, except maybe for the selection bias mentioned below. Thus my fast preliminary review below can now be considered outdated. Have fun.]

Anthony Watts put his blog on hold for two days because he had to work on an urgent project.

Something’s happened. From now until Sunday July 29th, around Noon PST, WUWT will be suspending publishing. At that time, there will be a major announcement that I’m sure will attract a broad global interest due to its controversial and unprecedented nature.

What has happened? Anthony Watts, President of IntelliWeather has co-written a manuscript and a press release! As Mr. Watts is a fan of review by bloggers, here is my first reaction after looking through the figures and the abstract.

Investigation of methods for hydroclimatic data homogenization

The self-proclaimed climate sceptics have found an interesting presentation held at the General meeting of the European Geophysical Union.

In the words of Anthony Watts, the "sceptic" with one of the most read blogs, this abstract is a ”new peer reviewed paper recently presented at the European Geosciences Union meeting.” A bit closer to the truth is that this is a conference contribution by Steirou and Koutsoyiannis, based on a graduation thesis (Greek), which was submitted to the EGU session "Climate, Hydrology and Water Infrastructure". An EGU abstract is typically half a page, it is not possible to do a real review of a scientific study based on such a short text. The purpose of an EGU abstract is in practice to decide who gets a talk and who gets a poster, nothing more, everyone is welcome to come to EGU.