Monday, 4 November 2013

Weather variability and Data Management Workshop 2013 in San Lorenzo de El Escorial, Spain

This week I will be at the Data Management Workshop (DMW) in San Lorenzo de El Escorial. Three fun filled days about data rescue, homogenization, quality control and data products (database). While it is nice weather outside.

It is organized by EUMETNET, a network of 30 European National Meteorological Services. Thus I will be one of the few from a university, as is typical for homogenization. It is a topic of high interest to the weather services.

Most European experts will be there. The last meeting I was at was great. The program looks good. I am looking forward to it.

My contribution to the workshop will be to present a joint review of what we know about inhomogeneities in daily data. Much of this information stems from parallel measurements, in other words from simultaneous measurements with a modern and a historical set-up. We need to know about non-climatic changes in extremes and weather variability, to be able to assess the climatic changes.

The coming time, I hope to be able to blog about some of the topics of this review. It shows that the homogenization of daily data is a real challenge and that we need much more data from parallel measurements to study the non-climatic changes in the probability distribution of daily datasets. Please find our abstract below.

The slides of the presentation can be downloaded here.

Weather variability

Another topic, I plan to blog about soon is possible changes in the variability of the weather. A recent provocative perspective (editorial article) by Lisa Alexander and Sarah Perkins from the University of New South Wales, Australia, titled Debate heating up over changes in climate viability was just published.

I still have to read the paper, but one quote stands out and was already picked up by the Climate Depot:
There are two camps: one that says that temperature variability is increasing globally [9] and one that says that it isn't [10, 11] (or at least that the jury is still out [6]).
That would actually leave out no one in favour of increases of temperature variability. Because reference [9] is The climate dice paper by Hansen, Sate and Ruedy (PNAS 2012) and they already stated that their paper is not about variability. At the moment the state of the literature seems to suggest that at annual scales, the temperature variability will decrease.

However at daily scales, especially in summer, there are many (European) papers that suggest an increase in variability. This has been studied because of the recent heat waves for which weather variability at inter-seasonal scales is important. As far as I can see these papers are not mentioned in the 2-page perspective.

However, not all paper suggest an increase at daily scales, and I am also personally not yet convinced that climate data is sufficiently reliable to be able to give a robust answer to questions on changes in weather variability. Annual and monthly data is only homogenized with respect to the mean, not the variability. Daily data is often not homogenized, removing most biases will be difficult and we currently know just little about the non-climatic biases. Where the topic of parallel measurements resurfaces again. More later.

Parallel measurements to study inhomogeneities in daily data

Victor Venema (1), Enric Aguilar (2), Renate Auchmann (3), Ingeborg Auer (4), Theo Brandsma (5), Barbara Chimani (4), Alba Gilabert (2), Olivier Mestre (6), Andrea Toreti (7), and Gregor Vertacnik (8)

(1) University of Bonn, Meteorological Institute, Bonn, Germany,
(2) University Rovira i Virgili, Center for Climate Change, C3, Tarragona/Tortosa, Spain,
(3) University of Bern, Institute of Geography, Bern, Switzerland,
(4) Zentralanstalt für Meteorologie und Geodynamik, Austria,
(5) Royal Netherlands Meteorological Institute, The Netherlands,
(6) Météo-France, Direction de la Production, Toulouse, France,
(7) Justus-Liebig Universitaet, Giessen, Germany,
(8) Slovenian Environment Agency, Ljubljana, Slovenia.

Abstract. Daily datasets have become a focus of climate research because they are essential for studying the variability and extremes in weather and climate. However, long observational climate records are usually affected by changes due to nonclimatic factors, resulting in inhomogeneities in the time series. Looking at the known physical causes of these inhomogeneities, one may expect that the tails of the distribution are especially affected. Fortunately, the number of national and regional homogenized daily temperature datasets is increasing. However, inhomogeneities affecting the tails of the distribution are often not taken into account.

In this literature review we investigate the physical causes of inhomogeneities and how they affect the distribution with respect to its mean and its tails. We review what is known about changes in the distribution from existing historical parallel measurements. We discuss the state of the art in the homogenization methods for the temperature distribution. Finally, we provide an overview of the quality of available daily datasets that are often used for studies on changes in extremes and additionally describe well-homogenized regional datasets.

As expected, this review shows that the tails of the distribution are more affected by non-climatic changes than the means. How much daily homogenization methods can reduce the inhomogeneity of index time series is insufficiently studied. It is advised to study whether the current deterministic correction methods, should be succeeded by stochastic methods. Many often-used large-scale daily datasets are not homogenized (with respect to the distribution), whereas the number of national and regional homogenized is strongly growing.

Given the strong interest in studying changes in weather variability and extremes and the existence of often large inhomogeneities in the raw data, the homogenization of daily data and the development of better methods should have a high research priority.

This research would be much facilitated by a global reference database with parallel measurements. The climate community, and especially those involved in homogenization, bias correction and the evaluation of uncertainties, should take an active role to foster the compilation of such a reference database. We have started an initiative collecting parallel datasets. Its aims will be explained and its progress will be presented.


  1. Victor, I was just about to ask you to explain what is meant by homogenisation in a "science for dummies" kind of way but I thought I should check your blog for an explanation first. And lo and behold, you have exactly what I was looking for -

    Enjoy the workshop.

  2. Good you found it. :-)

    That is the post that explains the principle behind the homogenization algorithms. A bit more on why this is needed can be found here.

    It is a real challenge to write posts that are interesting for the general public and my colleagues. This was more one for the colleagues, with little explanation. Maybe I should mark them, but it is also difficult to estimate how difficult a certain post is, which knowledge the reader already has.

  3. Keep the good work Dr. Venema, I am only starting to follow your blog, had naively done some homogenization and only found how complicated and involve it can be, from a small state in the middle of Mexico

  4. Abraham deAlba-Avila, thank you very much for your kind word. You can find all posts about homogenization here.

    And if you want to do something back, ;-) please have a look here. We have much too little studies using parallel measurements on the influence of inhomogeneities on the distribution of daily data, especially outside of Europe.


Comments are welcome, but comments without arguments may be deleted. Please try to remain on topic. (See also moderation page.)

I read every comment before publishing it. Spam comments are useless.

This comment box can be stretched for more space.