The WMO Task Team on Homogenisation (TT-HOM) is working on a guidance for scientists and weather services who want to homogenise their data. I thought the draft chapter on the history of homogenisation doubles as a nice blog post. It is a pretty long history, starting well before people were worrying about climate change. Comments and other important historical references are very much appreciated.
Although this “Conference of directors” of the national weather services recommended maintaining a sufficient number of stations under unchanged conditions today these basic inhomogeneity problems still exist.
Detection and adjustments
Homogenisation has a long tradition. For example, in early times documented change points have been removed with the help of parallel measurements. Differing observing times at the astronomical observatory of the k.k. University of in Vienna (Austria) have been adjusted by using multi-annual 24 hour measurements of the astronomical observatory of the k.k. University of Prague (today Czech Republic). Measurements of Milano (Italy) between 1763 and 1834 have been adjusted to 24 hour means by using measurements of Padova (Kreil, 1854a, 1854b).However, for the majority of breaks we do not know the break magnitude; furthermore it is most likely that series contain undocumented inhomogeneities as well.Thus there was a need for statistical break detection methods. In the early 20th century Conrad (1925) applied and evaluated the Heidke criterion (Heidke, 1923) using ratios of two precipitation series. As a consequence he recommended the use of additional criteria to test the homogeneity of series, dealing with the succession and alternation of algebraic signs, the Helmert criterion (Helmert, 1907) and the “painstaking” Abbe criterion (Conrad and Schreier, 1927). The use of Helmert’s criterion for pairs of stations and Abbe’s criterion still has been described as appropriate tool in the 1940s (Conrad 1944). Some years later the double-mass principle was popularised for break detection (Kohler, 1949).
German Climate Reference Station which was founded in 1781 in Bavaria on the mountain Hohenpeißenberg.
Reference series
Julius Hann (1880, p. 57) studied the variability of absolute precipitation amounts and ratios between stations. He used these ratios for the quality control. This inspired Brückner (1890) to check precipitation data for inhomogeneities by comparison with neighbouring stations;he did not use any statistics.In their book “Methods in Climatology” Conrad and Pollak (1950) formalised this relative homogenization approach, which is now the dominant method to detect and remove the effects of artificial changes. The building of reference series, by averaging the data from many stations in a relatively small geographical area, has been recommended by the WMO Working Group on Climatic Fluctuations (WMO, 1966).
The papers by Alexandersson (1986) and Alexandersson and Moberg (1997) made the Standard Normal Homogeneity Test (SNHT) popular. The broad adoption of SNHT was also for the clear guidance on how to use this test together with references to homogenize station data.
Modern developments
SNHT is a single-breakpoint method, but climate series typically contain more than one break. Thus a major step forward was the design of methods specifically designed to detect and correct multiple change-points and work with inhomogeneous references (Szentimrey, 1999; Mestre, 1999; Caussinus and Mestre, 2004). These kind of methods were shown to be more accurate by the benchmarking study of the EU COST Action HOME (Venema et al., 2012).The paper by Caussinus and Mestre (2004) also provided the first description of a method that jointly corrects all series of a network simultaneously. This joint correction method was able to improve the accuracy of all but one contribution to the HOME benchmark that was not yet using this approach (Domonkos et al., 2013).
The ongoing work to create appropriate datasets for climate variability and change studies promoted the continual development of better methods for change point detection and correction. To follow this process the Hungarian Meteorological Service started a series of “Seminars for Homogenization” in 1996 (HMS 1996, WMO 1999, OMSZ 2001, WMO 2004, WMO 2006, WMO 2010).
Related reading
- Homogenization of monthly and annual data from surface stations
- A short description of the causes of inhomogeneities in climate data (non-climatic variability) and how to remove it using the relative homogenization approach.
- Statistical homogenisation for dummies
- A primer on statistical homogenisation with many pictures.
- Just the facts, homogenization adjustments reduce global warming
- Many people only know that climatologists increase the land surface temperature trend, but do not know that they also reduce the ocean surface trend and that the net effect is a reduction of global warming. This does not fit to well to the conspiracy theories of the mitigation sceptics.
- Five statistically interesting problems in homogenization
- Series written for statisticians and climatologists looking for interesting problems.
- Why raw temperatures show too little global warming
- The raw land surface temperature probably shows too little warming. This post explains the reasons why: thermometer screen changes, relocations and irrigation.
- New article: Benchmarking homogenization algorithms for monthly data
- Raw climate records contain changes due to non-climatic factors, such as relocations of stations or changes in instrumentation. This post introduces an article that tested how well such non-climatic factors can be removed.
References
Brückner, E., 1890: Klimaschwankungen seit 1700 nebst Bemerkungen über Klimaschwankungen der Diluvialzeit. E.D. Hölzel, Wien and Olnütz.
Alexandersson, A., 1986: A homogeneity test applied to precipitation data. J. Climatol., 6, pp. 661-675.
Alexandersson, H. and A. Moberg, 1997: Homogenization of Swedish temperature data .1. Homogeneity test for linear trends. Int. J. Climatol., 17, pp. 25-34.
Caussinus, H. and O. Mestre, 2004: Detection and correction of artificial shifts in climate series. Appl. Statist., 53, Part 3, pp. 405-425.
Conrad, V. and C. Pollak, 1950: Methods in Climatology. Harvard University Press, Cambridge, MA, 459 p.
Conrad V., O. Schreier, 1927: Die Anwendung des Abbe’schen Kriteriums auf physikalische Beobachtungsreihen. Gerland’s Beiträge zur Geophysik, XVII, 372.
Conrad, V., 1925: Homogenitätsbestimmung meteorologischer Beobachtungsreihen. Meteorologische Zeitschrift, 482–485.
Conrad V., 1944: Methods in Climatology. Harvard University Press, 228 p.
Domonkos, P., V. Venema, O. Mestre, 2013: Efficiencies of homogenisation methods: our present knowledge and its limitation. Proceedings of the Seventh seminar for homogenization and quality control in climatological databases, Budapest, Hungary, 24 – 28 October 2011, WMO report, Climate data and monitoring, WCDMP-No. 78, pp. 11-24.
Hann, J., 1880: Untersuchungen über die Regenverhältnisse von Österreich-Ungarn. II. Veränderlichkeit der Monats- und Jahresmengen. S.-B. Akad. Wiss. Wien.
Heidke P., 1923: Quantitative Begriffsbestimmung homogener Temperatur- und Niederschlagsreihen. Meteorologische Zeitschrift, 114-115.
Helmert F.R., 1907: Die Ausgleichrechnung nach der Methode der kleinsten Quadrate. 2. Auflage, Teubner Verlag.
Peterson T.C., D.R. Easterling, T.R. Karl, P. Groisman, N. Nicholls, N. Plummer, S. Torok, I. Auer, R. Boehm, D. Gullett, L. Vincent, R. Heino, H. Tuomenvirta, O. Mestre, T. Szentimrey, J. Salinger, E.J. Forland, I. Hanssen-Bauer, H. Alexandersson, P. Jones, D. Parker, 1998: Homogeneity adjustments of in situ atmospheric climate data: A review. Int. J. Climatol., 18, 1493-1517.
Hungarian Meteorological Service (HMS), 1996: Proceedings of the First Seminar for Homogenization of Surface Climatological Data, Budapest, Hungary, 6-12 October 1996, 44 p.
Kohler M.A., 1949: Double-mass analysis for testing the consistency of records and for making adjustments. Bull. Amer. Meteorol. Soc., 30: 188 – 189.
k.k. Hof- und Staatsdruckerei, 1873: Bericht über die Verhandlungen des internationalen Meteorologen-Congresses zu Wien, 2.-10. September 1873, Protokolle und Beilagen.
k.k. Zentralanstalt für Meterologie und Geodynamik. 1906: Bericht über die internationale meteorologische Direktorenkonferenz in Innsbruck, September 1905. Anhang zum Jahrbuch 1905. k.k. Hof-und Staatsdruckerei.
Kreil K., 1854a: Mehrjährige Beobachtungen in Wien vom Jahre 1775 bis 1850. Jahrbücher der k.k. Central-Anstalt für Meteorologie und Erdmagnetismus. I. Band – Jg 1848 und 1849, 35-74.
Kreil K., 1854b: Mehrjährige Beobachtungen in Mailand vom Jahre 1763 bis 1850. Jahrbücher der k.k. Central-Anstalt für Meteorologie und Erdmagnetismus. I. Band – Jg 1848 und 1849, 75-114.
Mestre O., 1999: Step-by-step procedures for choosing a model with change-points. In Proceedings of the second seminar for homogenisation of surface climatological data, Budapest, Hungary, WCDMP-No.41, WMO-TD No.962, 15-26.
OMSZ, 2001: Third Seminar for Homogenization and Quality Control in climatological Databases, Budapest.
Szentimrey, T., 1999: Multiple Analysis of Series for Homogenization (MASH). Proceedings of the second seminar for homogenization of surface climatological data, Budapest, Hungary; WMO, WCDMP-No. 41, 27-46.
Venema, V., O. Mestre, E. Aguilar, I. Auer, J.A. Guijarro, P. Domonkos, G. Vertacnik, T. Szentimrey, P. Stepanek, P. Zahradnicek, J. Viarre, G. Müller-Westermeier, M. Lakatos, C.N. Williams,
M.J. Menne, R. Lindau, D. Rasol, E. Rustemeier, K. Kolokythas, T. Marinova, L. Andresen, F. Acquaotta, S. Fratianni, S. Cheval, M. Klancar, M. Brunetti, Ch. Gruber, M. Prohom Duran, T. Likso,
P. Esteban, Th. Brandsma. Benchmarking homogenization algorithms for monthly data. Climate of the Past, 8, pp. 89-115, doi: 10.5194/cp-8-89-2012, 2012. See also the introductory blog post and a post on the weaknesses of the study.
M.J. Menne, R. Lindau, D. Rasol, E. Rustemeier, K. Kolokythas, T. Marinova, L. Andresen, F. Acquaotta, S. Fratianni, S. Cheval, M. Klancar, M. Brunetti, Ch. Gruber, M. Prohom Duran, T. Likso,
P. Esteban, Th. Brandsma. Benchmarking homogenization algorithms for monthly data. Climate of the Past, 8, pp. 89-115, doi: 10.5194/cp-8-89-2012, 2012. See also the introductory blog post and a post on the weaknesses of the study.
WMO, 1966: Climatic Change, Report of a working group of the Commission for Climatology. Technical Note 79, WMO – No. 195. TP.100, 79 p.
WMO 1999: Proceedings of the Second Seminar for Homogenization of Surface Climatological Data, Budapest, Hungary, 9 – 13 November 1998, 214 p.
WMO, 2004: Fourth Seminar for Homogenization and Quality Control in Climatological Databases, Budapest, Hungary, 6-10 October 2003, WCDMP-No 56, WMO-TD No. 1236, 243 p.
WMO, 2006: Proceedings of the Fifth Seminar for Homogenization and Quality Control in Climatological Databases, Budapest, Hungary, 29 May – 2 June 2006. Climate Data and Monitoring WCDMP- No 71, WMO/TD- No. 1493.
WMO, 2010: Proceedings of the Meeting of COST-ES0601 (HOME) Action, Management Committee and Working groups and Sixth Seminar for Homogenization and Quality Control in Climatological Databases, Budapest, Hungary, 26 – 30 May 2008, WMO reports on Climate Data and Monitoring, WCDMP-No. 76.
In your 'References' did you forget Benchmarking homogenization algorithms for monthly data, Venema et al 2012?
ReplyDeleteYes, thanks, now added.
ReplyDeleteVictor
ReplyDeleteDr julius hann is one of my favourite authors. He was very disparaging about methodology and the habit of using inconsistent readings of temperatures by using hourly readings taken somewhat at random
In case your readers have not seen the English version of perhaps his most interesting book, here is a link.
https://ia600205.us.archive.org/16/items/cu31924002957219/cu31924002957219.pdf
It is well worth a look as many of the issues he raises would be recognised today
Tonyb
Thanks for the link. Was that a comment by Hann on data quality or about homogenisation? In the latter case we should mention it. Do you know the paragraph without having to search?
ReplyDeleteOne of the main reasons why a change in observer often causes an inhomogeneity is that they have different tendencies to observe too early or too late. (The other, for amateur networks, is that a change in observer is often a sign of a change in location.)
Dr Hann made a variety of pertinent comments on a whole range of temperature related data.
ReplyDeleteI particularly enjoyed these ones but there are very many other observations on other weather related topics
'This material is taken from Chapter 6 which describes how mean daily temperatures are taken;
“If the mean is derived from frequent observations made during the daytime only, as is still often the case, the resulting mean is too high…a station whose mean is obtained in this way seems much warmer with reference to other stations than it really is and erroneous conclusions are therefore drawn on its climate, thus (for example) the mean annual temperature of Rome was given as 16.4c by a seemingly trustworthy Italian authority, while it is really 15.5c.”
That readings should be routinely taken in this manner as late as the 1900’s, even in major European centers, is somewhat surprising.
'There are numerous veiled criticisms in this vein;
“…the means derived from the daily extremes (max and min readings) also give values which are somewhat too high, the difference being about 0.4c in the majority of climates throughout the year.”
Other complaints made by Doctor von Hann include this comment, concerning the manner in which temperatures are observed;
“…the combination of (readings at) 8am, 2pm, and 8pm, which has unfortunately become quite generally adopted, is not satisfactory because the mean of 8+2+ 8 divided by 3 is much too high in summer.”
And; “…observation hours which do not vary are always much to be preferred.”
That the British- and presumably those countries influenced by them- had habits of which he did not approve, demonstrate the inconsistency of methodology between countries, cultures and amateurs/professionals.'
Tonyb
Now it is easy to measure round the clock with automatic equipment and get a decent estimate of the absolute temperature. In the past that was much harder. Making a measurement by hand every hour is hard. A mechanical thermograph was expensive and fragile.
ReplyDelete