Thursday 12 February 2015

Just the facts, homogenization adjustments reduce global warming

Climatologists make adjustments to climate data to remove non-climatic changes (homogenization). This fact is used to accuse them of fiddling with temperature data to create or exaggerate global warming. This is often done by showing for a small piece of the data and suggesting it is typical. Often mentioned is the USA, where the raw data only show half the warming of the adjusted data. However, the USA is big, but still only 2% of the Earth's surface.

In recent weeks we had a similar case in The Telegraph about Paraguay. Last year we had similar misleading stories about two stations in Australia and the stations in New Zealand.

Global temperature collections contain thousands of stations. CRUTEM contains 4,842 quality stations and Berkeley Earth collected 39,000 unique stations. No wonder some are strongly adjusted up, just as some happen to be strongly adjusted down. In fact it would be easy to present a station where the raw data shows a cooling trend of several degrees being adjusted to a warming trend. However, then the reader might start to think if the raw data is really better.

The information on small regions or a few stations is normally not put into perspective: the average trend over all stations is only adjusted upwards slightly.

It is normally not explained why these adjustments are made nor how these adjustments are made.

Zeke Hausfather, an independent researcher that is working with Berkeley Earth, made a beautiful series of plots to show the size of the adjustments.

The first plot is for the land surface temperature from climate stations. The data is from the Global Historical Climate Dataset (GHCNv3) of NOAA (USA). Their method to remove non-climatic effects (homogenization) is well validated and recommended by the homogenization community.

They adjust the trend upwards. In the raw data the trend is 0.6°C per century since 1880 while after removal of non-climatic effects it becomes 0.8°C per century. See the graph below. But it is far from changing a cooling trend into strong warming. (A small part of the GHCNv3 raw data was already homogenized before they received it, but this will not change the story much.)

Not many people know, however, that the sea surface temperature trend is adjusted downward. These downward adjustments happen to be about the same size, but go into the other direction. See below the sea surface temperature of the Hadley Centre (HadSST3) of the UK MetOffice.

Being land creatures people do not always realise how big the ocean is, but 71% of the Earth is ocean. Thus if you combine these two temperature signals taking the area of the land and the ocean into account you get the result below. The net effect of the adjustments is a reduction of global warming.

It is pure coincidence that this happens, the reasons for the adjustments are fully different.

The land surface temperature trend has to be adjusted up because old temperatures were often too high due to insufficient protection against warming by the sun, possibly because the siting of the stations improved and there are likely more reasons.

The old sea surface temperature are adjusted downward because old measurements were made by taking a bucket of water out of the ocean and the water cooled by evaporation during the measurement. Furthermore, modern measurements are made at the water inlet of the engine and the hull of the ship warms the water a little before it is measured.

But while it is a pure coincidence and while other datasets may show somewhat different numbers (the BEST adjustments are smaller), the downward adjustment does clearly show that climatologists do not have an agenda to exaggerate global warming. That would still be true if the adjustments had happened to go upward.

Related reading

If you need a peer reviewed reference, the influence of the adjustments on the global mean temperature is also shown in Karl et al. (2015).

Phil Plait at Bad Astronomy comment on the Telegraph piece: No, Adjusting Temperature Measurements Is Not a Scandal

Kevin Cowtan made two videos on the claim of the Telegraph on Paraguay and the Arctic. The second video shows how to check such claims yourself.

John Timmer at Ars Technica is also fed up with being served the same story about some upward adjusted stations every year: Temperature data is not “the biggest scientific scandal ever” Do we have to go through this every year?

The astronomer behind And Then There's Physics writes why the removal of non-climatic effects makes sense. In the comments he talks about adjustments made to astronomical data. Probably every numerical observational discipline of science performs data processing to improve the accuracy of their analysis.

Steven Mosher, a climate "sceptic" who has studied the temperature record in detail and is no longer sceptical about that reminds of all the adjustments demanded by the "sceptics".

Nick Stokes, an Australian scientist, has a beautiful post that explains the small adjustments to the land surface temperature in more detail.

My two most recent posts were about some reasons for temperature trend biases: Temperature bias from the village heat island and Changes in screen design leading to temperature trend biases

You may also be interested in the posts on how homogenization methods work (Statistical homogenisation for dummies) and how they are validated (New article: Benchmarking homogenisation algorithms for monthly data)


  1. And the dog ate your homework.

    Anyone can come up with numerous reasons to explain something. But most of us known when something very fishy is going on and will dismiss claims such as "the dog ate my homework" without any direct knowledge of the dog or what actually happened to the homework.

    And to be quite frank, the people I most expect to fabricate warming ... by some miracle have the highest warming. So, e.g. those headed up by environmental activists show the highest warming. Those headed by "I was a sceptic for a few days" ... show moderate warming and the Satellite which isn't as far as I know in any environmental campaign, shows the least warming.

    So, it is patently obvious what is going on and now we've seen so many well substantiated incidents of data manipulation that unless or until you hand over the raw data to the public or at least some body that if anything is sceptical about warming, then no one will ever believe these non-science records again.

    It's time to put up the data or shut up because all this moaning from people like you that we should believe you is getting tiresome.

  2. Unknown, I was already afraid that even climatologists reducing global warming would not convince the hardcore conspiracy theorists. It is a closed world view, where everything in the world happens for a reason and that reason is that the rest of the world is against you. I have some hope for more moderate people.

    Are you sure that the correlation you perceive between warming and activism is not the other way around? The more warming a measurement shows, the more you feel the group producing that dataset is activist? The more that group is attacked on WUWT & Co.?

    And the differences in temperature trend are very small between the groups. Scientifically interesting, but hardly relevant for the public debate. Even the satellite data of the tropospheric temperature has almost the same trend as the land surface data. You notice the trend less because the tropospheric trend is more influenced by El-Nino/ENSO. But just look at the numbers for the trend.

    The raw data is available. Just go to GHCN or the ISTI and you will find the raw data these temperature curves are computed from.

    Some countries do not give out a part of their data or do not give out raw data. Climatologists also find that a pity. That is the doing of the governments, climate data has commercial and strategic value. Please join the climatologists in calling on your government to release their data.

  3. Maybe I should add for the reader that Unknown did not respond to the content of the above post. That is typical for mitigation sceptics, when they lose an argument, they move to the next without acknowledging that they were wrong. Probably so that they can use the same hook again at another place, hoping that there they do not meet someone who knows why it is wrong. It is one of the reasons why it is clear that mitigation sceptics are not interested in science, but in stealth politics.

  4. "Furthermore, modern measurements are made at the water inlet of the engine and the hull of the ship warms the water a little before it is measured."

    Really, "the hull of the ship warms the water"? At night, the hull of the ship is probably cooler than the water, and what when the ship is drawing water from the shady side of the ship?

    But the big oversight here is that the ship is draawing water not at the surface but at depth, usually at some 5 to 9 metres below the surface (it can be more than 16 m below the surface0 depending upon the design and configuration of the ship.

    The fact that the water is drawn at depth means that it is cooler than the surface, and this fact more than offsets the causes that you mention.

    The adjustments made to the modern ship's data are unsound, and are probably made in the wrong direction.

  5. So you think that global warming has progressed much farther than currently thought? That is interesting. Do you have any literature about measurements confirming that? I would be very grateful for that. With ships getting bigger, the bias due to sampling deeper water may have gradually increased.

    I think the traditional reasoning is that the water inlet is close to the engine, which heats the inlet and the hull, even if the thermometer is installed close to the entrance.

  6. Dear anonymous,

    Victor pointed your comment out to me.

    There is an interesting puzzle here. Yes, ship intakes are pulling water in from a greater depth, which you'd expect to be colder. However, on average (ie not always), intake measurements are warmer than measurements made by drifting buoys at the surface by about 0.15-0.2C on average.

    The depth of measurement is not overlooked. It's simply that, if you look at the data, the bias in ship measurements happens to go the other way.

    If you adjust the blended ship and buoy data as we do in HadSST3 the result is consistent (globally at least) with the best available satellite record.

    Various hypotheses have been suggested to explain the warm-ship paradox, but none is a clear winner. e.g. it has been suggested that somehow the intake samples surface water displaced downwards by the hull, or friction heats the water, or pumps heat the water, or flow in the pipes around the thermometer is somehow blocked, or that there's some kind of thermal contact between the thermometer and the engine room, or...

    Best regards,

    John Kennedy

  7. Victor, you and your readers are invited to visit my new blog on reconstructing climate history from instrumental temperature data:

    I'm starting with Australia before 1910, a period that (for reasons unknown) the Australian BoM have not yet dealt with.

  8. Anonymous, you may be interested in the PhD thesis of Linden Ashcroft. She also worked on the temperature record of Australia before 1910.

    The reasons why BOM did not publish homogenized data before 1910 is well explained in their publications. Strange you do not know this if you are serious about your reconstruction. Before this time the measurements in Australia were of very bad quality, measurement were made inside of telegraph offices or on their veranda. There are reports of thermometers inside near heat sources and of screens that are hardly painted leading to radiation errors.

    Thus there is expected to be a very large warm bias in the earlier measurements. An error that is hard to remove by statistical homogenization because the network is very sparse and you need well correlated measurements from neighbouring stations to remove non-climatic changes well.

  9. Victor, thank you for your comment about the BoM, I am familiar with the work of Ashcroft et al. What they found (and I am confirming) is that for monthly averages and longer much of the late 19th century data is not a scary place at all, you just need to apply corrections in the same way as for any other change. Some places suffered from human organisation (i.e. several stations made changes at very similar times), but there is enough nearby continuity to sort things out reliably for at least several decades prior to 1910.

  10. Nice to see that you got into contact with Linden Ashcroft. Her guest post on your blog on her work on the homogenization of Australian data before 1908 sounds a little less optimistic than your version. :)

    She had to use absolute homogenization. Absolute homogenization has much larger uncertainties than normal relative homogenization using other stations as reference. Also the quality of relative homogenization depends on how similar the reference stations are. If you combine larger uncertainties with likely strongly too warm observations, you end up with homogenized data that likely is still too warm and not well suited for trend analysis.

    Good luck with your challenging job, do try to compute uncertainties due to remaining inhomogeneities.

  11. Thanks. Can you give me a more detailed or specific reference for the 3rd graph, of Global Land and Ocean Temps? I'd love to use it against the "The data is faked" crowd.

  12. The data sources are mentioned in the text. There is now a very similar plot in the Science article on the new NOAA dataset: Karl et al. (2015).

  13. This post is about early adjustments, but the Telegraph graph of one Paraguay station was about post 1960 adjustments, so your response doesn't seem relevant. I noticed in RealClimate that they also avoided answering the specific point, talking about global trend adjustments overall rather than level adjustments or the Paraguay stations.

    There are lots of other stations, of course, but some stations have oversized effects because they stand in for nearby areas (e.g. the Amazon) not covered by weather stations. Those are the stations to check for manipulation.

  14. People do not always respond to specific cases because there are tenth of thousands of stations. To reply to every case is a tiring game of whack-a-mole.

    But the tireless Kevin Cowtan did the skeptical work, which the people calling themselves did not do. See his video on the temperature in Paraguay.

    I think that the long term trend, which is most relevant for the climate sensitivity is very relevant.

  15. I found this link among comments on a global warming article in the UK Guardian. Thank you for your clear explanation. I'm sending this to others who are interested in climate change.

  16. As requested on twitter.

    Quick search shows numerous articles on both side with opposite claims supported by seemingly contradicting graph about whether adjustment increase or decrease apparent GW. A second look show they may use different data sets (source, period, LM or Global, etc.).

    A service to the community would be to do a good old "state of the question" including blog posts (which laymen read) as opposed to sc articles only (which they don't) from both sides and, if possible, investigating/commenting the pros and cons of their content in your view.

    Now that assume being able to read say a Goddard's post and notice legit sources being included, no dislike of the person can legitimize not critically analyzing his content to either beat his affirmations with better science or learn something.

    Unless someone did that already and I missed it?

  17. This post already explains a large part of the difference. Climate "sceptics" tend to deceive the public by showing only the adjustments for land data and not mentioning that for the global mean the direction is the other way around.

    The warming adjustments for the USA are larger than typical for the rest of the globe. Thus they also like to show that and pretend/imply it is typical or maybe even less because the US network is so good; as if that mattered.

    I do not debate conman. He can write a scientific article. A blog post is normally not accurate enough to see what someone did wrong.

  18. I've found myself at the recieving end of this nonsense before. I was working for the department of parks in australia around the time the usual suspects started hammering the bureau of meterology for supposedly cooking the books. And since some of those datapoints came from us, running software I had actually written myself, the evil eye turned on me.

    It was horrifically insulting and fundamentally illiterate. We where not part of some vast left wing conspiracy to lie about the sky. The weather stations we ran where there specifically to assist in firefighting efforts, and fire prevention. You don't muck about with that sort of thing. Getting a windspeed or direction wrong in a fire is a mistake measured in firefighter lives.

    And so we would calibrate the data, because it would *neglectful* and *dishonest* not to!. So yes, we'd adjust rainfall to account for dew-point calculations. We'd adjust temperature to account for innacuracies when the thermometers where in high or low pressure systems, or in some situations where we knew a particular machine had known innacurracies.

    Did we have an agenda? Hell yes we did. Our agenda was "This better be accurate or I'm going to get a fire-chief screaming at me down the line that the wind is blowing the opposite direction to what the datafeed says."

    Likewise the Bureau of meteorolgy, an organization that doesnt do any research beyond the usual reporting expected of government agencies and minor bits of research around improving the products they deliver to the public and partner agencies (Ie our satelite feeds for fire fighters, etc) And yes they follow accepted science and ensure that their results are calibrated and processed to ensure accurate results.

    I have no positive words for the people who harass public servants like BOM, NASA, and Academics. When I was doing work with CSIRO, the harrassment and bullying I saw scientists get from politicians and members of the public was beyond the pale. It baffles me that knowing how harrassed climate scientists are, and how damaging working on the science is to funding, that they still accuse them of making things up "for grants". Its dishonest, immoral and when it comes to the violence threats and harrasment, its actually evil.


Comments are welcome, but comments without arguments may be deleted. Please try to remain on topic. (See also moderation page.)

I read every comment before publishing it. Spam comments are useless.

This comment box can be stretched for more space.