Showing posts with label global temperature. Show all posts
Showing posts with label global temperature. Show all posts

Monday, July 10, 2017

An ignorant proposal for a BEST project rip-off




It looks like the "Competitive Enterprise Institute" (CEI) just conned their dark money overlords with a stupid report rehashing all the same old claims of the mitigation sceptical movement the BEST project of Richard Muller already studied as a Red Team.

Conservative physics professor Richard Muller claimed that before his BEST project he "did not know whether global warming was real, was completely bogus or may was twice as bad as people said". He was at least open to all sides.

Joe D’Aleo, co-author of the CEI-affiliated report, made the embarrassingly uninformed and wrong claim that “nearly all of the warming they are now showing are in the adjustments.

The report "On the Validity of NOAA, NASA and Hadley CRU Global Average Surface Temperature Data & The Validity of EPA’s CO2 Endangerment Finding - Abridged Research Report" (with annotations) by James P. Wallace III, Joseph S. D’Aleo, and Craig D. Idso provides no evidence for this claim; the graph above shows the opposite is true.

They don't do subtle. But really? You want to claim the Earth is not warming? In 2017?

Glaciers are melting, from the tropical [[Kilimanjaro]] glaciers, to the ones in the Alps and Greenland. Arctic sea ice is shrinking. The growing season in the mid-latitudes has become weeks longer. Trees bud and blossom earlier. Wine can be harvested earlier. Animals migrate earlier. The habitat of plants, animals and insects is shifting poleward and up the mountains. Lakes and rivers freeze later and break-up the ice earlier. The oceans are rising.

Even without looking at any thermometer data, even if we would not have invented the thermometer, physics professor Muller was not sure the Earth is warming? Some corporate lobbyists of the CEI claim the Earth is hardly warming? Really? And the same group of people like to say scientists should get out of the lab more often.

Richard Muller explained in the New York Times the main objections of the mitigation sceptics, which he studied and the CEI wants to study:
We carefully studied issues raised by skeptics: biases from urban heating (we duplicated our results using rural data alone), from data selection (prior groups selected fewer than 20 percent of the available temperature stations; we used virtually 100 percent), from poor station quality (we separately analyzed good stations and poor ones) and from human intervention and data adjustment (our work is completely automated and hands-off). In our papers we demonstrate that none of these potentially troublesome effects unduly biased our conclusions.
In the end Muller and his team found:
Our results turned out to be close to those published by prior groups. We think that means that those groups had truly been very careful in their work, despite their inability to convince some skeptics of that. They managed to avoid bias in their data selection, homogenization and other corrections.



The CEI report carefully avoids any mention of the BEST project. If fact it avoids any mention of previous studies on their "issues". That could be because they are uninformed henchmen, because they want to con their even dumber sponsors or because they want to deceive their friends and keep the public "debate" going on ad nauseam.

If they were real sceptics they would inform themselves and if they do not agree with a claim respond to the arguments. A scientific article thus starts with a description of what is already known and then puts forward new arguments or new evidence. Just repeating ancient accusations, ignoring previous studies, does not lead to a better understanding or a better conversation.

Global mean temperature estimates

Before going over the main mistakes of the report, let me explain how much the Earth is estimated to have warmed, why adjustments need to be made and how these adjustments are made.

The graph below shows the warming since 1880. The red line is the raw data, the blue line the warming estimate after adjustments to account for changes in the way temperature was measured. Directly using raw data, the warming estimate would have been larger. Due to adjustments about 10% of the warming is removed.

This would be a good point to remember that Joe D’Aleo wrong claimed that “nearly all of the warming they are now showing are in the adjustments.”  It is really really hard to be more wrong. Joe D’Aleo gets points for effort.



The main reason why the raw data suggests more warming is how sea surface temperature was historically measured. The ocean surface warming estimates of the UK Hadley centre are shown below. The main adjustment necessary is for the transition of bucket observations to measurement at the engine cooling water inlet, which mostly happened in the decades around the WWII. The war itself is an especially difficult case.

Bucket measurements are made by hauling a bucket of water from the ocean and stirring a thermometer until it has the temperature of the water. The problem is that the water cools due to evaporation between the time it is lifted from the ocean and the time the thermometer is read.

This is not only a large adjustment, but also a large uncertainty. Initially is was estimated that the bucket measurements were about 0.4 °C colder. Nowadays the estimate, depending on the bucket and the period, is about 0.2 °C, but it can be anywhere between 0.4 °C and zero. We studied these biases with experiments on research vessels, in labs and numerical modelling and by comparing measurements made by different nearby ships/platforms.



The graph below shows the warming over land as estimated from weather station data by US NOAA (GHCNv3). Over land the warming was larger than the raw observations suggest. The adjustments are made by comparing every candidate station with its nearby neighbours. Changes in the regional climate will be the same in all stations, any change that only happens at the candidate station is not representative for the region, but likely a change in how temperature was observed.

There are many reasons why stations may not measure the regional climate correctly. The best known is urbanization of the local surrounding of the station. Cities are often warmer than the surrounding region and when cities grow this can produce a warming signal. This is a correct measurement, but not the large-scale warming of interest and should thus be removed. The counterpart of urbanization is that city stations are often moved to the outskirts, which typically produces a cooling jump that also needs to be removed. This can even be important for small villages.

City stations moving to cooler airports can produce an artificial cooling. Also modern equipment generally measures a bit cooler than early instruments.



Where the CEI report gives examples of data before and after adjustment, do you want to guess whether they showed the sea surface temperature or the land surface temperature?

Sea or land? What do you think? I'll wait.


If you guessed the land surface temperature you won the price: a free twitter account to tell the Competitive Enterprise Institute what you think of the quality of their propaganda.

The ocean is 71% of the Earth's surface. Thus if you combine these two temperature signals taking the area of the land and the ocean into account the net effect of the adjustments is a reduction of global warming.


The Daily caller interviewed D'Aleo and calls the report a "peer-reviewed study". Suggesting that it underwent the quality control by scientists with expertise in the field that is typical for scientific publications. There is no evidence that the report is published in the scientific literature and the blog science quality, lack of clarity how the figures were computed and where their data comes from, the lack of evidence for the claims, the lack of references to the scientific literature makes it highly unlikely that this work is peer reviewed, to say it in a friendly way. There is no quality bar they will not limbo underneath; they don't do subtle.

Homogenisation

The estimation of the climatic changes at a station using neighbouring stations to remove local artefacts is called statistical homogenisation. The basic idea of comparing a candidate station with its neighbours is easy, but with typically multiple jumps at one station and also jumps in the neighbouring stations it becomes a beautiful statistical problem people can work on for decades.

Naturally scientists also study how well they can remove these artefacts. It is sad this needs to be mentioned, but the more friendly blog posts of the mitigation sceptical movement (implicitly) assume scientists are stupid and don't do any due diligence. Right, that is how we got the moon and produced smart phones.



Such a study was actually how I started with this topic. The homogenisation community needed an outsider to make a blind benchmarking of their methods. So I generated a dataset with homogeneous station data where you need to get the variability of the stations right and the variability (correlations) between the stations. As the name of this blog suggests just the job I like.

To this homogeneous data we added inhomogeneities. For me that was the biggest task, talking with dozens of experts from many different countries how inhomogeneities typically look like. How many (about one per 20 years), how big (about 0.8 °C per jump), how many gradual inhomogeneities and how big (to model urbanization), how often do multiple stations have a simultaneous jump (for example, due to a central change in the time of observation).

I gave this inhomogeneous station dataset to my colleagues, who homogenised it and, after everyone returned the data, we analysed the results. We found that all methods improved the quality of monthly temperature data. More importantly for us was that modern homogenisation packages were clearly better than traditional methods. The work of the last decade had paid off.

The figure to the right is from a similar blind validation study for the homogenisation method NOAA used to homogenise GHCNv3 and shows something important. The four panels are four different assumptions about how inhomogeneities and the climate looks like. This study chose to make some inhomogeneity cases that were really easy and some that were really hard.

On the horizontal axis are three periods. The red crosses are the trends in the inhomogeneous data, the green crosses the ones in the homogeneous data, which the homogenisation algorithms are supposed to recover and the yellow/orange crosses are the trends of the homogenised data.

The important thing is that the yellow cross is always in between: homogenisation improved the trend estimates, but part of the error in the trend remains. In the most difficult case of this study, which I consider unrealistic, the homogenised result was in the middle. Half of the trend error was removed, half remained.

Because real raw station data shows too little warming and statistical homogenisation makes the trend larger, better homogenisation thus also means stronger temperature trends over land. Homogenisation became better because of better homogenisation algorithms and because we have more data due to continual digitisation efforts. With more data, the stations will on average be closer together and thus experience more similar weather. This means that it becomes easier to see homogeneities in their differences.

CEI claims from Daily Caller

Michael Bastasch of the Daily Caller makes several unsupported or wrong claims about the report. Other claims are already wrong in report.
A new study found adjustments made to global surface temperature readings by scientists in recent years “are totally inconsistent with published and credible U.S. and other [New Zealand and upper air] temperature data.”
No shit, Sherlock. Next you will tell me that cassoulet does not taste like a McDonald’s Hamburger, sea food or a cream puff. The warming of different air masses is different? Who would have thought so?

This becomes most Kafkaesk when the authors want to see the high number of 100 Fahrenheit days of the 1930s US [[Dust Bowl]] in the global monthly average temperature and call this a "cyclical pattern". Not sure whether a report aimed at the Tea Party folks should insult American farmers and claim they will mismanage their lands to produce a Dust Bowl in regular cycles.


The report is drenched in conspiratorial thinking:
Basically, “cyclical pattern in the earlier reported data has very nearly been ‘adjusted’ out” of temperature readings taken from weather stations, buoys, ships and other sources.
They do not even critique the methods used or even mention them and do acknowledge that adjustments are necessary, but the pure outcome being inconvenient for their donors is enough to complain.

It also illustrates that the mitigation sceptical movement is preoccupied with the outcome and not with the quality of a study. Whether a new study is praised or criticized on their movement blog Watts Up With That depends on the outcome, on whether it can be spun as making their case against solving climate change stronger or weaker. On science blogs it depends on the quality and the strength of the evidence.

As I already showed above, the adjustments make the estimated warming smaller. The exact opposite is claimed by the Daily Caller:
In fact, almost all the surface temperature warming adjustments cool past temperatures and warm more current records, increasing the warming trend, according to the study's authors.
The study provides no evidence for this. They do not show the warming before and after adjustment for the global temperature, only for the land temperature.

Is it too much to ask to inform yourself before you accuse scientists of wrongdoing? Is it too much to ask if you write a report about the global temperature to read some scientific articles on data problems in the sea surface temperature? Is is too much to ask if you talk about the 1940s to wonder whether the WWII might have influences the measurements?
“Each dataset pushed down the 1940s warming and pushed up the current warming.”
The war increased the percentage of American navy vessels, which make engine intake measurements, and decreased the percentage of merchant ships, which make bucket measurements. That produces a spurious warm peak in the raw data.

Modern data also have a better coverage over the Earth. Locally there is more decadal variability, what they call "cyclical pattern". A better coverage will remove spurious decadal variability from the global average.

I have no clue why they would think this:
“You would think that when you make adjustments you’d sometimes get warming and sometimes get cooling. That’s almost never happened,” said D’Aleo, who co-authored the study with statistician James Wallace and Cato Institute climate scientist Craig Idso.
The transitions in the measurements methods due to technological and economic changes can naturally affect the global average temperature. For example ships in the 19th century used bucket measurements, now most sea surface temperature data comes from buoys.

If you assume inhomogeneities can have no influence on the global mean, like D'Aleo, then why are the mitigation sceptics claiming to be worried about the influence of urbanization on the global mean temperature? If that were the main problem, the adjustments would tend to produce cooling more often than warming to remove this problem. They would not "sometimes get warming and sometimes get cooling".

The report was an embarrassing mixture of the worst of blog science. The Daily Caller post managed to make it worse.

The positive side of Trump claiming that his inauguration was the biggest evah, is that the public now understands were such wild claims come from. Science is harder to check than crowd sizes. Even if you do not know them personally, there are people on this globe willing to deny the existence of global warming without blinking an eye.



Related reading

Quality of climate data

The climate scientists of Climate Feedback had a look at an Breitbart article on the same report. Seven scientists analyzed the article and estimate its overall scientific credibility to be 'very low'. Breitbart article falsely claims that measured global warming has been “fabricated”.

Fact checker of urban legends Snopes judged the Breitbart article to be: False. Surprise. Had Breitbart known it to be true, they would not have published it.

Ars Technica: Thorough, not thoroughly fabricated: The truth about global temperature data. How thermometer and satellite data is adjusted and why it must be done.

John Timmer at Ars Technica is fed up with being served the same story about some upward adjusted stations every year: Temperature data is not “the biggest scientific scandal ever” Do we have to go through this every year?

Steven Mosher, a climate sceptic and member of the BEST project: all the adjustments demanded by the "sceptics".

The astronomer behind And Then There's Physics writes why the removal of non-climatic effects makes sense. In the comments he talks about adjustments made to astronomical data. Probably every numerical observational discipline of science performs data processing to improve the accuracy of their analysis.

Nick Stokes, an Australian scientist, has a beautiful post that explains the small adjustments to the land surface temperature in more detail.

Two posts of mine about some reasons for temperature trend biases: Temperature bias from the village heat island and Changes in screen design leading to temperature trend biases.

You may also be interested in the posts on how homogenization methods work (Statistical homogenisation for dummies) and how they are validated (New article: Benchmarking homogenisation algorithms for monthly data).

Just the facts, homogenization adjustments reduce global warming.

Zeke Hausfather: Major correction to satellite data shows 140% faster warming since 1998.

If you would like to read a peer reviewed scientific article showing the adjustments, the influence of the adjustments on the global mean temperature is also shown in Karl et al. (2015).

NOAA's benchmarking study: Claude N. Williams ,Matthew J.Menne, Peter W. Thorne, 2012: Benchmarking the performance of pairwise homogenization of surface temperatures in the United States. Journal Geophysical Research, doi: 10.1029/2011JD016761.

On my benchmarking study: New article: Benchmarking homogenisation algorithms for monthly data.

Corporate war on science

The Guardian on the CEI report and their attempt to attack the endangerment finding: Conservatives are again denying the very existence of global warming.

Another post on the CEI report: Silly Non-Study Supposedly Strengthens Endangerment Challenge.

My first post on the Red Cheeks Team.

My last post on the Red Team idea: The Trump administration proposes a new scientific method just for climate studies.

Great piece by climate scientist Ken Caldeira: Red team, blue team.

Phil Newell: One Team, Two Team, Red Team, Blue Team.

Why doesn't Big Oil fund alternative climate research? Hopefully a rhetorical question. They would have had a lot to gain if they thought the science were wrong, but they fund PR not science.

Union of Concerned Scientists on the funding of the war by Exxon: ExxonMobil Talks A Good Game, But It’s Still Funding Climate Science Deniers.

The New Republic on several attacks on science by Scott Pruitt: The End Goal of Trump’s War on Science.

Mother Jones: A Jaw-Dropping List of All the Terrible Things Trump Has Done to Mother Earth. Goodbye regulations designed to protect the environment and public health.

Tuesday, June 20, 2017

On the recent warming surge

"Incidentally, when in the journal Science in 2007 we pointed to the exceptionally large warming trend of the preceding 16 years, which was at the upper end of the [climate] model range, nobody cared, because there is no powerful lobby trying to exaggerate global warming."

"And of course in our paper we named natural intrinsic variability as the most likely reason. But when a trend at the lower end of the model range occurs it suddenly became a big issue of public debate, because that was pushed by the fossil fuel climate sceptics’ lobby. There is an interesting double standard there."

Maybe the comment is deleted in shame. At least I cannot find it any more. Someone on Reddit was making exactly the same errors as the fans of the infamous "hiatus" to argue that global warming since 2011 is exploding and we're soon gonna die. Deleted comments can be freely paraphrased.

The data

So let's have a look at what the data actually says. Below are warming curves of two surface temperature datasets and two upper air temperature satellite retrievals. I shortened the warming curve of Berkeley Earth to match the period to the one of GISTEMP. All the other datasets are shown over their full lengths. Taking this step back, looking at the overview, there clearly is no "hiatus" and no "warming surge".

The red dots are the annual temperature anomalies, which are connected by a thin grey line. The long-term trend is plotted by a thick blue line, which is a [[LOESS]] estimate.

If you want to see the "hiatus" you have to think the last two data points away and start in the temperature peak of 1998. Don't worry, I'll wait while you search for it.






A hiatus state of mind

After seeing reality, let's now get into the mindset of a climate "sceptic" claiming to have evidence for a "hiatus", but do this differently by looking at the data since 2011.

Naturally we only plot the recent part of the data, so that context is lost. I am sure the climate "sceptics" do not mind, they also prefer to make their "hiatus" plots start around the huge 1998 El Nino warming peak and not show the fast warming before 1998 for context.




The thick blue lines are quadratic functions fitted to the data. They fit snugly. As you can see both the linear and quadratic coefficients are statistically significant. So clearly the recent warming is very fast, faster than linear and we can soon expect to cross the 2 °C limit, right?

I am sure the climate "sceptics" do not mind what I did. They also cherry picked a period for their "hiatus" and applied a naive statistical test as if they had not cherry picked the period at all. The climate "sceptics" agreeing with Christopher Monckton doing this on WUWT will surely not object to our Redditor doing the same and will conclude with him that the end is nigh.

By the way, I also cherry picked the dataset. The curves of the upper air temperature retrievals are not as smooth and the quadratic terms were not statistically significant. But in case of the "hiatus" debate our "sceptical" friends also only showed data for the datasets showing the least warming. So I am sure they do not object to this now.

If you look at the full period you can see that the variability of the temperature signal is much larger than the variability around the quadratic fit. It is thus clearly a complete coincidence that the curve is so smooth. But, well, the "hiatus" proponents also just look statistically at their cherry picked period and ignore the actual uncertainties (including slowly varying ones.)

Some more knowledgeable catastrophists may worry that it looks as if 2017 may not be another record scorching year. No worries. Also no worries if 2018 is colder again. The Global Warming Policy Foundation thinks it is perfectly acceptable to ignore a few politically inconvenient years and claims that we just have to think 2015 and 2016 away and that thus the death of the "hiatus" has been greatly exaggerated. I kid you not. I have not seen any climate "sceptic" or any "luckwarmer" complaining about this novel statistical analysis method, so surely our Redditor can do the same. Catastrophic warming surge claims are save till at least 2019.



Don't pick cherries

Back to reality. What to do against cherry picking periods? My advice would be: don't cherry pick periods. Not for the global mean temperature, not for the temperature of the Antarctic Peninsula, not for any other climate variable. Just don't do it.

If you have a physical reason to expect a trend change, by all means use that date as start of the period to compute a trend. But for 1998 or 2011 there is no reason to expect a trend change.

Our cheerful Redditor had even a bit more physics in his claim. He said that the Arctic was warming and releasing more carbon dioxide and methane. However, these emissions were not large enough to make the increase in the global greenhouse concentrations speed up. And they count. From our good-natured climate "sceptics" I have never heard a physical explanation why global warming would have stopped in 1998. But maybe I have missed their interest in what happens with the climate system.

If you have no reason to expect a trend change the appropriate test would be for a trend change test at an unknown date. Such a test "knows" that cherry picked periods can have hugely different trends and thus correctly only sees larger trend changes over longer periods as statistically significant. Applied to temperature data the result of such as test does not see any "hiatus" nor any "warming surge".

This test gave the right result during the entire "hiatus" madness period. Don't fool yourself. Use good stats.



Related reading

Stefan Rahmstorf Grant Foster and Niamh Cahill: Global temperature evolution: recent trends and some pitfalls. Environmental Research Letters, 2017. See also "Change points of global temperature".

Cranberry picking short-term temperature trends

Statistically significant trends - Short-term temperature trend are more uncertain than you probably think

How can the pause be both ‘false’ and caused by something?

Atmospheric warming hiatus: The peculiar debate about the 2% of the 2%

Sunday, February 5, 2017

David Rose's alternative reality in the Daily Mail

Peek-a-boo! Joanna Krupa shows off her stunning figure in see-through mesh dress over black underwear
Bottoms up! Perrie Edwards sizzles in plunging leotard as Little Mix flaunt their enviable figures in skimpy one-pieces
Bum's the word! Lottie Moss flaunts her pert derriere in a skimpy thong as she strips off for steamy selfie

Sorry about those titles. They provide the fitting context right next to a similarly racy Daily Mail on Sunday piece of David Rose: "Exposed: How world leaders were duped into investing billions over manipulated global warming data". Another article on that "pause" thingy that mitigation skeptics do their best to pretend not to understand. For people in the fortunate circumstances not to know what the Daily Mail is, this video provides some context about this Murdoch "newspaper".

[UPDATE: David Rose' source says in an interview with E&E News on Tuesday: “The issue here is not an issue of tampering with data”. So I guess you can skip this post, except if you get pleasure out of seeing the English language being maltreated. But do watch the Daily Mail video below.

See also this article on the void left by the Daily Mail after fact checking. I am sure all integrityTM-waving climate "skeptics" will condemn David Rose and never listen to him again.]



You can see this "pause" in the graph below of the global mean temperature. Can you find it? Well you have to think those last two years away and then start the period exactly in that large temperature peak you see in 1998. It is not actually a thing, it is a consequence of cherry picking a period to get a politically convenient answer (for David Rose's pay masters).



In 2013 Boyin Huang of NOAA and his colleagues created an improved sea surface dataset called ERSST.v4. No one cared about this new analysis. Normal good science. One of the "scandals" Rose uncovered was that NOAA is drafting an article on ERSST.v5.

But this post is unfortunately about nearly nothing, about the minimal changes in the top panel of the graph below. I feel the important panel is the lower one. It shows that in the raw data the globe seems to warm more. This is because before WWII many measurements were performed with buckets and the water in the bucket would cool a little due to evaporation before reading the thermometer. Scientists naturally make corrections for such problems (homogenization) and that helps make a more accurate assessment of how much the world actually warmed.

But Rose is obsessed with the top panel. I made the graph extra large, so that you can see the differences. The thick black line shows the new assessment (ERSST.v4) and the thin red line the previously estimated global temperature signal (ERSST.v3). Differences are mostly less than 0.05°C, both warmer and cooler. The "problem" is the minute change at the right end of the curves.

The mitigation skeptical movement was not happy when a paper in Science in 2015, Karl and colleagues (2015), pointed out that due to this update the "pause" is gone, even if you use the bad statistics the mitigation skeptics like. As I have said for many years now about political activists claiming this "pause" is highly important: if your political case depends on such minute changes, your political case is fragile.



In the mean time a recent article in Science Advances by Zeke Hausfather and colleagues (2016) now shows evidence that the updated dataset (ERSSTv4) is indeed better than the previous version (ERSSTv3b). They do so by comparing the ERSST dataset, which comes from a large number of data sources, with data that comes only from only one source (buoys, satellites (CCl) or ARGO). These single-source datasets are shorter, but without trend uncertainties due to the combination of sources. The plot below shows that the ERSSTv4 update improves the fit with the other datasets.



The trend change over the cherry-picked "pause" period were mostly due to the changes in the sea surface temperature of ERSST. Rose makes a lot of noise about the land data, where the update was inconsequential. As indicated in Karl and colleagues (2015) this was a beta-version dataset. The raw data was published; that is the data of the International Surface Temperature Initiative (ISTI) and the homogenization method was published. The homogenization method works well; I checked myself.

The dataset itself is not published yet. Just applying a known method to a known dataset is not a scientific paper. Too boring.

So for the paper NOAA put a lot of work into estimating the uncertainty due to the homogenization method. When developing a homogenization method you have to make many choices. For example, inhomogeneities are found by comparing one candidate station with multiple nearby reference stations. There are settings for now many stations and for how nearby the reference stations need to be. NOAA studied which of these settings are most important with a nifty new statistical method. These settings were varied to study how much influence that has. I look forward to reading the final paper. I guess Rose will not read it and stick to his role as suggestive interpreter of interpreters.

The update of NOAA's land data will probably remove a precious conspiracy of the mitigation skeptical movement. While, as shown above, the adjustments reduce our estimate for the warming of the entire world, the adjustments make the estimate for the warming over land larger. Mitigation skeptics like to show the adjustments for land data only to suggest that evil scientists are making global warming bigger.

This is no longer the case. A recommendable overview paper by Philip Jones, The Reliability of Global and Hemispheric Surface Temperature Records, analyzed the new NOAA dataset. The results for land are shown below. The new ISTI raw data dataset shows more warming than the previous NOAA raw data dataset. As a consequence the homogenization now does not change the global mean appreciably any more to arrive at about the same answer after homogenization; compare NOAA uncorrected (yellow line) with NOAA (red; homogenized).



The main reason for the smaller warming in the old NOAA raw data was that this smaller dataset contained a higher percentage of airport stations. That is because airports report their data very reliably in near real time. Many of these airport stations were in cities before and cities are warmer than airports due to the urban heat island effect. Such relocations thus typically cause cooling jumps that are not related to global warming and are removed by homogenization.

So we have quite some irony here.
Still Rose sees a scandal in these minute updates and dubs it Climategate 2; I thought we were already at 3 or 4. In this typical racy style he calls data "wrong", "rogue", "biased". Knowing that data is never perfect is why scientists do their best to assess the quality of the data, remove problems and make sure that the quality is good enough to make a certain statement. In return people like David Rose simultaneously pontificate about uncertainty monsters and assume data is perfect and then get the vapors when updates are needed.

Rose gets some suggestive quotes from an apparently disgruntled retired NOAA employee. The quotes themselves seem to be likely inconsequential procedural complaints, the corresponding insinuations seem to come from Rose.

I thought journalism had a rule that claims by a source need to be confirmed by at least a second source. I am missing any confirmation.

While Rose presents the employee as an expert on the topic, I have never heard of him. Peter Thorne, who worked at NOAA, confirms that the employee did not work with surface station data himself. He has a decent publication record, mainly on satellite climate datasets of clouds, humidity and radiation. Ironically, I keep using that word, he also has papers about the homogenization of his datasets, while homogenization is treated by the mitigation skeptical movement as the work of the devil. I am sure they are willing to forgive him his past transgressions this time.

It sounds as if he made a set of procedures for his climate satellite data, which he really liked, and wanted other groups in NOAA to use it as well. Was frustrated when others did not prioritize enough updating their existing procedures to his.

For David Rose this is naturally mostly about politics and in his fantasies the Paris climate treaty would not have existed with the Karl and colleagues (2015) paper. I know that "pause" thingy is important for the Anglo-American mitigation skeptical movement, but let me assure Rose that the rest of the world considers all the evidence and does not make politics based on single papers.

[UPDATE: Some days you gotta love journalism: a journalist asked several of the diplomats who worked for years on the Paris climate treaty, they gave the answer you would expect: Contested NOAA paper had no influence on Paris climate deal. The answers still give an interesting insight into the sausage making. What is actually politically important.]

David Rose thus ends:
Has there been an unexpected pause in global warming? If so, is the world less sensitive to carbon dioxide than climate computer models suggest?
No, there never was an "unexpected pause." Even if there were, such a minute change is not important for the climate sensitivity. Most methods do not use the historical warming for that and those that do consider the full warming of about 1°C since the 19th century and not only short periods with unreliable, noisy short-term trends.

David Rose:
And does this mean that truly dangerous global warming is less imminent, and that politicians’ repeated calls for immediate ‘urgent action’ to curb emissions are exaggerated?
No, but thanks for asking.

Post Scriptum. Sorry that I cannot talk about all errors in the article of David Rose, if only because in most cases he does not present clear evidence and because this post would be unbearably long. The articles of Peter Thorne and Zeke Hausfather are mostly complementary on the history and regulations at NOAA and on the validation of NOAA's results, respectively.

Related information

Buzzfeed (October 2017): This Is How A Bogus Climate Story Becomes Unstoppable On Social Media

New York Times (September 2017): British Press Watchdog Says Climate Change Article Was Faulty

2 weeks later. The nailing New York Times interviewed several former colleagues of NOAA retire Bates: How an Interoffice Spat Erupted Into a Climate-Change Furor. "He’s retaliating. It’s like grade school ... At that meeting, Dr. Bates shouted that Ms. McGuirk was not trustworthy and belonged in jail, according to an internal log ..." Lock her up, lock her up, ...

Wednesday. The NOAA retiree now says: "The Science paper would have been fine had it simply had a disclaimer at the bottom saying that it was citing research, not operational, data for its land-surface temperatures." To me it was always clear it was research data, otherwise they would have cited a data paper and named the dataset. How a culture clash at NOAA led to a flap over a high-profile warming pause study

Tuesday. is a balanced article from the New York Times: Was Data Manipulated in a Widely Cited 2015 Climate Study? Steve Bloom: "How "Climategate" should have been covered." Even better if mass media would not have to cover office politics on archival standards fabricated into a fake scandal.

Also on Tuesday, an interview of E&E News: 'Whistleblower' says protocol was breached but no data fraud: The disgruntled NOAA retiree: "The issue here is not an issue of tampering with data".

Associated Press: Major global warming study again questioned, again defended. "The study has been reproduced independently of Karl et al — that's the ultimate platinum test of whether a study is to be believed or not," McNutt said. "And this study has passed." Marcia McNutt, who was editor of Science at the time the paper was published and is now president of the National Academy of Sciences.

Daily Mail’s Misleading Claims on Climate Change. If I were David Rose I would give back my journalism diploma after this, but I guess he will not.

Monday. I hope I am not starting to bore people by saying that Ars Technica has the best science reporting on the world wide web. This time again. Plus inside scoop suggesting all of this is mainly petty office politics. Sad.

Sunday. Factcheck: Mail on Sunday’s ‘astonishing evidence’ about global temperature rise. Zeke Hausfather wrote a very complementary response, pointing out many problems of the Daily Mail piece that I had to skip. Zeke works at the Berkeley Earth Surface Temperature project, which produces one of the main global temperature datasets.

Sunday. Peter Thorne, climatology professor in Ireland, former NOAA employee and leader of the International Surface Temperature Initiative: On the Mail on Sunday article on Karl et al., 2015.

Phil Plait (Bad Astronomy) — "Together these show that Rose is, as usual, grossly exaggerating the death of global warming" — on the science and the politics of the Daily Mail piece: Sorry, climate change deniers, but the global warming 'pause' still never happened

You can download the future NOAA land dataset (GHCNv4-beta) and the land dataset used by Karl and colleagues (2015), h/t Zeke Hausfather.

The most accessible article on the topic rightly emphasizes the industrial production of doubt for political reasons: Mail on Sunday launches the first salvo in the latest war against climate scientists.

A well-readable older article on the study that showed that ERSST.v4 was an improvement: NOAA challenged the global warming ‘pause.’ Now new research says the agency was right.

One should not even have to answer the question, but: No, U.S. climate scientists didn't trick the world into adopting the Paris deal. A good complete overview at medium level.

Even fact checker Snopes sadly wasted its precious time: Did NOAA Scientists Manipulate Climate Change Data?
A tabloid used testimony from a single scientist to paint an excruciatingly technical matter as a worldwide conspiracy.

Carbon Brief Guest post by Peter Thorne on the upcoming ERSSTv5 dataset, currently under peer review: Why NOAA updates its sea surface temperature record.

Monday, January 30, 2017

With some programing skills you can compute global mean temperatures yourself

This is a guest post by citizen scientist Ron Roeland (not his real name, but I like alliteration for some reason). Being an actually sceptical person, he decided to compute the global mean land temperature from station observations himself. He could reproduce the results of the main scientific groups that compute this signal and, new for me, while studying the data noticed how important the relocation of temperature stations to airports is for the NOAA GHCNv3 dataset. (The headers in the post are mine.)

This post does not pretend to present a rigorous analysis of the global temperature record; instead, it intends to show how easy it is for someone with basic programming/math skills to debunk claims that NASA and NOAA have manipulated temperature data to produce their global-average temperature results, i.e. claims like these:

From C3 Headlines: By utilizing questionable adjustments based on even more questionable assumptions, NOAA managed to produce an entirely fabricated increase in the global warming trend from 1998 to 2012.

From a blogger on the Hill: There’s going to have to be a massive effort to pick apart failing climate models and questionably-adjusted data.

From Climate Depot: Over the past decade, NASA and NOAA have continuously altered the temperature record to cool the past and warm the present. Their claims are straight out Orwell's 1984, and have nothing to do with science'

The routine

Some time ago, after reading all kinds of claims (like the ones above) about how NASA and NOAA had improperly adjusted temperature data to produce their global-average temperature results, I decided to take a crack at the data myself.

I coded up a straightforward baselining/gridding/averaging routine that is quite simple and “dumbed down” in comparison to the NASA and NOAA algorithms. Below is a complete description of the algorithm I coded up.
  1. Using GHCN v3 monthly-average data, compute 1951-1980 monthly baseline temperatures for all GHCN stations. If a station has 15 or more valid temperatures in any given month for the 1951-1980 baseline period, retain that monthly baseline value; otherwise drop that station/month from the computations. Stations with no valid monthly baseline periods are completely excluded from the computations.
  2. For all stations and months where valid baseline temperature estimates were computed per (1) above, subtract the respective baseline temperatures from all of the station monthly temperature temperatures to produce monthly temperature anomalies for the years 1880-2015.
  3. Set up a global gridding scheme to perform area-weighting. To keep things really simple, and to minimize the number of empty grid-cells, I selected large grid-cell sizes (20 degrees x 20 degrees at the Equator). I also opted to recalculate the grid-cell latitude dimensions as one goes north/south of the equator in order to keep the grid-cell areas as nearly constant as possible. I did this to keep the grid-cell areas from shrinking (per the latitude cosines) in order to minimize the number of empty grid cells.
  4. In each grid-cell, compute the average (over all stations in the grid-cell) of the monthly temperature anomalies to produce a single time-series of average temperature anomalies for each month (years 1880 through 2015).
  5. Compute global average monthly temperature anomalies by averaging together all the grid-cell monthly average anomalies, weighted by the grid-cell areas (again, for years 1880 through 2015).
  6. Compute global-average annual anomalies for years 1880 through 2015 by averaging together the global monthly anomalies for each year.
The algorithm does not involve any station data adjustments (obviously!) or temperature interpolation operations. It’s a pretty basic number-crunching procedure that uses straightforward math plus a wee bit of trigonometry (for computing latitude/longitude grid-cell areas).

For me, the most complicated part of the algorithm implementation was managing the variable data record lengths and data gaps (monthly and annual) in the station data -- basically, the “data housekeeping” stuff. Fortunately, modern development libraries such as the C++ Standard Template Library make this less of a chore than it used to be.

Why this routine?

People unfamiliar with global temperature computational methods sometimes ask: “Why not simply average the temperature station data to compute global-average estimates? Why bother with the baselining and gridding described above?”

We could get away with straight averaging of the temperature data if it were not for the two problems described below.

Problem 1: Temperature stations have varying record lengths. The majority of stations do not have continuous data records that go all the way back to 1880 (the beginning of the NASA/GISS global temperature calculations). Even stations with data going back to 1880 have gaps in their records -- there are missing months or even years.

Problem 2: Temperature stations are not evenly distributed over the Earth’s surface. Some regions, like the continental USA and western Europe, have very dense networks of stations. Other regions, like the African continent, have very sparse station networks.

As a result of problem 1, we have a mix of temperature stations that changes from year to year. If we were simply to average the absolute temperature data from all those stations, the final global-average results would be significantly skewed from year to year due to the changing mix of stations from one year to the next.

Fortunately, the solution for this complication is quite straightforward: the baselining and anomaly-averaging procedure described above. For those who already familiar with this procedure, please bear with me while I illustrate how it works with a simple scenario constructed from simulated data.

Let’s consider a very simple scenario where the full 1880-2016 temperature history for a particular region is contained in data reported by two temperature stations, one of which is located on a hilltop and the other located on a nearby valley floor. The hilltop and valley floor locations have identical long-term temperature trends, but the hilltop location is consistently about 1 degree C cooler than the valley floor location. The hilltop temperature station has a temperature record starting in 1880 and ending in 1990. The valley floor station has a temperature record beginning in 1930 and ending in 2016.

Figure 1 below shows the simulated temperature time-series for these two hypothetical stations. Both time-series were constructed by superimposing random noise on the same linear trend, with the valley-floor station time-series having a constant offset temperature 1 degree C more than that of the hilltop station time-series. The simulated time-series for the hilltop station (red) begins in 1880 and continues to 1990. The simulated valley floor station temperature (blue) data begins in 1930 and runs to 2016. As can be seen during their period of overlap (1930-1990), the simulated valley-floor temperature data runs about 1 degree warmer than the simulated hilltop temperature data.


Figure 1: Simulated Hilltop Station Data (red) and Valley Floor Station Data (blue)

If we were to attempt to construct a complete 1880-2016 temperature history for this region by computing a straight average of the hilltop and valley floor data, we would obtain the results seen in Figure 2 below.


Figure 2: Straight Average of Valley Floor Station Data and Hilltop Station Data

The effects of the changing mix of stations (hilltop vs. valley floor) on the average temperature results can clearly be seen in Figure 2. A large temperature jump is seen at 1930, where the warmer valley floor data begins, and a second temperature jump is seen at 1990 where the cooler hilltop data ends. These temperature jumps obviously do not represent actual temperature increases for that particular region; instead, they are artifacts introduced by the changes in the mix of stations in 1930 and 1990.

An accurate reconstruction of the regional temperature history computed from these two temperature time-series obviously should show the warming trend seen in the hilltop and valley floor data over the entire 1880-2016 time period. That is clearly not the case here. Much of the apparent warming seen in Figure 2 is a consequence of the changing mix of stations.

Now, let’s modify the processing a bit by subtracting the (standard NASA/GISS) 1951-1980 hilltop baseline average temperature from the hilltop temperature data and the 1951-1980 valley floor baseline average temperature from the valley floor temperature data. This procedure produces the temperature anomalies for the hilltop and valley floor stations. Then for each year, compute the average of the station anomalies for the 1880-2016 time period.

This is the baselining and anomaly-averaging procedure that is used by NASA/GISS, NOAA, and other organizations to produce their global-average temperature results.

When this baselining and anomaly-averaging procedure is applied to the simulated temperature station data, it produces the results that can be viewed in figure 3 below.


Figure 3: Average of Valley Floor Station Anomalies and Hilltop Station Anomalies

In Figure 3, the temperature jumps associated with the beginning of the valley floor data record and the end of the hilltop data record have been removed, clearly revealing the underlying temperature trend shared by the two temperature time-series.

Also note that although neither of my simulated temperature stations have a full 1880-2016 temperature record, we were still able to compute a complete reconstruction for the 1880-2016 time period because there was enough overlap between the station records to allow us to “align” them via baselining.

The second problem, the non-uniform distribution of temperature stations, can clearly be seen in Figure 4 below. That figure shows all GHCNv3 temperature stations that have data records beginning in 1900 or earlier and continuing to the present time.


Figure 4: Long-Record GHCN Station Distribution

As one can see, the stations are highly concentrated in the continental USA and western Europe; Africa and South America, in contrast, have very sparse coverage. A straight unweighted average of the data from all the stations shown in the above image would result in temperature changes in the continental USA and western Europe “swamping out” temperature changes in South America and Africa in the final global average calculations.

That is the problem that gridding solves. The averaging procedure using grid-cells is performed in two steps. First, the temperature time-series for all stations in each grid-cell are averaged together to produce a single time-series per grid-cell. Then all the grid-cell time-series are averaged together to construct the final global-average temperature results (note: in the final average, the grid-cell time-series are weighted according to the size of each grid-cell). This eliminates the problem where areas on the Earth with very dense networks of stations are over-weighted in the global average relative to areas where the station coverage is more sparse.

Now, some have argued that the sparse coverage of certain regions of the Earth invalidate the global-average temperature computations. But it turns out that the NASA/GISS warming trend can be confirmed even with a very sparse sampling of the Earth’s surface temperatures. (In fact, the NASA/GISS warming trend can be replicated very closely with data from as few as 30 temperature stations scattered around the world.)

Real-world results

Now that we are done with the preliminaries, let’s look at some real-world results. Let’s start off by taking a look at how my simple “dumbed-down” gridding/averaging algorithm compares with the NASA/GISS algorithm when it is used to process the same GHCNv3 adjusted data that NASA/GISS uses. To see how my algorithm compares with the NASA/GISS algorithm, take a look at Figure 5 below, where the output of my algorithm is plotted directly against the NASA/GISS “Global Mean Estimates based on Land Data only” results.

(Note: All references to NASA/GISS global temperature results in this post refer specifically to the NASA/GISS “Global Mean Estimates based on Land Data only” results. Those results can be viewed on the NASA/GISS web-site; scroll down to view the “Global Mean Estimates based on Land Data only” graph).


Figure 5: Adjusted Data, All Stations: My Simple Gridding/Averaging (blue) vs. NASA/GISS (red)

In spite of the rudimentary nature of my algorithm, my algorithm produces results that match the NASA/GISS results quite closely. According to the R-squared statistic I calculated (seen in the upper-left corner of Figure 5), I got 98% of the NASA/GISS answer with a only tiny fraction of the effort!

But what happens when we use unadjusted GHCNv3 data? Well, let’s go ahead and compare the output of my algorithm with the NASA/GISS algorithm when my algorithm is used to process the unadjusted GHCNv3 data. Figure 6 below shows a plot of my unadjusted global temperature results vs. the NASA/GISS results (remember that NASA/GISS uses adjusted GHCNv3 data).


Figure 6: Unadjusted Data, All Stations: My Simple Gridding /Averaging (green) vs. NASA/GISS (red)

My “all stations” unadjusted data results show a warming trend that lines up very closely with the NASA/GISS warming trend from 1960 to 2016, with my results as well as the NASA/GISS results showing record high temperatures for 2016. However, my results do show a visible warm-bias relative to the NASA/GISS results prior to 1950 or so. This is the basis of the accusations that NOAA and NASA “cooled the past (and warmed the present)” to exaggerate the global warming trend.

Now, why do my unadjusted data results show that pre-1950 “warm bias” relative to the NASA/GISS results? Well, this excerpt from NOAA’s GHCN FAQ provides some clues:
Why are there more cold (negative) step changes than warm(positive) step changes in the historical land surface air temperature records represented in the GHCN v3 dataset?

The reason for the larger number of cold step changes is not completely clear, but they may be due in part to systematic changes in station locations from city centers to cooler airport locations that occurred in many parts of the world from the 1930s to through the 1960s.
Because the GHCNv3 metadata contains an airport designator field for every temperature station, it was quite easy for me to modify my program to exclude all the “airport” stations from the computations. So let’s exclude all of the “airport” station data and see what we get. Figure 7 below shows my unadjusted data results vs. the NASA/GISS results when all “airport” stations are excluded from my computations.


Figure 7: Unadjusted Data, Airports Excluded (green) vs. NASA/GISS (red)

There is a very visible reduction in the bias between my unadjusted results and the NASA results (especially prior to 1950 or so) when airport stations are excluded from my unadjusted data processing. This is quite consistent with the notion that many of the stations currently located at airports were moved to their current locations from city centers at some point during their history.

Now just for fun, let’s look at what happens when we do the reverse and exclude non-airport stations (i.e. process only the airport stations). Figure 8 shows what we get when we process unadjusted data exclusively from “airport” stations.


Figure 8: Unadjusted Data, Airports Only (green) vs. NASA/GISS (red)

Well, look at that! The pre-1950 bias between my unadjusted data results and the NASA/GISS results really jumps out. And take note of another interesting thing about the plot -- in spite of the fact that I processed only “airport” stations, the green “airports only” temperature curve goes all the way back to 1880, decades prior to the existence of airplanes (or airports)! It is only reasonable to conclude that those “airport” stations must have been moved at some point in their history.

Now, for a bit more fun, let’s drill down a little further into the data and process only airport stations that also have temperature data records going back to 1903 (the year that the Wright Brothers first successfully flew an airplane) or earlier.

When I drilled down into the data, I found over 400 “airport” temperature stations with data going back to 1903 or earlier. And when I computed global-average temperature estimates from just those stations, this is what I got (Figure 9):


Figure 9: Unadjusted Data, Airport Stations with pre-1903 Data (green) vs. NASA/GISS (red)

OK, that looks pretty much like the previous temperature plot, except that my results are “noisier” due to the fact that I processed data from fewer temperature stations.

And for even more fun, let’s look at the results we get when we process data exclusively from non-airport stations with data going back to 1903 or earlier:


Figure 10: Unadjusted Data, Non-Airport Stations with pre-1903 Data (green) vs. NASA/GISS (red)

When only non-airport stations are processed, the pre-1950 “eyeball estimate” bias between my unadjusted data temperature curve and the NASA/GISS temperature curve is sharply reduced.

The results seen in the above plots are entirely consistent with the notion that the movement of large numbers of temperature stations from city centers to cooler outlying airport locations during the middle of the 20th Century is responsible for much of the bias seen between the unadjusted and adjusted GHCNv3 global-average temperature results.

It is quite reasonable to conclude, based on the results presented here, that one major reason for the bias seen between the GHCNv3 unadjusted and adjusted data results is the presence of corrections for those station moves in the adjusted data (corrections that are obviously absent from the unadjusted data). Those corrections remove the contaminating effects of station moves and permit more accurate estimates of global surface temperature increases over time.

Take-home lessons (in no particular order):

  1. Even a very simple global temperature algorithm can reproduce the NASA/GISS results very closely. This really is a case where you can get 98% of the answer (per my R-squared statistic) with less than 1% of the effort.
  2. NOAA’s GHCNv3 monthly data repository contains everything an independent “citizen scientist” needs (data and documentation) to conduct his/her own investigation of the global land station temperature data.
  3. A direct comparison of unadjusted data results (all GHCN stations) vs. the NASA/GISS adjusted data temperature curves reveals only modest differences between the two temperature curves, especially for the past 6 decades. Furthermore, my unadjusted and the NASA/GISS adjusted results show nearly identical (and record) temperatures for 2016. If NASA and NOAA were adjusting data to exaggerate the amount of planetary warming, they sure went to an awful lot of trouble and effort to produce only a small overall increase in warming in the land station data.
  4. Eliminating all “airport” stations from the processing significantly reduced the bias between my unadjusted data results and the NASA/GISS results. It is therefore reasonable to conclude that a large share of the modest bias between my GHCN v3 unadjusted results and the NASA/GISS adjusted data results is the result of corrections for station moves from urban centers to outlying airports (corrections present in the adjusted data, but not in the unadjusted data).
  5. Simply excluding “airport” stations likely eliminates many stations that were always located at airports (and never moved) and also fails to eliminate stations that were moved out from city centers to non-airport locations. So it is not a comprehensive evaluation of the impacts of station moves. However, it is a very easy “first step” analysis exercise to perform; even this incomplete “first step” analysis produces results that strongly consistent with the hypothesis that corrections for station moves are likely the dominant reason for the pre-1950 bias seen between the adjusted and unadjusted GHCN global temperature results. Remember that many urban stations were also moved from city centers to non-airport locations during the mid-20th century. Unfortunately, those station moves are not recorded in the simple summary metadata files supplied with the GHCNv3 monthly data. An analysis of NOAA’s more detailed metadata would be required to identify those stations and perform a more complete analysis of the impacts of station moves. However, that is outside of the scope of this simple project.
  6. For someone who has the requisite math and programming skills, confirming the results presented here should not be very hard at all. Skeptics should try it some time. Provided that those skeptics are willing and able to accept results that contradict their original views about temperature data adjustments, they could have a lot of fun taking on a project like this.

Related reading

Also the Clear Climate Code project was able to reproduce the results of NASA-GISS. Berkeley Earth made an high-level independent analysis and confirmed previous results. Also (non-climate) scientist Nick Stokes (Moyhu) computed his own temperature signal: TempLS which also fits well.

In 2010 Zeke Hausfather analyzed the differences in GHCNv2 between airport and other stations and found only minimal differences: Airports and the land temperature record.

At about the same time David Jones at Clear Climate Code also looked at airport station, just splitting the dataset in two groups, and did found differences: Airport Warming. Thus making sure both groups are regionally comparable is probably important.

The global warming conspiracy would be huge. Not only the 7 global datasets also national datasets from so many groups show clear warming.

Just the facts, homogenization adjustments reduce global warming.

Why raw temperatures show too little global warming.

Irrigation and paint as reasons for a cooling bias.

Temperature trend biases due to urbanization and siting quality changes.

Temperature bias from the village heat island

Cooling moves of urban stations. From cities to airports or simply to outside a city or village.

The transition to automatic weather stations. We’d better study it now. It may be a cooling bias.

Changes in screen design leading to temperature trend biases.

Early global warming

Cranberry picking short-term temperature trends

How climatology treats sceptics

Sunday, January 8, 2017

Much ado about NOAAthing


I know NOAAthing.

This post is about nothing. Nearly nothing. But when I found this title I had to write it.

Once upon a time in America there were some political activists who claimed that global warming had stopped. These were the moderate voices, with many people in this movement saying that an ice age is just around the corner. Others said global warming paused, hiatused or slowed down. I feel that good statistics has always shown this idea to be complete rubbish (Foster and Abraham, 2015; Lewandowsky et al., 2016), but at least in 2017 it should be clear that it is nothing, nothing what so ever. It is interpreting noise. More kindly: interpreting variability, mostly El Nino variability.

Even if you disingenuously cherry-pick 1998 the hot El Nino year as the first year of your trend to get a smaller trend, the short-term trend is about the same size as the long-term trend now that 2016 is another hot El Nino year to balance out the first crime. Zeke Hausfather tweeted to the graph below: "You keep using that word, "pause". I do not think it means what you think it means." #CulturalReference



In 2013 Boyin Huang of NOAA and his colleagues created an improved sea surface dataset called ERSST.v4. No one cared about this new analysis. Normal good science.




Thomas Karl of NOAA and his colleagues showed what the update means for the global temperature (ocean and land). The interesting part is the lower panel. It shows that the adjustments make global warming smaller by about 0.2°C. Climate data scientists naturally knew this and I blogged about his before, but I think the Karl paper was the first time this was shown in the scientific literature. (The adjustments are normally shown for the individual land or ocean datasets.)

But this post is unfortunately about nearly nothing, about the minimal changes in the top panel of the graph below. I made the graph extra large, so that you can see the differences. The thick black line shows the new assessment (ERSST.v4) and the thin red line the previous estimated global temperature signal (ERSST.v3). Differences are mostly less than 0.05°C, both warmer and cooler. The "problem" is the minute change at the right end of the curves.



The new paper by Zeke Hausfather and colleagues now shows evidence that the updated dataset (ERSSTv4) is indeed better than the previous version (ERSSTv3b). It is a beautifully done study of high technical quality. They do so by comparing the ERSST dataset, which comes from a large number of data sources, with  data that comes only from only one source (buoys, satellites (CCl) or ARGO). These single-source datasets are shorter, but without trend uncertainties due to the combination of sources.



The recent trend of HadSST also seems to be too small and to a lesser amount also COBE-SST. This problem with HadSST was known, but not published yet. The warm bias of ships that measure SST at their engine room intake is getting smaller over the last decade. The reason for this is not yet clear. The main contender seems to be that the fleet has become more actively managed and (typically warm) bad measurements have been discontinued.

Also ERSST uses ship data, but it gives them a much smaller weight compared to the buoy data. That makes this problem less visible in ERSST. Prepare for a small warming update for recent temperatures once this problem is better understood and corrected for. And prepare for the predictable cries of the mitigation skeptical movement and their political puppets.



Karl and colleagues showed that as a consequence of the minimal changes in ERSST and if you start a trend in 1998 and compute a trend, this trend is statistically significant. In the graph below you can see in the left global panel that the old version of ERSST (circles) had a 90% confidence interval (vertical line) that includes zero (not statistically significantly different from zero), while the confidence interval of updated dataset did not (statistically significant).



Did I mention that such a cherry-picked begin year is a very bad idea? The right statistical test is one for a trend change at an unknown year. This test provides no evidence whatsoever for a recent trend change.

That the trend in Karl and colleagues was statistically significant should thus not have mattered: Nothing could be worse than define a "hiatus" period as one were the confidence interval of a trend includes zero. However, this is the definition public speaker Christopher Monckton uses for his blog posts at Watts Up With That, a large blog of the mitigation skeptical movement. Short-term trends are very uncertain, their uncertainty increases very fast the shorter the period is. Thus if your period is short enough, you will find a trend whose confidence interval includes zero.

You should not do this kind of statistical test in the first place because of the inevitable cherry picking of the period, but if you want to statistically test whether the long-term trend suddenly dropped, the test should have the long-term trend as null-hypothesis. This is the 21st century, we understand the physics of man-made global warming, we know it should be warming, it would be enormously surprising and without any explanation if "global warming had stopped". Thus continued warming is the thing that should be disproven, not a flat trend line. Good luck doing so for such short periods given how enormously uncertain short-term trends are.



The large uncertainty also means that cherry picking a specific period to get a low trend has a large impact. I will show this numerically in an upcoming post. The methods to compute a confidence interval are for a randomly selected period, not for a period that was selected to have a low trend.

Concluding, we have something that does not exist, but which was made into an major talking point of the mitigation skeptical movement. This movement put their credibility on fluctuations that produced a minor short-term trend change that was not statistically significant. The deviation was also so small that it put an unfounded confidence in the perfection of the data.

The inevitable happened and small corrections needed to be made to the data. After this even disingenuous cherry-picking and bad statistics were no longer enough to support the talking point. As a consequence Lamar Smith of TX21 abused his Washington power to punish politically inconvenient science. Science that was confirmed this week. This should all have been politically irrelevant because the statistics were wrong all along. This was politically irrelevant by now because the new El Nino produced record temperatures in 2016 and even cherry picking 1998 as begin year is no longer enough.


"Much Ado About Nothing is generally considered one of Shakespeare's best comedies because it combines elements of mistaken identities, love, robust hilarity with more serious meditations on honour, shame, and court politics."
Yes, I get my culture from Wikipedia)


To end on a positive note, if your are interested in sea surface temperature and its uncertainties, we just published a review paper in the Bulletin of the American Meteorological Society: "A call for new approaches to quantifying biases in observations of sea-surface temperature." This focuses on ideas for future research and how the SST community can make it easier for others to join the field and work on improving the data.

Another good review paper on the quality of SST observations is: "Effects of instrumentation changes on sea surface temperature measured in situ" and also the homepage of HadSST is quite informative. For more information on the three main sea surface temperature datasets follow these links: ERSSTv4, HadSST3 and COBE-SST. Thanks to John Kennedy for suggesting the links in this paragraph.

Do watch the clear video below where Zeke Hausfather explains the study and why he thinks recent ocean warming used to be underestimated.





Related reading

The op-ed by the authors Kevin Cowtan and Zeke Hausfather is probably the best article on the study: Political Investigation Is Not the Way to Scientific Truth. Independent replication is the key to verification; trolling through scientists' emails looking for out-of-context "gotcha" statements isn't.

Scott K. Johnson in Ars Technica (a reading recommendation for science geeks by itself): New analysis shows Lamar Smith’s accusations on climate data are wrong. It wasn't a political plot—temperatures really did get warmer.

Phil Plait (Bad Astronomy) naturally has a clear explanation of the study and the ensuing political harassment: New Study Confirms Sea Surface Temperatures Are Warming Faster Than Previously Thought

The take of the UK MetOffice, producers of HadSST, on the new study and the differences found for HadSST: The challenge of taking the temperature of the world’s oceans

Hotwhopper is your explainer if you like your stories with a little snark: The winner is NOAA - for global sea surface temperature

Hotwhopper follow-up: Dumb as: Anthony Watts complains Hausfather17 authors didn't use FUTURE data. With such a response to the study it is unreasonable to complain about snark in the response.

The Christian Science Monitor gives a good non-technical summary: Debunking the myth of climate change 'hiatus': Where did it come from?

I guess it is hard for a journalist to not write that the topic is not important. Chris Mooney at the Washington Post claims Karl and colleagues is important: NOAA challenged the global warming ‘pause.’ Now new research says the agency was right.

Climate Denial Crock of the Week with Peter Sinclair: New Study Shows (Again): Deniers Wrong, NOAA Scientists Right. Quotes from several articles and has good explainer videos.

Global Warming ‘Hiatus’ Wasn’t, Second Study Confirms

The guardian blog by John Abraham: New study confirms NOAA finding of faster global warming

Atmospheric warming hiatus: The peculiar debate about the 2% of the 2%

No! Ah! Part II. The return of the uncertainty monster

How can the pause be both ‘false’ and caused by something?

References

Grant Foster and John Abraham, 2015: Lack of evidence for a slowdown in global temperature. US CLIVAR Variations, Summer 2015, 13, No. 3.

Zeke Hausfather, Kevin Cowtan, David C. Clarke, Peter Jacobs, Mark Richardson, Robert Rohde, 2017: Assessing recent warming using instrumentally homogeneous sea surface temperature records. Science Advances, 04 Jan 2017.

Boyin Huang, Viva F. Banzon, Eric Freeman, Jay Lawrimore, Wei Liu, Thomas C. Peterson, Thomas M. Smith, Peter W. Thorne, Scott D. Woodruff, and Huai-Min Zhang, 2015: Extended Reconstructed Sea Surface Temperature Version 4 (ERSST.v4). Part I: Upgrades and Intercomparisons. Journal Climate, 28, pp. 911–930, doi: 10.1175/JCLI-D-14-00006.1.

Thomas R. Karl, Anthony Arguez, Boyin Huang, Jay H. Lawrimore, James R. McMahon, Matthew J. Menne, Thomas C. Peterson, Russell S. Vose, Huai-Min Zhang, 2015: Possible artifacts of data biases in the recent global surface warming hiatus. Science. doi: 10.1126/science.aaa5632.

Lewandowsky, S., J. Risbey, and N. Oreskes, 2016: The “Pause” in Global Warming: Turning a Routine Fluctuation into a Problem for Science. Bull. Amer. Meteor. Soc., 97, 723–733, doi: 10.1175/BAMS-D-14-00106.1.