Wednesday, 24 June 2015

Overconfidence in the nut test


My apologies, yesterday I promoted an erroneous article on twitter. Science journalist Dan Vergano wrote about his simple nut test, based on the hallmark of almost any nut: overconfidence. Overconfidence is also common among mitigation sceptics, who are quick to shout fraud, rather than first trying to understanding the science and ask polite specific questions trying to clear any misunderstandings.

Thus when Vergano explained his nut test, I accepted that as easy as the word of God is accepted by an elder, as the Dutch say. A typical case of confirmation bias. He writes:

"A decade after my first climate science epiphany, I was interviewing a chronic critic of global warming studies, particularly the 1998 “hockey stick” one that found temperatures in our century racing upward on a slope that mirrored a hockey blade pointed skyward. He argued vociferously that the study’s math was all messed up, and that this meant all of climate science was a sham.

I listened, and at the end of the interview, I gave him the nut test.

“What are the odds that you are wrong?” I asked, or so I remember.

“I’d say zero,” the critic replied. “No chance.”

That’s how you fail the nut test.

I had asked a climate scientist the same question on the phone an hour before.

“I could always be wrong,” the scientist said. Statistically, he added, it could be about a 20% to 5% chance, depending on what he might be wrong about.

That’s how you pass the nut test: by admitting you could be wrong.

And that’s how a climate denier finally convinced me, once and for all, that climate science was on pretty safe ground.
"

The problem of the test is, it is possible to be confident that a scientific statement is wrong. A scientific hypothesis should be falsifiable. One should mainly not be too confident that one is right. Making of positive claim about reality is always risky.



For example, you can be confident that someone cannot claim that all of climate science is a sham after studying temperature changes in the distant past ("hockey stick"). That is a logical fallacy. The theory of global warming is not only based on the hockey stick, but also on our physical understanding of radiative transfer and the atmosphere and on our understanding of the atmospheres of the other planets and on global climate models. Science is confident it is warming not only because of the hockey stick, but also because of historical temperature measurements, other changes in the climate (precipitation, circulation), changes in ecosystems, warming of lakes and rivers, decreases of the snow cover and so on.

So in this case, the mitigation sceptic is talking nonsense, but theoretically it would have been possible that he was rightly confident that the maths was wrong. Just like I am confident that many of the claims on WUWT & Co on homogenization are wrong. That does not mean that I am confident the data is flawless, but just that you should not get your science from WUWT & Co.

Three years ago Anthony Watts, host of WUWT, called a conference contribution a peer reviewed article. I am confident that that is wrong. The abstract claimed without arguments that half of the homogenization should go up and half should go down. I am confident that that assumption is wrong. The conference contribution offered a span of possible values. Anthony Watts put the worst extreme in his headline. That is wrong. Now after three years with no follow-up it is clear that the authors accept that the conference contribution contained serious problems.

Anthony Watts corrected his post and admitted that the conference contribution was not a peer reviewed article. This is rare and the other errors remain. Next to overconfidence, not admitting to be wrong is also common among mitigation sceptics. Anyone who is truly sceptical and follows the climate "debate", please pay attention, when a mitigation sceptic loses an argument, he ignores this and moves to the next try. This is so common that one of my main tips on debating mitigation sceptics is to make sure you stay on topic and point out to the reader when the mitigation sceptic tries to change the topic. (And the reader is the person you want to convince.)

Not being able to admit mistakes is human, but also a sure way to end up with a completely wrong view of the world. That may explain this tendency of the mitigation sceptics. It is also possible that the mitigation sceptic knows from the start that his argument is bogus, but hopes to confuse the public. Then it is better not to admit to be wrong, because then this mitigation sceptic runs the risk of being reminded of that the next time he tries his scam.

Less common, but also important is the second order nut test for people who promote obvious nonsense, claim not to know who is right to give the impression that there is more uncertainty than there really is. Someone claiming to have doubts about a large number of solid results is a clear warning light. One the above mitigation sceptic is apparently also guilty of ("chronic critic"). It needs a lot of expertise to find problems, it is not likely that some average bloke or even some average scientist pulls this off.

Not wanting to look like a nut, I make an explicit effort to not only talk about what we are sure about (it is warming, it is us, it will continue if we keep on using fossil fuels), but also what we are nor sure about (the temperature change up to the last tenth of a degree). To distinguish myself from the nuts, I try to apologize even when this is not strictly necessary. In this case even when the nut test is quite useful and the above conclusions were probably right.



Related reading

Science journalist Dan Vergano wrote a nice article article on his journey from conservative Catholic climate "sceptic" to someone who accepts the science (including the nut test): How I Came To Jesus On Global Warming.

The three year old conference contribution: Investigation of methods for hydroclimatic data homogenization.

Anthony Watts calls inhomogeneity in his web traffic a success.

Some ideas on how to talk with mitigation sceptics and some stories of people who managed to find their way back to reality.

Falsifiable and falsification in science. Falsifiable is essential. Falsification not that important nor straightforward.

8 comments:

David Sanger said...

Victor, I have long thought that not all of climate science is actually science as we usually think of it, like physics or chemistry. Part of climate science is more like geography, the discovery of what is actually out there (rather than the development of theories to explain and predict.)

Thus Amerigo Vespucci, and later Columbus, had a "theory of North America" and after some exploration, after people had actually been there and seen it with their own eyes, it turned out to be true. There really is a continent between the Atlantic and the Pacific. North America exists!

To be sure we may not know the exact boundaries of the new land. Early mapmakers incorporated all sorts of inaccuracies as they worked out the detailed coastline of the "new" continent. But even with imperfect maps of the time it was a futile and incorrect argument to say that North America did not exist.

Victor Venema said...

The simple model of the greenhouse effect, the basic 1-dimensional radiative transfer computations are quite close to physics. The equations are old and well-known, the empirical constants can be measured in the lab. The radiative transfer equation can be validated in experiments.

Fortunately, climate models do not predict much other temperature changes than the 1-dimensinal model. But when you get away from temperature changes due to greenhouse gases and study feedbacks and other changes in the climate system you get nearer to astronomy for the physical part and geography for the impacts. That is where it becomes every more likely that we get surprises. (I do not get why people think surprises are nice, they are what I fear the most about climate change, we are taking the planet outside of the range we understand somewhat well.)

We only have one Earth, thus we cannot make experiments with the full climate system. Only with specific processes that influence the climate. That makes for a large difference with an experimental science such as physics.

Torbjörn Larsson, OM said...

[I bookmarked this article as a reference on skepticism, it was useful.]

The "everything except physics is stamp collecting" is inconsequential as long as there is empirical testing. Which I would call an experiment, albeit a passive one, we aren't changing all the parameters.

I appreciate the idea that we have only one modern Earth, but I am interested in astrobiology and it is striking that, precisely as for geology and biology, the exploration of other planets and other eras rely on what we see here and vice versa. Trying to understand the climates of Hadean/Archean Earth (commonly said to be "another planet we have vital observations of"), Venus, Mars and Titan analogs, the hot Jupiters/Neptunes and the potential habitables out there has been cross-fertilizing the area.

And climate science has a theory, of greenhouse gases. (Initially because people studied the Venus analog, I think.) If the practice is complicated doesn't mean there will be poor predictivity. It can mean the predictivity is superior. I am reminded of biology with its theories of cells and evolution, respectively. In both cases the complexity makes for a few outstanding observations. Theobald developed a test so that he could show in Nature 2010 that life bottlenecked through a universal common ancestor population. (Most likely a relic from the emergence of life, but that is neither here nor there.) That is the best observation in all of science (seen as a bayesian likelihood ratio test), with a likelihood of > 10^2000 against multiple common ancestors (emergences), due to the combinatorial nature of phylogenies.

Who knows what climate science eventually can come up with, admitting that the complexity likely makes for a slow start as it did in biology? The "genome" of habitable planet's climates is out there. Tying global climate together is a forceful accomplishment too from the viewpoint of predictively tying together disparate data points.

MPCraig said...

"what we are sure about (it is warming, it is us, it will continue if we keep on using fossil fuels)"

I agree with this with the caveat "all else being equal" But for the "it will continue" part, are you certain about future natural variability? What about the possible combination of changes of ocean circulation (e.g. AMO), changes of solar activity or any forcing like cosmic ray variation as we orbit Sagittarius A?

In other words, we have predicted a warming signal given emissions scenarios. How sure are we about future natural signals which additive and could augment or counteract?

I know I wouldn't be comfortable claiming I am sure it will continue to warm.

Victor Venema said...

Torbjörn Larsson, ecosystems are part of the climate system, but the rest of the climate system I would see as simpler than biological systems. Adaptation / evolution makes biological systems very complex and almost anything is possible. Physical systems drop down if you release them from your hand, a living system may fly away.

The big advantage evolution has is that it can study so many examples, while climatology only has one Earth. And the life sciences get much more funding. You can make money with their results, not only annoy governments and citizen with inconvenient results that they would rather not hear.

The complexity is what I find so interesting about the atmosphere and the climate system. Complexity both in the sense of "complicated" and in the sense of having many degrees of freedom. I have studied physics, but found its topics somewhat irrelevant, not inspiring to me; it looks like it is improving nowadays, physics is opening up to interesting complex system sciences.

Victor Venema said...

MPCraig, anything is possible, but if you are thinking of natural variability (not feedbacks to our warming), then it could also go the other way.

To offset global warming by greenhouse gasses it would have to cool the climate system (indefinitely) by several degrees. That seems to be very rare, we do not know of an analogue to that in the climates of the past.

Do you have a fire insurance? Are you 100% sure your house will burn down? That sounds like what you would require before taking an insurance against dangerous climate change. Do you buckle up driving on the highway?

EliRabett said...

Torbjörn Larsson the idea of a greenhouse effect goes back over 150 years to Fourier and Tyndall in the 1800s. The theory was ready and waiting when the first probes reached Venus. FWIW it was his work on Venus that dragged Jim Hansen into Earth modeling as colleagues repurposed his Venus model for Earth studies.

EliRabett said...

David Sanger: There are indeed two strands to climate studies, physics and observation. There is also a personal conflict between the two, with the physics part becoming increasingly dominant since the 1960s as computational capacity increased. The same thing has happened in chemistry