Sunday, 27 April 2014

Are debatable scientific questions debatable?

In a previous post I tried to make clear why debates are such a bad way to improve scientific understanding. In the comments Mark Ryan pointed to a great article by John Ziman: "Are debatable scientific questions debatable?" In the article, political debates are compared to scientific disputes. Ziman seem to agree with me: a short summary of his position could be that the only similarity between political debates and scientific disputes is that they are both public.

The main difference is that while debates are typically verbal, scientific disputes are resolved in the scientific literature. That makes a large difference. In the scientific literature the ideas have to be and can be presented in all their gory details. Detail that allows one so see its errors or improve upon the idea. A written dialogue is also slow, which helps to check details and to rethink ones position multiple times before answering.

Debate

In the article a distinction is made between political debates and scientific disputes.
I shall use this word ‘debate’ as a term of art for an argumentation ritual defined by the following features:

* A debate is a one-off, public event, initiated and completed within a limited period of time. Typically, it is conducted in a forum before an audience, most of whom take no direct part in the proceedings and may be quite ill-informed on the topic. ...
* A debate is conducted orally, in direct speech, by named participants. ...
* A debate has a specific topic, in the form of a ‘black or white’ proposition. This is typically understood to be a ‘question’ to be answered ‘yes’ or ‘no’ ...
* The argumentation is polarized, and balanced ritually between a protagonist and an antagonist, (‘proponent’ vs ‘opponent’, ‘plaintiff’ vs ‘defendant’ , etc.) ...
* The proceedings are adversarial, in that each ‘party’ endeavours not only to make their own case but also to negate the case of their opponents, even to the extent of attacking their credibility as expert advocates or cross-examining them severely on disputable points.
* A debate is normally conducted by a chair (or ‘speaker’), who simply ensures that the rules are observed. ...
* The debate almost always concludes with a decision — i.e. a ‘yes’ or ‘no’ answer to the question— determined typically by a simple majority of those who have (supposedly!) attended to the debate and are qualified to vote on such matters. Sometimes this includes the whole audience ...
A debate is about winning and convincing, it is not about improving our understanding of reality, of the problem at hand, not even of the position of the other party. As it is a one off event, presenting misinformation is an effective strategy, especially if you have no reputation to lose, like the climate pseudo-sketpics. Another effective debating strategy if you only want to win is a Gish Gallop, after the strategy of the creationist Duane Gish, to spew such a gallop of misinformation, that the other side would not have enough time to correct all the falsehoods.

Scientific dispute

The rules of the game are very different in science.
"Scientific disputation ... typically involves sequences of written arguments and counter-arguments in the official scientific 'literature', rather than oral exchanges in a public forum."
Scientists naturally do talk with each other a lot, but what counts is the literature. There are presentations at conferences and workshops, fitting to scientific presentations not being that important "a remarkably low level of verbal fluency is acceptable. ... The conventional presumption, emphasized by stock courtesies from the chair, is that the theme of the paper [i.e. presentation] is worth of serious attention, even if, at first hearing, it seems incomprehensible, wildly speculative, inconsistent, totally boring or otherwise unconvincing or uninteresting!" This statement is funny because it is so true. Not uncommonly you have to make quite an effort to understand the speaker and often a talk is mainly useful as an announcement of a new article.
Scientific papers ... [do not] avoid controversial statements. On the contrary, one of the norms of science is that a would-be contribution to knowledge must always be 'original'.
I am not sure if I fully agree with this. While scientific papers do not avoid controversy, they also do not aim for it. Papers should be original. Original and controversial are different. You can have a new way to compute something already known, you could analyze a new dataset about which no one had a preexisting opinion, etc.

I had to think some time to find topics under scientific dispute. That presupposes that two people or groups hold a strong opinion about something without sufficient evidence. It is more common that scientists keep all options open until the evidence is there. A scientist may even use two conflicting theories depending on circumstances, such as relativity and quantum mechanics. "Scientists learn by experience to hold simultaneously in mind a number of uncertain, perhaps inconsistent, ideas, without begin paralysed by logical gridlock or complete scepticism." Also if a paper changes our current understanding, if the evidence is strong, there is no reason for dispute, it is simply accepted. Or as Ziman writes:
Ideally, [innovative ideas] are presented as if they were already part of the (thus newly-achieved) consensus, in a form that is so unquestionable that it can be treated as camera-ready copy for a new chapter in the next generation of text-books!
The only scientific dispute in my surrounding that is not related to the political climate debate, is whether station measurements contain platform-like pairs of break inhomogeneities of short duration. More precisely, whether this is more common as one would expect from random breaks. That this became a dispute was because a decision with a deadline had to be made whether to include them in our HOME validation dataset for homogenization methods. Doing both was unfortunately no option because of man power. The need for a decision, like in politics, did not allow us to withhold opinion until enough evidence was gathered. Fortunately, it was a friendly dispute as no one expects these platforms to be too important. The other two disputes I have experienced, long memory in temperature series and naturally the so-called hiatus, I would view as intrusions of the political debate into science.

Another ritual is that "every paper starts with a very conventional 'survey of the field', including a lengthy bibliography that cites all the usual suspects. ... It can be interpreted functionally as a rhetorical device designed to show that the author is so completely at home in the field that their novel ideas are worthy of attention." It also helps the peer reviewers to judge the novelty of the work and avoids double work by forcing the researcher to be up to date. Its main role for scientific progress is that it aids think neutrally, because it forces the author to write about the work of others in an objective way. "Every author is well aware that one of the reviewers may support another side in such a controversy and is likely to turn down a paper that does not present fairly the case of that side." If that is the only peeve, the reviewer will more likely ask for a revision, rather than immediately request rejection.

The questions also have a different nature. A debate, in a formal sense, is normally about the acceptability of a specific proposition, but scientific controversies are seldom that simple. ... In reality, scientific progress does not follow Popper's recipe of successive theoretical conjectures being winnowed away by empirical refutations. As other philosophers have pointed out, there are always just too many other untested assumptions that might be the cause of the apparent disconfirmation. While I do not think that Karl Popper was that naive, it is clearly true that if a discrepancy is seen in science, it is not clear in advance where the problem lies. Finding that is the largest part of solving the problem.

Further differences are:
* The audience is assumed to be already very well-informed on the topic;
* The proceedings are not adversarial, as between officially identified protagonists and antagonists;
* Attacks on the personal motivation or credibility of the disputants are unacceptable;
* The discussion does not close with a formal decision or verdict.
Concluding.
Conventional 'debating' practises just do not fit into the evolution of scientific knowledge in its traditional academic mode. In effect, 'debatable' scientific issues are never actually 'debated'. One must therefor seriously ask whether much of value can be achieved by a formal procedure designed to bring scientists together for just such an activity.
All the more so for bringing scientists and anti-scientists together. It will certainly not help scientific progress. I fail to see how it would help in communicating our current understanding to the public. A documentary seems much better suited for that.



Highly recommended reading

John Ziman: "Are debatable scientific questions debatable?"

[UPDATE. WhatsUpWithThatWatts wrote about the same topic today. The street fighter version, so to speak: Why do Global Warming "Skeptics" Reject Honest Debate?]

11 comments:

Mark Ryan said...

Hi Victor, thanks for another interesting post on such an important topic.

I don't think we can ever say too much towards this question of how scientific knowledge is actually built. So many people nowadays call for more public involvement in science -but as this article suggests, public methods of debate are really quite different to the ways scientists work knowledge out.

In several conversations I've had with ecologists and environmental scientists, they've conveyed how uncomfortable they are about debating science in the way it takes place in public forums. It isn't just that they shy away from the spotlight, it's also that they feel kind of compromised by what seems like a degenerate way of deliberating.

This is very tricky, because of course the public has no patience for any group that says 'this is not how we talk about our work'-the crude mob mentality just caricatures you as ivory-tower elitists. Scientists who can function in the public forum are rare -but do they also feel conflicted?

How about you Victor? What was it made you create this blog, and what are your ambitions for it?

And if I may ask another personal question, how do your peers feel about engaging the public in forums like this?

citizenschallenge said...

Victor, thank you for taking the time to describe scientific dialogue - as opposed to political debate which does more to confuse than clarify.

Your description of the scientific process in action is clear and understandable - I hope many folks will read this.

Victor Venema said...

I do not think scientists are afraid to the spotlight. If you are a successful scientist, it can happen that you have to speak to audiences with hundreds of people. But hundreds of people who are interested and want to understand you in the hope that it helps them understand the world or science a little better.

Like I wrote in this post, I think the aversion these people have against "debates" is fully warranted. Debates are scientifically completely dysfunctional.

Blogging is already a lot different from debating. More like giving a presentation and hoping for good questions in the Q&A afterwards. Furthermore, blogging gives you the possibility to think and to check and find information before your answer. Still if science only had blogs, scientific progress would likely come to a halt.

Science takes place in the scientific literature. (Citizen scientists are welcome to contribute to the literature.) I understand that this will be abused by the enemies of science by claiming it is elitist, but I see no way around that. There are too little scientists, that everyone could see that also scientists are normal people that go shopping, do their dishes and get colds.

Before I had this blog, I sometimes wrote essays on my homepage. Writing helps me think, when you write you notice the holes in your understanding better than when you are just staring at the clouds. And by writing for an audience, you are forced to make a strong case. I changed to blogging in the hope of getting more feedback and good ideas in the comments. Your literature tips are a good example, that that works. Thanks a lot!

And it was also a way to write up stuff that I found interesting, but knew I would not have to the time for to convert it into science. Some people are very efficient and have trouble generating enough ideas, I generate more ideas as I can work on. By putting it out there, I was hoping that maybe someone else would pick it up. That has not happened as far as I can judge. People like their own ideas more. At least they do not get lost due to my bad memory.

Then after the Watts et al. manuscript, people started reading my blog and I also started writing more informative posts. I feel it would be sad if people would knowingly caused a mass extinction, but if people know what they are doing and want it that way, so be it. An open democratic society is an even higher value for me. However, what would be horrible, if we react in the wrong way due to the misinformation campaign. I became a scientist because I love to understand how things work and I like rational thought. Thus nowadays also fighting the forces of irrationality from the Middle Ages is an important motivation.

Victor Venema said...

Ambitions with blogging? :) Not much. Just enjoying that more and more people are reading. It is a good feeling that the readers are apparently enjoying it and that I can contribute something back to society that gave me so much. When people started reading, I had hoped a little that it would improve my visibility in the scientific community and help spread my ideas, which every scientist things are worthwhile ;-), and maybe get more citations to my articles, which helps in becoming more independent and to work on larger projects. However, I have the feeling that blogs hardly exist in the scientific community. There is only one colleague I know that regularly reads my blog.

But maybe I am too pessimistic and is this also a matter of time. I just got an invitation from some econometricians (probably climate "sceptics" if I may have my prejudices). They want to start working on homogenization of climate data and had read my series on 5 statistically interesting problems in homogenization. Furthermore, I could mention my blog as Outreach in two recent research proposals. So maybe it does help a little professionally.

My peers typically have no idea what I am doing. They might know blogs from the newspapers, if at all. When I tell them about debunking pseudo-skeptics, they typically think that that is a weird thing to do and not something a scientist should do. They assume that people calling themselves skeptics are at least a little bit skeptical and thus lovable and should be supported and not debunked. One would have to force them to read WUWT for a week to get that idea out of their head. But if I would force a scientist to read WUWT, I would probably get into trouble with Amnesty International.

Those are the comments I am not supposed to make. ;-) I am supposed to follow the rules of scientific disputes. Thus, I would guess that all over, blogging about the pseudo-skeptics hurts my career, because of the conflict between nature of the debate and the scientific rules of conduct. But it probably only hurts a little as no one pays attention. And you only have one life, have to do what is important and there is more to life than just a career.

Thanks citizenschallenge, good to hear the post was clear to a street fighter. :-)

Mark Ryan said...

"But if I would force a scientist to read WUWT, I would probably get into trouble with Amnesty International." :)

Mark Ryan said...

Of course, your blog is appreciated.

Thanks for sharing some of your experiences too, Victor. I've had a growing feeling for some time that one of the things we need to do better is communicate what climate scientists are really like. From outside it is easy to project onto the scientific community -harder if the particular cultures of scientists are better understood.

It's fascinating that some of your peers are so removed from the political conflicts over ACC that they have innocent assumptions about the so-called 'skeptics'. I assumed anyone working in climate these days feels like they were under a political microscope!
The scientists I have spoken to are perhaps already following the politics, hence their willingness to chat with me.

Victor Venema said...

An important difference is that I am Dutch and work in Germany. I guess that the climate "debate" is more or less limited to the Anglo-Saxon countries. Before I discovered WUWT, I was also working in innocent bliss. I knew that something like that existed and that like creationism was a problem in the USA, but had no idea how bad it was.

Most of the climatologists are working on topics that are not of interest to or to complicated for the climate pseudo-skeptics. That also reduces contact. I have some posts on the quality of daily climate data, which would theoretically be of interest to them. After one of my talks on this a colleague asked whether I was a climate skeptic. :) No I am not, I have evidence to back up my claims. But either they do not get it or do not want to link to a blog that thinks that Wattists are fools.

That is also why the pseudo-skeptics manage to think that The Team is in control of climatology. They just know a few scientists and cannot imagine how impossible it would be for a few people to be in control of so many scientists, many of them only working partially on the topic, who are rooted in other disciplines. The bottom up organization of science is another post I dearly should write.

Mark Ryan said...

So, that doesn't sound much like Richard Lindzen's characterisation of the domain of climatology as pervaded by “…a willingness to debase climate science into a triangle of alarmism. Ambiguous scientific statements about climate are hyped by those with a vested interest in alarm, thus raising the political stakes for policy makers who provide funds for more science research to feed more alarm to increase the political stakes."

Maybe it's only in the US that climate scientists are all getting rich on padded grants, and attracting hot young environmentalists...:)

citizenschallenge said...

Hello Victor, I've reposted this article
Thanks for another excellent read.

"scientific debate vs political debate, Victor Venema"
http://citizenschallenge.blogspot.com/2014/04/venema-debatable-debates.html

=================
"Ambitions with blogging? :) " ???

Would it be fair to say a part of the "need" to blog is because it's too painful to standby doing nothing while that relentless attack on rational science keep getting more shrill and disconnected from reality?

EliRabett said...

Again Victor thanks for your blog and POV.

However, and there always is one, there is an oral debate and it happens at conferences, often around a talk, in coffee rooms before and after a seminar, but even then, the talk is only a start to the oral conversations and exchanges of informal notes (Email today) which continue long after. The written exchange is, if you wish, a summary of the oral one with the kinks (hopefully) worked out.

Victor Venema said...

The Q&A after a conference talk, chats at the water cooler and jointly working on a study involve talking, but they have a fully different purpose and format as a debate. People are not out to win, sometimes they may want to impress, but mostly they want to understand.

I would see a scientific article as much more than a summary of a conversation. Already because it normally contains more detail than a conversation. It is the heard of science.