Peer review helps fringe ideas gain credibility

Stoat has a nice new post explaining how peer review works in practice. The climate ostriches sometimes suggest that peer review is just there to keep their ideas out of the scientific literature and derogatively call it pal review. Well, peer review is nowadays practiced in all sciences, so that seems rather far fetched. Thinking about it, I would argue that peer review could actually help the climate ostriches. There is one caveat: they would have a valid critique. That is the bigger problem for them.

So what is peer review, what is its function in science and how could it help the climate ostriches?

How does it work

The short description of peer review at Stoat is:

For a working scientist, peer review is just part of the job. You write up your work, you show it to your colleagues ..., you send it to the best journal you think you can get away with, and eventually you get the reviews back. These will be a mixture of “please cite my paper” (usually disguised as “you need to consider X”), typos, and the occasional well-considered thoughtful comment that genuinely improves things. You sigh, you happily incorporate the thoughtful stuff, you work out how much of the not-very-helpful stuff you can get away with blowing off, and you resubmit ... And sometimes you get a reviewer who really really doesn’t like your paper for what you regard as invalid reasons, and you have to decide whether to fight to the death or go elsewhere.

Function in science

In my previous post, The value of peer review for science and the press, I wrote about its function in science:

Peer review gives an article credibility. As such peer review is "just" a filter, it does not guarantee that an article is right. Many peer-reviewed articles contain errors, many ideas outside of the peer-reviewed literature are worthwhile. However, on average the quality of peer-reviewed work is better. Thus peer-reviewed work is more likely worthy of your attention. If you are a scientist and an idea/study is about something you are knowledgeable about there is no reason to limit yourself exclusively to peer-reviewed articles, but it is smart to prefer them.

Given that an important function of peer review is to give the article credibility, it is also logical that reviewers pay extra attention if an article makes strong claims, that is claims that clearly deviate from our current understanding. In an ideal world, without any time pressures, peer review would be perfect every instance. However, a run of the mill article by a well-known author is much less likely to contain problems.

I would argue that that should be no problem for the people making strong claims. Every claim should hold up the scrutiny of the reviewers any way. That may be more work, but such an article also brings much more acclaim and is thus worth some work. As quoted above, sometimes reviewers will block your beautiful manuscript. That has also happened to me and probably to any active scientist. Then you just go to another journal. If the idea is valid, you will find a place for it.

That peer review is not perfect may be more of a problem for seasoned scientists. I especially notice that well-written article, by native speakers, are more likely to contain small errors. The smooth language seems to make the reviewers less critical and the author is punished by publishing articles with embarrassing errors.

Pattern Recognition in Physics

The discussion around peer review started again because the publisher Copernicus recently closed the journal Pattern Recognition in Physics. The worst problem with the journal was in my view that they hardly published anything about pattern recognition, which is a machine learning (statistical) approach and not the same as a scientist studying a dataset and recognizing an empirical pattern.

Another reason to close the journal was that the peer review was not performed by independent scientists. Given that the editorial board and many authors were climate ostriches, it would have been important that the reviews were performed by main stream scientists. That would have given the articles a minimum of credibility. The idea of Lord Monckton to become the new publisher of Pattern Recognition in Physics is thus ludicrous. A journal could hardly start with a stronger credibility handicap than having such a publisher. It is not for nothing that the climate ostriches approached Copernicus to be the publisher of their journal. That gave them some credibility.

I am sorry, the situation is not symmetrical. Also for a main stream journal or article, the reviewers should be independent, but the need is much less strong, if only because most article just provide some incremental information.

Help for fringe ideas

I would argue that peer review is actually most important for outsiders. First of all, it forces the peer reviewers to study the manuscript and in this way brings fringe ideas to their attention that may otherwise have been ignored. The most important reason is, however, that the fringe idea gains some initial credibility due to the peer review. Consequently, the reader is much more likely to take notice and to take a fringe idea in a peer reviewed journal serious as the reader would if it were written in a blog post.

From my own experience. Some months ago, WUWT has a post about what was wrong with temperature measurements (again). At least it was not written by Watts himself and it contained some new ideas. It also contained some obvious errors, the standard well-known often-repeated ostrich memes. In the end, I did not check the ideas that were not a-priory wrong. My experience with WUWT tells me that when I do check the facts, the argument is usually false. Thus if you are busy and you still have a pile of papers from credible sources on your desk, from which you are more likely to learn something important, you skip the WUWT post and forget about it.

Had this person not published at WUWT, but put in the effort to make a solid study out of his evidence, had he submitted a manuscript to a peer reviewed journal and had the reviewers noticed the obvious problematic arguments and had this resulted in somewhat credible peer-reviewed publication, I would almost surely have read it. What is more interesting as a challenge to your current understanding? That is the power of credibility and of peer review. (That is also why I would advice all climate ostriches not to put up with nonsense at WUWT and Co. Every single error on your blogs hurts the credibility of your cause.)

Another good example is that Stephen Hawkins just wrote an new manuscript on black holes and posted it to Arxiv, an internet database for scientific manuscript. Scientists will read such a manuscript and will not wait until it is published. Even Nature immediately wrote of piece about it and the press directly noticed. Hawkins does not need peer review to give his paper some initial credibility, to suggest people that it is worthwhile to invest their time in trying to understand it. Hawkins does not need to borrow credibility from a scientific journal and its peer review. People on the fringe do need that help, peer review helps them. Next to filtering for quality, that is another way in which peer review helps scientific progress.

Thus all you need is a good idea, made rigorous in a strong manuscript and peer review. That is the way to gain credibility. Surely the intellectual heavyweights at WUWT & Co. have no lack of ideas, right. And surely they are burning with soul intensity to tell the world their truth, they will be willing to put in the work to make their ideas strong and rigorous. Thus the climate ostriches should cheer at peer review, it helps to get their good ideas known.

---

Related reading

The post of Stoat about how peer review works in practice is really worthwhile if you are not familiar with it, including its comments. WMC:"I find it fairly amusing that the majority of commenters here are able to say My experience of peer review has been... as opposed to the denial-o-sphere's fairy tales about what they imagine peer review is like."

Eli Rabett has an amusing story about peer review.

Peer review is not only important for science, but also for non-scientitst, which is what I wrote about in my previous post on the value of peer review.

Ed Wiebe responds to a proposal to replace peer review by social network voting. An important point he makes is that review is not just selection, but also feedback for improvement.