No, blog posts cannot replace scientific articles

Journalist Nate Silver of FiveThirtyEight proposes to replace peer reviewed scientific articles by blog posts. Well almost.

Proposed: cases of academic fraud would be caught sooner if academics mostly wrote blog posts instead of journal articles.

— Nate Silver (@NateSilver538) May 30, 2015

That would be a disaster for science, but somehow some twitterers apparently liked the idea. Maybe this is because the journalistic view and public view is on a minute portion of science. The tip of the iceberg could even be an understatement here. For example, together with Ralf Lindau I just have a nice paper out on how well we can determine the date of an abrupt non-climatic change in a temperature time series.

Lindau & @VariabilityBlog The uncertainty of break positions detected by homogenization algorithms in climate records http://t.co/tFTzyU5RBc

— O. Bothe (@geschichtenpost) May 19, 2015

The one "retweet" is me. The two "favourites" are two colleagues working on homogenization. Even I am wondering whether I should write a blog post about this paper and it certainly will not get into the press. That does not mean that it is not important for science, just that it is highly technical and that it will "just" help some scientists to better understand how to remove non-climatic changes. This will hopefully lead to better methods to remove non-climatic changes and finally to better assessments of the climatic changes. The latter study would be interesting to the public, all the studies it is based on, not so much. I may actually write a blog post about this paper, but then mainly explaining why it is important, rather than what is in the paper itself. The why could be interesting.

Of all the articles I have written only one was marginally interesting for the press. The paper were we showed that homogenization methods to remove non-climatic changes from station data work. The FAZ, a German conservative newspaper reported on it, ironically concluding: Climate change is not a measurement error. The University of Bonn is setting up a crisis helpline for mitigation sceptics. With that angle they could make sufficiently interesting for their news ticker. On the other hand, for the scientific community working on homogenization this was one of the main papers. For the scientific community the importance was the improvement in the validation methodology and the results that showed which kind of method works best, for the press this was already too much detail, they just reported that the methods work, which is nothing new for us.

Scientific literature

The science the public does not see, neither in the media nor on blogs, is also important for science. We will need a way to disseminate science that also works for the other 99.9% of science. Blogs and "blog review" won't do for this part.

Peer review is sometimes seen as gate keeping, but peer review actually helps the underdogs with fringe ideas. Helps ideas that otherwise would not be taken seriously. Helps ideas in which people would otherwise not be willing to invest their time to check it and see how they could build on it. And it saves a lot of time that otherwise every reader would have to invest to check everything much more carefully.

Without peer review, scientists would have a stronger tendency of focussing more on big-name scientists whose work is more likely to be worthwhile. No peer review worked in a time when all scientists still knew each other a century ago, it would not work well with the current large scientific community. To me it seems to be a really bad idea for interdisciplinary science because it is hard to judge how credible a paper is from another field. Just like it is hard for journalists to judge the quality of a paper without peer review.

Moving science dissemination to blogs may make this the-winner-takes-all principle even stronger. That is how media work. It gives the incumbents much more visibility and power. I do not think that my blog posts are better now than two years ago, but I have much more readers now. Building up an audience takes time.

Quality and deliberation are really important in science. Maybe it goes frustratingly slow, but it more likely goes into the right direction. The Winnower jumped on the tweet of Nate Silver to promote his tool to give blog posts a doi, a digital object identifier with which you can cite the object and that guarantees that it is stored for a decade. An interesting tool that may be useful, but personally, I do not think I have written a blog post that was good enough that it should have a doi. The ideas you find on this blog are hopefully useful or inspiring, but this blog is no science. The precision needed in science can only be found in my scientific articles. I actually wrote in my "about" page:

Some of the posts contain ideas, which may be converted into a work of science. If you are interested in this, there is no need to refer to these posts: you are welcome to call the idea your own. The main step is not to write down a vague idea in a few hours, but to recognize that an idea is worth working on for a year to convert it into a scientific study.

Given my posts a doi may hinder that. Then people may feel they have to cite me, then it may no longer be worthwhile for them to invest so much time in the idea any more and the idea would not be turned into science.

It would be wonderful if there was a solution for the dissemination of science away from the scientific publishers. Many publishers have demonstrated with their actions that they do not see themselves as part of the scientific community any more, but that their main priority is the protection of their near-monopoly profit margins of 30 to 40%. If the people designing such new dissemination solutions think only of the high-interest paper and do not find a solution for everyday papers, their solution will not help science and it will not be adopted. Every paper will need to be bench rejected or peer reviewed, every paper will have to be weaved into a network of trust. When popular papers are continually reviewed that would be an interesting innovation, but a review at the beginning is also needed.

Nate Silver implicitly assumes that fraud is important and hinders scientific progress. That could well be the journalistic and public impression because these cases get a lot of publicity. But actually fraud is extremely rare and hardly a problem in the natural sciences. It is thus not a particularly good reason to change the customs of the scientific community. If such a change makes something else worse, it is probably not worth it. If it makes something else better, we should do it for that reason. (Fraud seems to be more prominent in the medical and social sciences; it is hard for me to judge whether it is so big there that it would be worthwhile to accept trade offs to fight it.)

Informal communication

Chinese calligraphy with water on a stone floor. More ephemeral communication can lead to more openness, improve the exchange of views and produce more quality feedback.

As a blogger I am naturally not against blogging. Even if I mainly see it as a hobby and not as part of my work. It would be great if more colleagues would blog, I would certainly read those posts. Blogging could replace a part of the informal communications that would otherwise happen at conferences, on email distribution lists, normal email and the famous water cooler. It is good for keeping people up to date on the latest papers, conferences and datasets. And I would include twitter in that category as well.

Social media will not be able to replace conferences completely. Scientists being humans, I feel that real contact is still important, especially in the beginning of collaborations. Discussions also often work better in person. The ephemeral quality of debating is important. On twitter and blogs everything is stored for eternity, that stifles debate.

The blog of Nature Chemistry recently commented on social media and how it has changed post publication review, which used to take place at lab meetings or over coffee at conferences. Now it is written in print for all eternity. That creates the problems for both sides. The attacked paper will be damaged even if the attack was found to be unwarranted. While talking about a paper over coffee with people you trust and who may immediately correct you if you are wrong, has much less repercussions than writing it under your name on a blog.

If we go to more internet discussions and internet video presentations we should try to make it more like a real meeting, more like [[snapchat]]; limited in time and you can see it only once.

Another aspect of conferences that is currently missing on the net is that a group of experts is present at the same time and location. The quick bouncing of ideas is missing in the virtual world where discussions go much slower and many are not present. Typically only two persons are discussing with each other on a scientific level, rarely a few more.

Concluding, I would say that social media cannot replace scientific journals. They will not replace scientific conferences and workshop, but may create a new place for informal discussions. To make the internet more useful, we may have to make it more ephemeral.

Tweet

Related reading

Nature Chemistry blog: Post-publication peer review is a reality, so what should the rules be?

Peer review helps fringe ideas gain credibility

The value of peer review for science and the press

Three cheers for gatekeeping


* Photo of "Taoist monk" by Antoine Taveneaux - Own work. Licensed under CC BY-SA 3.0 via Wikimedia Commons.

4th anniversary of Variable Variability

The previous birthdays I have simply forgotten, my apologies, but today is the 4th anniversary of Variable Variability.

More than usual birthdays, the day is a bit arbitrary. Many years I already wrote the occasional essay for my homepage. Ideas that I wanted to share, but were not big enough for an article or thoughts that I was afraid I would forget again with my terrible memory.

The first post of Variable Variability on the 30th of April 2011 was thus a republication of an essay from 2004 on the fractal nature of clouds and the limits of the fractal approximation.

The first two posts specially written for Variable Variability are: Is obesity bias evolutionary? and Good ideas, motivation and economics, both on the 28th of May 2011 somehow. Must have been a writing-fever day.

They are still good posts, but not much read because of the time they were published. To reduce your weight it may help to consciously eat less and move more, but that is an empirical question, it is not as obvious as many claim. Science is a creative profession and knowing where good ideas come from can thus be helpful. The second skill would be a good intuition, which of these problems to tackle. You also need technical skills, but they are overrated.

Another contender for the beginning of this blog would be January 2012, when I posted a press release on the famous homogenization validation study of the COST Action HOME. From a meeting several years before, I vaguely knew that Roger Pielke Sr. was interested in homogenization. Thus I asked him if he was interested in a repost. Roger referred me to someone called Anthony Watts of Watts Up With That. Coming from a good source, Watts asked me whether he could repost, maybe he noticed that I was not too enthusiastic (I had read some WUWT posts by that time) or maybe he read the post (and did not like the message that homogenization methods improve temperature trend estimates) because in the end he did not post my press release.

To promote the press release I started to look a bit more at other blogs and comment there. Now I see this as part of blogging, thus you could see this as the start of blogging.

Mitigation sceptics seem to have a tendency to think that scientific articles are written for the purpose of pissing them off. But they are not that important. Most scientists hardly know they exist. I did know that there was some trouble on the other side of the Atlantic in the USA, that a larger group there was sceptical about climate change. Being sceptical is sympathetic and there are so many real uncertainties. So why not? I was completely unprepared for the industrial production of deceit and misinformation and the rock bottom quality of the nonsense these people made up. I still wonder why mitigation sceptics do not orient themselves by the real uncertainties and problems; they are spelled out in detail in every IPCC report. It sometimes looks like winning a debate is less important to the most vocal mitigation sceptics than angering greenies, which is best done by stubbornly repeating the most stupid arguments you have. (In the light of the discussion we are just having at ATTP, let me add that I this is not an argument, I do not think this paragraph will convince anyone of climate change, I am just describing how I personally see the situation.)

Like the blogger of And Then There's Physics, I naively thought that it may be helpful to explain the mistakes in the posts of WUWT. Now keeping more of an eye on other blogs, I noticed a WUWT post about an erroneous conference abstract by Koutsoyiannis, which was called a "new peer reviewed paper". I wrote: "As a goal-oriented guy, Anthony Watts found the two most erroneous statements". That was the first time I got some readers and at least Watts admitted that there was no peer review. The other errors remained.

Earlier this year I wrote a short update:

Now two and a half year later, there is still no new activity on this [Koutsoyiannis] study. I guess that that means that Koutsoyiannis admits his conference abstract did not show what the State of the Climate and WUWT wanted you to believe. Together with all the other WUWT fails, it is clear that someone interested in climate change should never get his information from WUWT.

But well, who is listening to me, I do not have the right party book.

A little later Anthony Watts put WUWT on hold for the weekend because he was working on something big. He had written a manuscript and a press release and asked for blog reviews. Thus I wrote a blog review about this still unpublished manuscript. This was the first time this blog got a decent number of comments and the time people informed each other in comments elsewhere that there was this new blog on the block. That was fun.

I have somewhat given up on debunking misinformation and aim to write more about what is really important. Sometimes the misinformation is a nice hook to get people interested. I had wanted to announce the birth of the new Task Team on Homogenization (TT-HOM) anyway. By connecting this to the review of the UK Policy Foundation with silly loaded questions, more people now know about TT-HOM than otherwise; normally setting up a a new Task Team would have been a boring bureaucratic act not much people would be interested in.

Debunking does not seem to have much impact, even in cases of clear misquotations and when Watts deceives his readers about his number of readers, there is no bulge. The number of WUWT readers is a small matter, but it does not require any science skills to check, everyone can see this was deception. However, no one complained about being lied to.

America's problems with mitigation sceptics are not because of the science, mitigation sceptics seem to make this claim to make adult political debate about (free market) solutions impossible. By not complaining about deception they make clear that their issue is not the science. Their response to a new scientific paper is almost completely determined by the question: can it be spun into a case against mitigation? If they were interested in science, their response would be determined by the quality of the study. America's problems with mitigation sceptics can thus also not be solved by science or science communication. The American society will have to find a political solution for their toxic political atmosphere. Maybe that simply starts by talking to your neighbour, even if he has the wrong bumper sticker. Go on the street and make it clear to everyone this is important and something to think more about and make it clear to politicians that it is costly to peddle nonsense.

Wishes

Blogging has been a lot of fun. I like writing, playing with ideas and debating.

The first two posts got 50 to 100 pageviews up to now. They were purely written for fun.

If you write a scientific article you can also be happy to get a 100 readers. But if it inspires someone else to continue the thought, to build on it, it is worth it.

Compared to the number of readers a scientist is used to, this blog is a huge megaphone. Thank you all for reading, your interest is very rewarding and nowadays another reason to write posts. With that megaphone also comes responsibility. I guess that mitigation sceptics will not like much of what I write, but when colleagues do not like something I hope for their honest feedback. That would be valuable and is much appreciated.

Blogging has given me a broader perspective on climatology and science in general. Where else would I have heard of the beautiful ideas of John Zyman on the social dimension of science? (Thanks Mark!)

A birthday is a good day to wish something. Do you have anything you would like me to write about? I will not promise anything, I still have about 100 drafts in the queue, but welcome new ideas. I am always surprised how much people read my posts about how science works and the intricacies of the scientific culture. It is my daily life, almost like writing about washing the dishes. It is easy for me to have a blind spot there, thus if you have any questions on science, just shoot.

Please do not wish a post why homogenization increased the trend for some station in Kyrgyzstan. Not on a birthday.

Tweet


* Photo of fireworks by AndreasToerl (Own work) [CC BY-SA 3.0], via Wikimedia Commons

Scientific meetings. The freedom to tweet and the freedom not to be tweeted

Some tweets from a meeting on Arctic sea ice reduction organised by the Royal Society recently caused a stir, when the speaker cried "defamation" and wrote letters to the employers of the tweeters. Stoat and Paul Matthews have the story.

The speaker's reaction was much too strongly, in my opinion, most tweets were professional and respectful critique should be allowed. I have only seen one tweet, that should not have been written ("now back to science").

I do understand that the speaker feels like people are talking behind his back. He is not on twitter and even if he were: you cannot speak and tweet simultaneously. Yes, people do the same on the conference floors and in bars, but then you at least do not notice it. For balance it should be noted that there was also plenty of critique given after the talk; that people were not convinced was thus not behind his back.

Related to this, a blog post is just a long tweet, Paige Brown Jarreau asks:

"Why doesn't every academic get more involved in blogging? Why shouldn't [it] become the default way of interacting with peers?" #MySciBlog

— Paige Brown Jarreau (@FromTheLabBench) October 14, 2014

Almost all scientists use both papers and meetings for communication. Tweets and blogs do not have that status; they could complement the informal discussions at meetings, but do differ in that everyone can read them, for all time. Social media will never be and should never be a substitute for the scientific literature.

Imagine that I had some preliminary evidence that the temperature increase since 1900 is nearly zero or that we may already have passed the two degree limit. I would love to discuss such evidence with my colleagues, to see if they notice any problems with the argumentation, to see if I had overlooked something, to see if there are better methods or data that would make the evidence stronger. I certainly would not like to see such preliminary ideas as a headline in the New York Times until I had gathered and evaluated all the evidence.

The problem with social media is that the boundaries between public and private are blurring. After talking about such a work at a conference, someone may tweet about it and before you know it the New York Times is on the telephone.

Furthermore, you always communicate with a certain person or audience and tailor your message to the receiver. When I write on my blog, I explain much more than when I talk to a colleague. Reversely, if someone hears or reads my conversation with a colleague this may be confusing because of the lack of explanation and give the wrong impression. In person at a conference a sarcastic remark is easily detected, on the written internet sarcasm does not work, especially when it comes to climate "debate" where there is no opinion too exotic.

This is not an imaginary concern. The OPERA team at CERN that found that neutrinos could travel faster than light got into trouble this way. The team was forced to inform the press prematurely because blogs started writing about their finding. The team made it very clear that this was still very likely a measurement error: “If this measurement is confirmed, it might change our view of physics, but we need to be sure that there are no other, more mundane, explanations. That will require independent measurements.” But a few months after the error was found, a stupid loose cable, the spokesperson and physics coordinator of OPERA had to resign. I would think that that would not have happened without all the premature publicity.

If I were to report that the two degree limit has already been reached, that the raw temperature data had a severe cooling bias, a multimedia smear campaign without comparison would start. Then I'd better have the evidence in my pocket. The OPERA example shows that even if you do not overstate your case, your job is in jeopardy. Furthermore, such a campaign would make further work extremely difficult, even in a country like Germany that has Freedom of Research in its constitution to prevent political interference with science:

Arts and sciences, research and teaching shall be free.
(Kunst und Wissenschaft, Forschung und Lehre sind frei)

This fortunate fact, for example, disallows FOIA harassment of scientists.

That openness is not necessary in the preliminary stages fits to the pivotal role of the scientific literature in science. In an article a scientist describes his findings in all the detail necessary for others to replicate it and build on it. That is the moment everything comes in the open. If the article is written well that is all one should need.

I hope that one day all scientific articles will be open access so that everyone can read them. I personally prefer to publish my data and code, if relevant, and would encourage all scientists to do so. However, how such a scientific article came into existence is not of anyone's business.

All the trivial and insightful mistakes that were made are not of anyone's business. And we need a culture in which people are allowed to make mistakes to get ahead in science. As a saying goes: if you are not wrong half of the time you are not pushing yourself enough to the edge of our understanding. By putting preliminary ideas in the limelight too soon you stifle experimentation and exploration.

In the beginning of a project I often request a poster to be able to talk about it with my most direct colleagues, rather than requesting a talk, which would broadcast the ideas to a much broader audience. (A secondary reason is that a well-organised poster session also provides much more feedback.) Once the ideas have matured a talk is great to tell everyone about it.

If a scientists chooses to show preliminary work before publication that is naturally fine. For certain projects the additional feedback my be valuable or even necessary as in case of collaboration with citizen scientists. And normally the New York Times will not be interested. However, we should not force people to work that way. It may not be ideal for every scientific question or person.

Opening up scientific meetings with social media and webcasts may intimidate (young) researchers and in this way limit discussion. Even at an internal seminar, students are often too shy to ask questions. On the days the professor is not able to attend, there are often much more questions. External workshops are even more intimidating, large conferences are even worse, and having to talk to a global audience because of social media is the worst of all.

More openness is not automatically more or better debate. It can stifle debate and also move it to smaller closed circles, which would be counter productive.

Personally I do not care much who is listening, as long as the topic is science I feel perfectly comfortable. The self-selected group of scientists that blogs and tweets probably feels the same. However, not everyone is that way. Some people who are much smarter than I am would like to first sharpen their pencils and think a while before they comment. I know from feedback by mail and at conferences that much more of my colleagues read this blog than I had expected because they hardly write comments. Writing something for eternity without first thinking about it for a few days, weeks or months is not everyone's thing. This is something we should take into account before we open informal communication up too much.

In spring I asked the organisers of a meeting how we should handle social media:

A question we may want to discuss during the introduction on Monday morning: Do people mind about the use of social media during the meeting? Twitter and blogs, for example. What we discuss is also interesting for people unable to attend the meeting, but we should also not make informal discussions harder by opening up to the public too much.
I was thinking about people saying in advance if they do not want their talk to be public and maybe we should also keep the discussions after the talks private, so that people do not have the think twice about the correctness of every single sentence.

The organisation kindly asked me to refrain from tweeting. Maybe that was the reply because they were busy and had never considered the topic. But that reply was fine by me. How appropriate social media are depends on the context and this was a small meeting, where opening it up to the world would be a large change in atmosphere.

I guess social media is less of a problem the general assembly of the European Geophysical Union (EGU), where you know that there is much press around. Especially for some of the larger sessions where there can be hundreds of scientists and some journalists in the audience. You would not use such large audiences to bounce some new ideas, but to explain the current state of the art.

Even EMS and EGU the organisation provides some privacy: it is officially not allowed to make photos of the posters. I would personally prefer that every scientist can indicate him or herself whether this is okay for his poster (and if you make rules, you should also enforce them).

Another argument against tweeting is that it distracts the tweeter. At last weeks EMS2014 there was no free Wi-Fi in the conference rooms (just in a separate working room). I thought that was a good thing. People were again listening to the talks, like in the past, and not tweeting, surfing or doing their email.

[UPDATE. Doug McNeall, the MetOffice guy that convinced me to start tweeting, has written a response on his blog.]

Related Reading

Letter to Science by Germán Orizaola and Ana Elisa Valdés: Free the tweet at scientific conferences

Kathleen Fitzpatrick (Director of Scholarly Communication) gives some sensible Advice on Academic Blogging, Tweeting, Whatever. For example: “If somebody says they’d prefer not to be tweeted or blogged, respect that” and “Do not let dust-ups such as these stop you from blogging/tweeting/whatever”.

I previously wrote about: The value of peer review for science and the press. It would be nice if the press would at least wait until a study is published. Even better would be to wait until several study have been made. But that is something we, as scientists, cannot control.

* Photo by Juan Emilio used with a Creative Commons CC BY-SA 2.0 licence.

WUWT and Co. not interested in my slanted opinion, part II

The Hockey Schtick broke a story about homogenization, which was picked up by Watts Up With That? They found an article, which they claimed showed that homogenization makes the influence of an enhanced Urban Heat Island stronger. I disagreed with their conclusions, the homogenization method in the paper was designed to study the UHI more accurately and removed other effects that masked it. However, for the global data sets other homogenization methods are used that reduce the influence of urbanization.

There was one thing about this interaction with WUWT and Co, that I found interesting. Interesting in the way Judith Curry uses the word, suggestive and non-committal.

I wrote a comment (see below) on five blogs that had covered the story, quite neutral comments not to violate any comment policy. They had a link to my blog article for more information. One would expect skeptical people, in the original meaning of the word, to be interested in an alternative opinion, especially so if it comes from someone that is reasonably well informed. However, only one of the five comments was published. [UPDATE: now two comments, see below.]

The post was published on the 1st of February, at the time of writing 3 days ago.

Five blogs

The Hockey Schtick: New paper finds adjusted temperature data in China has significantly exaggerated warming
Below the post at The Hockey Schtick are two comments, but mine did not appear.
[UPDATE: After a twitter exchange, I have resubmitted my comment and it has appeared. Problem solved.]

Watts Up With That?: Important study on temperature adjustments: ‘homogenization…can lead to a significant overestimate of rising trends of surface air temperature.’
Anthony Watts thought the homogenization article was very important. My comment has not appeared yet. Last time I tried a similar comment, at least the text "[sorry, but we aren't interested in your slanted opinion - mod]" appeared.
I am on the WUWT black list and normally it takes a few hours before my comments appear, at which time the discussion has typically progressed for fresh posts and most people do not seem to notice the comment any more. Three days would be a new record.

Tips on How to Study: Important study on temperature adjustments: 'homogenization…can …
Then I wrote a comment for Tips On How To Study. Looking in more detail now, it does not seem to be an ostrich blog, but rather someone who copies articles from other blogs, mostly not about How To Study.

The Drinking Water Advisor: Beijing adjusted surface temperature data found to have exaggerated warming trends
The fourth comments was at The Drinking Water Advisor, a blog about water, fluoridation and climate change. My comment did not yet show up here.

Errors in IPCC climate science: How many times does a truth have to be told ? – UHI warming has been cemented into global temperature series by adjusting for steps outward from cities
There is one blog where my comment was posted: Errors in IPCC climate science. Chapeau. One of the bloggers, Warwick S. Hughes, has written some scientific articles and comments on station data quality. He should have been able to see the mistake in the WUWT article.

Conclusions

The last few month I have trouble with getting comments published, also on science blogs, but I would estimate this to affect a few percent of my comments. Getting only one [two] comment[s] out of four or five published is less than expected. Being a scientist, I will naturally assume good faith, the reason may well be some sort of spam filter. I hope my critical readers can make up their own mind.

The comment

At WUWT and Tips, I wrote the comment below. On the other blogs shorter ones with a similar tone.

The authors wanted to accurately study the influence of the urban heat island effect and thus removed the cooling effect of two relocations using a special homogenization method. Without this method the importance of urbanization would have been underestimated.

This special homogenization method is not used for global climate data, where the aim is to remove the effect of urbanization as well. Thus this WUWT post is unfortunately inaccurate. For details, please have a look at my blog.

I would personally say, that that is quite neutral. Certainly compared to the way scientists are described in the comments of WUWT.

Further reading

WUWT comes right out and says "We Aren't Interested" in facts
HotWhoppers take on the previous time WUWT was not interested in my slanted opinion.