Proposed: cases of academic fraud would be caught sooner if academics mostly wrote blog posts instead of journal articles.
— Nate Silver (@NateSilver538) May 30, 2015That would be a disaster for science, but somehow some twitterers apparently liked the idea. Maybe this is because the journalistic view and public view is on a minute portion of science. The tip of the iceberg could even be an understatement here. For example, together with Ralf Lindau I just have a nice paper out on how well we can determine the date of an abrupt non-climatic change in a temperature time series.
Lindau & @VariabilityBlog The uncertainty of break positions detected by homogenization algorithms in climate records http://t.co/tFTzyU5RBc
— O. Bothe (@geschichtenpost) May 19, 2015The one "retweet" is me. The two "favourites" are two colleagues working on homogenization. Even I am wondering whether I should write a blog post about this paper and it certainly will not get into the press. That does not mean that it is not important for science, just that it is highly technical and that it will "just" help some scientists to better understand how to remove non-climatic changes. This will hopefully lead to better methods to remove non-climatic changes and finally to better assessments of the climatic changes. The latter study would be interesting to the public, all the studies it is based on, not so much. I may actually write a blog post about this paper, but then mainly explaining why it is important, rather than what is in the paper itself. The why could be interesting.
Of all the articles I have written only one was marginally interesting for the press. The paper were we showed that homogenization methods to remove non-climatic changes from station data work. The FAZ, a German conservative newspaper reported on it, ironically concluding: Climate change is not a measurement error. The University of Bonn is setting up a crisis helpline for mitigation sceptics. With that angle they could make sufficiently interesting for their news ticker. On the other hand, for the scientific community working on homogenization this was one of the main papers. For the scientific community the importance was the improvement in the validation methodology and the results that showed which kind of method works best, for the press this was already too much detail, they just reported that the methods work, which is nothing new for us.
Scientific literature
The science the public does not see, neither in the media nor on blogs, is also important for science. We will need a way to disseminate science that also works for the other 99.9% of science. Blogs and "blog review" won't do for this part.Peer review is sometimes seen as gate keeping, but peer review actually helps the underdogs with fringe ideas. Helps ideas that otherwise would not be taken seriously. Helps ideas in which people would otherwise not be willing to invest their time to check it and see how they could build on it. And it saves a lot of time that otherwise every reader would have to invest to check everything much more carefully.
Without peer review, scientists would have a stronger tendency of focussing more on big-name scientists whose work is more likely to be worthwhile. No peer review worked in a time when all scientists still knew each other a century ago, it would not work well with the current large scientific community. To me it seems to be a really bad idea for interdisciplinary science because it is hard to judge how credible a paper is from another field. Just like it is hard for journalists to judge the quality of a paper without peer review.
Moving science dissemination to blogs may make this the-winner-takes-all principle even stronger. That is how media work. It gives the incumbents much more visibility and power. I do not think that my blog posts are better now than two years ago, but I have much more readers now. Building up an audience takes time.
Quality and deliberation are really important in science. Maybe it goes frustratingly slow, but it more likely goes into the right direction. The Winnower jumped on the tweet of Nate Silver to promote his tool to give blog posts a doi, a digital object identifier with which you can cite the object and that guarantees that it is stored for a decade. An interesting tool that may be useful, but personally, I do not think I have written a blog post that was good enough that it should have a doi. The ideas you find on this blog are hopefully useful or inspiring, but this blog is no science. The precision needed in science can only be found in my scientific articles. I actually wrote in my "about" page:
Some of the posts contain ideas, which may be converted into a work of science. If you are interested in this, there is no need to refer to these posts: you are welcome to call the idea your own. The main step is not to write down a vague idea in a few hours, but to recognize that an idea is worth working on for a year to convert it into a scientific study.Given my posts a doi may hinder that. Then people may feel they have to cite me, then it may no longer be worthwhile for them to invest so much time in the idea any more and the idea would not be turned into science.
It would be wonderful if there was a solution for the dissemination of science away from the scientific publishers. Many publishers have demonstrated with their actions that they do not see themselves as part of the scientific community any more, but that their main priority is the protection of their near-monopoly profit margins of 30 to 40%. If the people designing such new dissemination solutions think only of the high-interest paper and do not find a solution for everyday papers, their solution will not help science and it will not be adopted. Every paper will need to be bench rejected or peer reviewed, every paper will have to be weaved into a network of trust. When popular papers are continually reviewed that would be an interesting innovation, but a review at the beginning is also needed.
Nate Silver implicitly assumes that fraud is important and hinders scientific progress. That could well be the journalistic and public impression because these cases get a lot of publicity. But actually fraud is extremely rare and hardly a problem in the natural sciences. It is thus not a particularly good reason to change the customs of the scientific community. If such a change makes something else worse, it is probably not worth it. If it makes something else better, we should do it for that reason. (Fraud seems to be more prominent in the medical and social sciences; it is hard for me to judge whether it is so big there that it would be worthwhile to accept trade offs to fight it.)
Informal communication
Chinese calligraphy with water on a stone floor. More ephemeral communication can lead to more openness, improve the exchange of views and produce more quality feedback.
Social media will not be able to replace conferences completely. Scientists being humans, I feel that real contact is still important, especially in the beginning of collaborations. Discussions also often work better in person. The ephemeral quality of debating is important. On twitter and blogs everything is stored for eternity, that stifles debate.
The blog of Nature Chemistry recently commented on social media and how it has changed post publication review, which used to take place at lab meetings or over coffee at conferences. Now it is written in print for all eternity. That creates the problems for both sides. The attacked paper will be damaged even if the attack was found to be unwarranted. While talking about a paper over coffee with people you trust and who may immediately correct you if you are wrong, has much less repercussions than writing it under your name on a blog.
If we go to more internet discussions and internet video presentations we should try to make it more like a real meeting, more like [[snapchat]]; limited in time and you can see it only once.
Another aspect of conferences that is currently missing on the net is that a group of experts is present at the same time and location. The quick bouncing of ideas is missing in the virtual world where discussions go much slower and many are not present. Typically only two persons are discussing with each other on a scientific level, rarely a few more.
Concluding, I would say that social media cannot replace scientific journals. They will not replace scientific conferences and workshop, but may create a new place for informal discussions. To make the internet more useful, we may have to make it more ephemeral.
Tweet
Related reading
Nature Chemistry blog: Post-publication peer review is a reality, so what should the rules be?Peer review helps fringe ideas gain credibility
The value of peer review for science and the press
Three cheers for gatekeeping
* Photo of "Taoist monk" by Antoine Taveneaux - Own work. Licensed under CC BY-SA 3.0 via Wikimedia Commons.
[This comment by Jos was erroneously published below the wrong post. Now copied here.]
ReplyDeleteHi Victor,
That last notion - a group of experts present at the same time and location with a quick bouncing of ideas that is missing in the virtual world where discussions go much slower and many are not present - is an interesting one.
The peer review process also does not really fit this description. Yet personal experience is that exactly that - having a group of experts together discussing ideas - can be extremely valuable, stimulating and provide a fast track for sharpening ideas.
In the past that would happen on conferences and workshops, but with an ever expanding research community and conferences such possibilities, combined with more pressure to (quickly) publish the possibility of letting your ideas gradually mature, are becoming more and more rare.
I could imagine that some form of pre-publication online discussion platform for experts and for example moderated by journal editors or journal staff might be something worth thinking about. Maybe also for continued post-publication discussion of papers. There would also have to be some reward for participatation of these experts, as they put in time/resources.
As an example, I've played around with the idea of journals inviting experts on 'hot' topics to discuss this hot topic at an online platform - like with Climate Dialogue - where the discussion would count as peer-reviewed , i.e. get the same status as a research paper. The latter would provide incentives for experts to participate, which was an issue when running Climate Dialogue.
Thoughts?
Cheers, Jos.
That was exactly what I was talking about with the difference between written and oral discussions. If you hold the same debate orally at a conference you would not have to pay for it. When you do it on the internet and you fix every written contribution for eternity it becomes work. Then you have to make sure that every statement is clear and makes sense, while in a normal conversation it is no big deal if someone has to ask for a clarification or makes a small correction.
ReplyDeleteYou also notice how little scientists write comments here, while I know from conferences that they do read my blog occasionally.
Well, there is an intermediate ground, whether or not it's where permanent results should live: arXiv.org. Larry Wasserman has suggested that arXiv.org could serve as a preliminary step to publication (as it actually does), because articles which are interesting in a field are accessed, and duds are not. In contrast Chopin, Gelman, Mengersen, and Robert feel refereeing is essential.
ReplyDeleteMy view is that refereeing is essential, but publications are like the "tombstone announcements" of stock offerings which often appear in the financial press: By the time they appear, they are formal notices and don't announce anything. The people in the know not only have received word of the development, they have moved past it.
And some fields have, to me, odd notions of what matters. Apparently, in Computer Science, and especially in Internet measurement work, appearance in conferences is the standard of scholarly achievement. I'm not sure of the role of papers there, but that sure contrasts with achievement in physical and observational sciences, statistics, and engineering, where the steps are typically poster, conference, and then publication.
Whether a publication is a tombstone may also depend on the field. In the atmospheric sciences details are often important. The atmosphere being a complex and complicated system. Conference presentation can only provide an abstract and you really need to read the article.
ReplyDeleteI used to use arxiv until they made it harder to get rid of the pseudo-science manuscripts and you had to get recommendations. Now I just put my manuscripts on my homepage. Because arxiv is not used that much in the atmospheric sciences that is likely almost as good.
Interesting topic. As for a group of experts meeting -- you might get that sooner or later via VR. I think no written social media thingy can replace this.
ReplyDeleteThe publication model stems from a time when publications had to be limited because they were printed. Today publication is free - now it is about efficient and fine-grained filtering, which can be guided by post-publication review. Simple as that. Putting it to work in the real world is the hard part ...
Virtual Reality. Yes!! And you can immediately walk through your 3-dimensional dataset.
ReplyDeleteAlso without paper journals, the limitation is how much information the reader can process. A good filter is important. If we do not know in advance which texts are of WUWT & Co. quality, that would severely hinder science. Maybe filtering is even more important than ever given the increase in number of publication due to the micro-management of science on the number of publications. Designing good filter, at least theoretically, should be easier on the internet.