Science Feedback on Steroids

Climate Feedback is a group of climate scientists reviewing press articles on climate change. By networking this valuable work with science-interested citizens we could put this initiative on steroids.

Disclosure, I am member of Climate Feedback.

How Climate Feedback works

Climate Feedback works as follows. A science journalist monitors which stories on climate change are shared much on social media and invites publishing climate scientists with relevant expertise to review the factual claims being made. The scientists make detailed reviews on concrete claims, ideally using web annotations (see example below), sometimes by email.

 

 

They also write a short summary of the article and grade its scientific credibility. These comments, summaries and grades are then summarized in a graphic and an article written by the science journalist. 

Climate Feedback takes care of spreading the reviews to the public and to the publication that was reviewed. Climate Feedback is also part of a network of fact checking organizations giving them more credibility and they add metadata to the review pages that social media and search engines can show their users.

 

 

For scientists this is a very efficient fact checking operation. The participants only have to respond to the claims they have expertise on. If there are many claims outside my expertise I can wait until my colleagues added their web annotations before I write my summary and determine my grade. Especially compared to writing a blog post Climate Feedback is very effective.

The initiative recently branched out to reviewing health claims with a new Health Feedback group. The umbrella is now called Science Feedback.

The impact

But there is only so much a group of scientists can do and by the time the reviews are in and summarized the article is mostly old news. Only a small fraction of readers would see any notifications social media systems could put on posts spreading them.

This is still important information for people who closely follow the topic, helps them to see how such reviews are done, assess which publication are reliable and helps to see which groups are credible. 

The reviews may be most important for the journalists and the publications involved. Journalists doing high quality work can now demonstrate this to editors who will mostly not be able to assess this themselves. Some journalists have even asked for reviews of important pieces to showcase the quality of their work. Reversely editors can seek out good journalists and cut ties with journalists regularly hurt their reputation. The latter naturally only helps publications that care about quality.

The Steroids

With a larger group we could review more articles and have results while people are still reading it. There are not enough (climate) scientists to do this. 

For Climate Feedback I only review articles on topics where I have expertise. But I think I would still do a decent job outside of my expertise. It is hard to determine how good a good article is, but the ones that are clearly bad are easy to identify and this does not require much expertise. At least in the climate branch of the US culture war the same tropes are used over and over again, the same "thinking" errors are made over and over again. 

Many who are interested in climate change are interested in scientific detail, but are not scientists, would probably do a good job identifying these bad articles. Maybe even better. They say that magicians were better at debunking paranormal claims than scientists. We live in a bubble where most argue in good faith and science-interested normal citizens may well have a better BS detector.

However, how do we know who is good at this? Clearly not everyone, otherwise such a service would not be needed. We would have the data from Climate Feedback and Health Feedback to determine which citizen scientist's assessments predict the assessments of the scientists well. We could also ask people to classify the topic of the article. I would be best at observational climatology, decent in physical climatology and likely only average when it comes to many climate change impacts and economic questions. We could also ask people how confident they are in their assessments.

In the end it would be great to ingest ratings in a user friendly way with 1) a browser add-on on the article homepage itself, 2) replying to posts mentioning the article on social media, like replying to a tweet adding the handle of the PubPeerBot automatically submits the tweet to PubPeer.

A server would compute the ratings and as soon as there is enough data create a review homepage with the ratings as metadata to be used by search engines and social media sites. We will have to see if they are willing to use such a statistical product. Also an application programming interface (API) and ActivityPub can be used to spread the information to interested parties.

I would be happy to use this information on the micro-blogging system for scientists Frank Sonntag and I have set up. I presume more Open Social Media communities would be grateful for the information to make their place more reality-friendly. A browser add-on could also display the feedback on the article's homepage itself and on posts linking to it.

How to start?

Before creating such a huge system I would propose a much smaller feasibility study. Here people would be informed about articles Climate or Health Feedback are working on and they can return their assessments until the one of Climate Feedback is published. This could be a simple email distribution list to distribute the articles and a cloud-based spread sheet or web form to return the results. 

This system should be enough to study whether citizens can distinguish fact from fiction well enough (I expect so, but knowing for sure is valuable) and develop statistical methods to estimate how well people are doing, how to compute an all over score and how many reviews are needed to do so.

This set-up points to two complications the full system would have. Firstly, only citizen's assessments that are made before the official feedback can be used. this should not be too much of a problem as most readers will read the article before the official feedback is published.

Secondly, as the number of official feedbacks will be small many volunteers will likely not review any of these articles themselves or just a few. Thus the assessment of how accurate the predictions of person A of articles X, Y and Z are may have to be assessed comparing their assessments with those of B, C and D who review X, Y or Z as well as one of the articles Climate Feedback reviewed. This makes the computation more complicated and uncertain, but if B, C and D are good enough, this should be doable. Alternatively, we would have to keep on informing our volunteers of the articles being reviewed by the scientists themselves.

This new system could be part of Science Feedback or an independent initiative. I feel, it would at least be good to have a separate homepage as the two systems are quite different and the public should not mix them up. A reason to keep it separate is that this system could also be used in combination with other fact checkers, but we could also make that organizational change when it comes to that.

Another organization question is whether we would like Google and Facebook to have access to this information or prefer a license that excludes them. Short term it is naturally best when they also use it to inform as many people as possible. Long-term it would also be valuable to break the monopolies of Google and Facebook. Having alternative services that can deliver better quality due to our assessments could contribute to that. They have money, we have people.

I asked on Twitter and Mastodon whether people would be interested in contributing to such a system. Fitting to my prejudice people on Twitter were more willing to review (I do more science on Twitter) and people on Mastodon were more willing to build software (Mastodon started with many coders).

What do you think? Could such a system work? Would enough people be willing to contribute? Is it technologically and statistically feasible? Any ideas to make the system or the feasibility study better?

Related reading

Micro-blogging for scientists without nasties and surveillance 

 

Climate Feedback explainer from 2016: Climate scientists are now grading climate journalism

 

Discussion of a controversial Climate Feedback and the grading system used: Is nitpicking a climate doomsday warning allowed?

 

Micro-blogging for scientists without nasties and surveillance

Two years ago I joined Mastodon to get to know a more diverse group of people here in Bonn. Almost two thousand messages later, I can say I really like it there and social networks like Mastodon are much more healthy for society as well. Together with Frank Sonntag we have recently set up a Mastodon server for publishing scientists. Let me explain how it works and why this system is better for the users and society.

Mastodon looks a lot like Twitter, i.e. it is a micro-blogging system, but many tweaks make it a much friendlier place where you can have meaningful conversations. One exemplary difference is that there are no quote tweets. Quoting rather than simply replying is often used by large accounts to bully small ones by pulling in many people into the "conversation" who disagree. I do miss quote tweets, they can also be used for good, to highlight what is interesting about a tweet or to explain something that the writer assumed their readers know, but your readers may not know. But quote tweets make the atmosphere more adversarial, less about understanding and talking with each other. Conflict leads to more engagement and more time on the social network, so Twitter and Facebook like it, but pitting groups against each other is not the public debate that makes humanity better.

The main difference under the hood is that the system is not controlled by one corporation. There is not one server, but many servers that seamlessly talk with each other, just like the email system. The communication protocol (ActivityPub) is a standard of the World Wide Web Consortium, just like HTML and HTTPS, which powers the web.

This means that you can chose the server and interface you like and still talk to others, while people on Twitter, Facebook, Instagram, WordPress and Tumblr can only talk to other people in their silo. As they say the modern internet is a group of five websites, each consisting of screenshots from the other four. It is hard to leave these silos, it would cut you off from your friends. This is also why the system naturally evolves into a few major players. Their service is as bad as one would expect with the monopoly power this network effect gives them.

ActivityPub is not only used by Mastodon, but also by other micro-blogging social networks such as Pleroma, blogging networks such as Write.as, podcasting services such as FunkWhale and file hosting such as NextCloud. There is a version of Instagram (PixelFed) and of YouTube (PeerTube). With ActivityPub all these social networks can talk to each other. Where they do different things, the system is designed to degrade gracefully. FixelFed shows photos more beautifully, has collections and filters, but Mastodon gracefully shows the recent photos as a photo below a message. PeerTube shows one large video on a page, just like Twitter, Mastodon shows the newest videos in small below a message in the news feed. The full network is called the fediverse, a portmanteau of federation and universe.

Currently all these services are ad-free and tracking-free. The coding of the open source software is largely a labor of love, even if some coders are supported by micro-funding, for example Patreon or Liberapay. Most servers are maintained by people as hobby, some (like for email) by organization for their members, some larger ones again use Patreon or Liberapay, some are even coops.

This means that technology enthusiasts from the middle class are mostly behind these networks. That is better than a few large ad corporations, but still not as democratic as one would like for such an important part of our society.

Moderation

Not only can these networks talk to each other, they also themselves consist of many different servers each maintained by another group, just as the email system. This means that moderation of the content is much better than on Twitter or Facebook. The owners of the servers want to create a functional community, while these communities are relatively small. So they can invest much more time per moderation decision than a commercial silo would. Also if the moderation fails, people will go somewhere else.

Individual moderation decisions only pertain one server and are thus less impactful and can consequently be more forceful. If you do not like the moderation, you can move to another server that fits your values better. If you are kicked off a server, you can go to another one and still talk to your friends. Facebook kicking someone off Facebook or Twitter kicking someone off Twitter is somewhat of a big deal and is thus only done in extreme cases, when someone already created a lot of damage to the social fabric, while others make the atmosphere toxic staying below the radar.

If someone is really annoying they may naturally be removed from many servers. Then it does become a problem for this person, but that only happens when many server administrator agree you are not welcome. So maybe that person is really not an enrichment for humanity.

The extreme example would be Nazis. Some Nazis were too extreme for Twitter and started their own micro-blogging network. Probably most Nazis know the name already, but I think it is a good policy not to help bad actors with PR. As this network was used to coordinate their violent and inhumane actions, Google and Apple have removed their apps from their app stores. I may like that outcome, but these corporations should not have that power. Next this network started using ActivityPub, so that they can use ActivityPub apps. The main Activity network does not like Nazis, so they all blocked this network.

I feel this is a good solution for society, everyone has their freedom of speech, but Nazis cannot harass decent people. They can tell each other pretty lies, where being responsible for killing more than 138 thousand Americans is patriotism, but 4 is treason, where the state brutalizing people expressing their 1st amendment rights is freedom, but wearing a mask not to risk the lives of others is tyranny. At least we do not have to listen to the insanity. (The police should naturally listen to stop crime.)

Many of the societal problems of Facebook and Co. would be much reduced if we would legislate that such large networks open up to competition by implementing open communication protocols like ActivityPub. Then they would be forced to deliver a good product to keep their customers. If they do not change many will flee the repulsive violent conspiracy surveilance hell they were only still part of to be able to talk to grandma.

Because there are nearly no Nazis and other unfriendly characters, the fediverse is very popular with groups they would otherwise harass and bully into silence. It is a colorful bunch. This illustrates that extending the right to free speech to the right to be amplified by others does not optimize the freedom of speech, but in reality excludes many voices.

A short encore: the coders of the ActivityPub apps also do not like Nazis. So they hard coded Nazi blocks into their apps. It is open source software, so the Nazis can remove this, but Google and Apple will not accept their apps. The latter is the societal problem, the coders are perfectly in their right not to want their work be used to destroy civilization.

Open Science

The fediverse looks a lot like the Open Science tool universe I am dreaming of. Many independent groups and servers that seamlessly communicate with each other. The Grassroots post-publication peer review system I am working on should be able to gather reviews from all the other review and endorsement systems. They and repositories should be able to display grassroots reviews.

The reviews could be aided by displaying information on retractions from the Retraction Watch database. I hope someone will build a service that also warns when a cited article is retracted. The review could show or link to open citations of the article and statistics checks, as well as plagiarism and figure tampering checks.

We could have systems that warn authors of new articles and manuscripts they may find interesting given their publication history and warn editors of manuscripts that fit to their journal. I recently made a longer list of useful integrations and services and put it on Zenodo.

These could all be independent services that work together via ActivityPub and APIs, but the legacy publishers are working on collaborative science pipelines that create network effects, to ensure you are forced to use the largest service where you colleagues are and cannot leave, just like Facebook, Google and Twitter.

FediScience

I am explaining all this to illustrate that such a federated social network is much better for society and its users. I really like the atmosphere on Mastodon. You can have real conversations with interesting people, without lunatics jumping in between or groups being pitted against each other. If people hear less and less of me on Twitter, that is one of the reasons.

So I hope that this kind of network is the future and to help getting there we have started a Mastodon server for publishing scientists. "We" is me and former meteorologist Frank Sonntag who leads a small digital services company, AKM-services. So for him setting up a Mastodon server was easy.

Two years ago he had to drag me to Mastodon a bit, when we tried to set up a server just for the Earth Sciences. That did not work out. By now that I have learned to love Mastodon, it has gotten a lot bigger and more people are aware of the societal problems due to social media. So it is time for another try with a larger target audience, all scientists. We have called it: FediScience.

Mastodon is still quite small with about half a million active users; Twitter is 100 times bigger. My impression is that at least many climate scientists are on Twitter for science communication. For many leaving Twitter is not yet a realistic option, but FediScience could be a friendly place to talk to colleagues, nerd out about detailed science, while staying on Twitter for more comprehensible Tweets on the main findings.

Once we have a nice group together, we can together decide on the local rules. How we would like to moderate, who will do the moderation, with whom our server federates, who is welcome, how long the messages are, whether we want equations, ... In the end I hope the server will be run by an association with the users as members.

My network empire

My solution to Mastodon still being small was to stay on Twitter to talk about climate science, the political problems leading to the climate branch of the American culture war and anything that comes up on this blog: Variable Variability. As the goal of my Mastodon account in Bonn is to build a local network for a digital non-profit, there I talk about the open web, data privacy more, often write in German and only occasionally write about climate. I aim to use my new account at FediScience to talk about (open) science and to enjoy finally a captive audience that understands the statistics of variability. As administrator I will try to help people find their way in the fediverse.

Next to this the grassroots open review journals are on Mastodon, Twitter and Reddit. And I have inherited the Open Science Feed from Jon Tennant, which is on Mastodon, Twitter and Reddit. Both deserve to get an IndieWeb homepage and a newsletter, but all newsletters I know are full of trackers, suggestions for ethical ones are welcome. For even more fun, I also created a Twitter feed for the climate statistics blog Tamino and scientific skeptic Potholer54's YouTube channel. I should probably put them on Mastodon as well. That makes this blog my 12th social media channel. Pro-tip: with Firefox "containers" you can be logged in into multiple Mastodon, Twitter or Reddit accounts.

Every member of FediScience can invite their colleagues to join the network. Please do. If you share the link in public, please make it time limited.

Please let other scientists know about FediScience, whether by mail or via one of the social media silos. These are good Tweets to spread:

My own Tweet: The post below explains why this #Mastodon system is better for us & for society.

Ruth Mottram: come and join us, the water is lovely...

Alex Holcombe: many tweaks make it a much friendlier place where you can have meaningful conversations.

 

Sign-up to FediScience.

Glossary

When you join Mastodon, the following glossary is helpful.

The Bird Site	Twitter
Fediverse	All federated social media sites together
Instance	Sever running Mastodon
Toot	Tweet
Boost	Retweet
ActivityPub (AP)	The main communication protocol in the fediverse
Content Warning (CW)	A convenient way to give a heads up
Nitter.net	A mirror site of Twitter without tracking, popular for linking to in Mastodon

 

Will we be wiped out by machine overlords?

The American Public Broadcasting Service (PBS) just aired a piece on artificial intelligence that seems to be quite typical for the American published opinion. The segment — titled "Will we be wiped out by machine overlords? Maybe we need a game plan now" — could not have been more wrong in my non-expert opinion.

They gave examples of the progress of machine intelligence. The example that impressed me the most is computers beating humans at playing Go, a game that starts from an empty board and has a huge number of possible progressions. From these examples of very specific tasks they extrapolate to machines soon having more general intelligent than humans. I think they are wildly optimistic (pessimistic?) about that. This will still take a long time.

A hint of how difficult handling reality and other "intelligent" beings is is the world soccer tournament for robots. And that is still just a game with a well defined surrounding and rules.



Notice that when the robots make a goal, they do not take off their shirts and do not jump on each other to celebrate.

But there will likely be a time where computers are smarter than we are. So what? They have long been better at mental arithmetic, now they are better at Go. That does not make them overlords. Machines are also faster than us, stronger than us, dive deeper than us, explored more of the planetary system than us. So what?

Just because machines are intelligent, does not mean they want to rule and even less that they would be evil. It will be hard enough to program them to survive and not to jump off a cliff. Making them want to survive will be even harder.

Just because they would be intelligent, does not mean they are like us. That is likely the main thinking error people are making: we are intelligent, thus an intelligent entity is like us. We evolved to want to survive and reproduce, mostly by collaborating with each other and nature, if necessary also in conflict. Intelligence is just a side show of this, that was apparently advantageous in our niche.

It is possible to make computers solve problems with methods that mimic evolution. Rather than tell the computer in detail what to do, with these methods you only tell the computer what problem you would like it to solve. That has to be a concrete aim so that the computer can determine if it is getting better at solving the problem. Even if you somehow are able to make the computer solve the problem "general intelligence", the computer would just be intelligent.

Being a human is so much more than being intelligent. There is currently a bonus on the labour market for smart people, but you need so much more capabilities and drive to make something of your life.

If being intelligent were so great, we would have been much more intelligent already. It probably helps if a tribe has a few intelligent people, but a tribe of philosophers would quickly go extinct. Getting the variability right is as important as the mean and bigger is normally not better, there are trade offs.

One wonders where this fear for intelligence comes from. There are so many people more dangerous than a nerd with stick arms. There are also such machine-overlord stories in Europe, but my impression is that is is more common in America and I wonder if this is anti-intellectualism being in vogue. A country where the government thinks scientists are the enemy and need to be defeated. Sad.



Or where the Trump-voter whisperers on the left blame kids who are interested in learning for all the societal ills of America and absolves the rest as innocent victims who cannot be expected to engage with society. This ignores that most of the elite were born into their wealth and have nice diplomas because of the wealth of their parents rather than their yearning for learning.

It is the failure of the front-row kids (elites, political-class, wealthy, etc), who have empowered, enriched, & removed themselves 6/

— Chris Arnade (@Chris_arnade) 28 July 2017

While I do not see an evil machine overlord ruling over humanity or destroying us, machine intelligence could be a game changer in several ways. Many, at least in newspapers, worry about its influence on the labour market and the creation of mass unemployment. This is possible, but I worry about this a lot less, it is just another step towards more automation and the additional efficiency has just made us more affluent. As far as I can see, we do not understand where unemployment comes from (apart from a small part of it due to changing jobs; [[frictional unemployment]]), so I am surprised that people are confident in making unemployment predictions, especially predictions into the far future.

One would expect that people worrying about mass unemployment would advocate shifting the tax burden away from labour. Making labour cheaper should increase demand. Alternatives would be taxing pollution instead. A reduction in environmental damages and better health would be additional economic benefits next to less unemployment or better wages.

Machine intelligence can change the balance of power. It is most worthwhile to invest in automation of large professions that serve needs for the coming decades. These are the professions everyone knows, which helps fuel the media scare. It will be a long time before someone invests money to make [[bell founders]] redundant. These kinds of jobs are not well known, but combined a decent part of the economy and in future likely even more. Collective bargaining is harder for these kinds of jobs, so labour may lose over capital, but these are also jobs where it is is hard to find replacements, where trust and good relations are important, so it could also be that labour wins over capital.

A recent survey of experts in machine intelligence predicted that in 2049 (pardon the accuracy) bestsellers will be written by computers and 11 years from now create a song that makes the US top 40. I do not believe this one bit. I would be happy to buy a book on coding in FORTRAN written by a computer, but when it comes to novels or a book on politics, I want to hear from a human. The computational methods I use to generate climate time series can also be used to generate pleasing music. That could have been a career option, but I would have hidden that the music was composed by a computer. Otherwise no one would have listened to it more than once. It may provide cheap background music in a supermarket.



Many jobs also need a lot more than just intelligence: sales people, doctors and teachers. At least for fast food workers it would have been easy to automate their jobs decades ago, but people prefer food made by humans handed to them by humans. Even simple restaurant now often have an open kitchen to show that the food it cooked by humans and not just nuked factory food.

If intelligence becomes a commodity that you can buy, the current bonus on the labour market for smart people may be gone. That was anyway just a recent invention. It would be interesting to see how that changes science; intelligence is an important skill for a scientist, but there are many more important ones. Also now a smarter colleague is often happy to do some complicated specialised task.

When worrying about overlords, a more sensible option would be to worry about humans aided by machine intelligence. Looking at ISIS and their "Christian" counterparts is seems that evil people are not particularly intelligent or creative. It could be dangerous if such people could buy their missing intelligence at Amazon. On the other hand maybe there is a reason for the anti-correlations, the more intelligent humans will be less sure of themselves and fundamentalism may disappear.

Initially likely only the elite can afford to buy more intelligence, but we would probably move quite quickly into a regime where everyone has such an add-on and intelligence just becomes normal and nearly worthless.



The main robots to worry about are the amoral machines we invented to create money. Corporations evolved with the aim of gaining money and power. They die, merge, split up and need to survive to make money. As long as they were small and made money by efficiently producing better goods and services within the bounds of the law they did a wonderful job, now they have grown large and started looking for political power. Corrupting the political system is an efficient was to grain money and power. When amoral robots do so, this may not end well for humans who are already squeezed out like lemons.

[UPDATE. I did not have to write this post, it has all been said before. I just listened to an EconTalk interview by Russ Roberts interviewing machine learning expert Pedro Domingos. Good to hear AI researchers seem to agree with me, that AI wiping us out is mainly Hollywood.

Russ Roberts: I love when you wrote--here's another quote from the book:

People worry that computers will get too smart and take over the world. But the real problem is they are too stupid, and they've already taken over the world.

Explain what you mean by that, and why you're not worried about some of the issues we've raised on this program before, with Nicholas Bostrom and others, that AI (artificial intelligence) is perhaps the greatest threat to humanity; machine learning could destroy the world; etc.

Pedro Domingos: Well, exactly. I think those worries are misguided, and frankly, I don't know too many, actually, AI researchers who take them too seriously. They are based on this confusion between AIs and people. Because humans are the only intelligent creatures on earth, when we think about intelligence we tend to confuse it with being human. But, being intelligent and being human are very different things. In Hollywood movies, the AIs and the robots are always humans in disguise. But the real AIs and robots are very different from humans, notably because they don't have goals of their own. People have this model of there will be a different set of agents who are competing with us for control of the planet. They are not going to be competing with us for anything, because we set their goals. Their intelligence is only being applied to solve the problems that we set them to solve, like cure cancer. And there, the more intelligent they are, the better.
]

Related reading

PBS: "Will we be wiped out by machine overlords? Maybe we need a game plan now"

BBC: "The automation resistant skills we should nurture"

Big Think: "Here's When Machines Will Take Your Job, as Predicted by AI Gurus"

The survey itself: "When Will AI Exceed Human Performance? Evidence from AI Experts"

Motherboard: How Garry Kasparov Learned to Stop Worrying and Love AI


* Photo Corpo Automi Robot by Bruno Cordioli used under a Creative Commons Attribution 2.0 Generic (CC BY 2.0) licence.