Free our climate data - from Geneva to Paris

Neglecting to monitor the harm done to nature and the environmental impact of our decisions is only the most striking sign of a disregard for the message contained in the structures of nature itself.
Pope Francis

The 17th Congress of the World Meteorological Organization in Geneva ended today. After countless hours of discussions they managed to pass a almost completely rewritten resolution on sharing climate data in the last hour.

The glass is half full. On the one hand, the resolution clearly states the importance of sharing data. It demonstrates that it is important to help humanity cope with climate change by making it part of the global framework for climate services (GFCS), which is there to help all nations to adapt to climate change.

The resolution considers and recognises:

The fundamental importance of the free and unrestricted exchange of GFCS relevant data and products among WMO Members to facilitate the implementation of the GFCS and to enable society to manage better the risks and opportunities arising from climate variability and change, especially for those who are most vulnerable to climate-related hazards...

That increased availability of, and access to, GFCS relevant data, especially in data sparse regions, can lead to better quality and will create a greater variety of products and services...

Indeed free and unrestricted access to data can and does facilitate innovation and the discovery of new ways to use, and purposes for, the data.

On the other hand, if a country wants to it can still refuse to share the most important datasets: the historical station observations. Many datasets will be shared: Satellite data and products, ocean and cryosphere (ice) observations, measurements on the composition of the atmosphere (including aerosols). However, information on streamflow, lakes and most of the climate station data are exempt.

The resolution does urge Members to:

Strengthen their commitment to the free and unrestricted exchange of GFCS relevant data and products;

Increase the volume of GFCS relevant data and products accessible to meet the needs for implementation of the GFCS and the requirements of the GFCS partners;

But there is no requirement to do so.

The most positive development is not on paper. Data sharing may well have been the main discussion topic among the directors of the national weather services at the Congress. They got the message that many of them find this important and they are likely to prioritise data sharing in future. I am grateful to the people at the WMO Congress who made this happen, you know who you are. Some directors really wanted to have a strong resolution as justification towards their governments to open up the databases. There is already a trend towards more and more countries opening up their archives, not only of climate data, but going towards open governance. Thus I am confident that many more countries will follow this trend after this Congress.

Also good about the resolution is that WMO will start monitoring data availability and data policies. This will make visible how many countries are already taking the high road and speed up the opening of the datasets. The resolution requests WMO to:

Monitor the implementation of policies and practices of this Resolution and, if necessary, make proposals in this respect to the Eighteenth World Meteorological Congress;

In a nice twist the WMO calls the data to be shared: GFCS data. Thus basically saying, if you do not share climate data you are responsible for the national damages of climatic changes that you could have adapted to and you are responsible for the failed adaptation investments. The term "GFSC data" misses how important this data is for basic climate research. Research that is important to guide expensive political decisions on mitigation and in the end again adaptation and ever more likely geo-engineering.

If I may repeat myself, we really need all the data we can get for an accurate assessment of climatic changes, a few stations will not do:

To reduce the influence of measurement errors and non-climatic changes (inhomogeneities) on our (trend) assessments we need dense networks. These errors are detected and corrected by comparing one station to its neighbours. The closer the neighbours are, the more accurate we can assess the real climatic changes. This is especially important when it comes to changes in severe and extreme weather, where the removal of non-climatic changes is very challenging.

The problem, as so often, is mainly money. Weather services get some revenues from selling climate data. These can't be big compared to the impacts of climate change or compared to the investments needed to adapt, but relative to the budget of a weather service, especially in poorer countries, it does make a difference. At least the weather services will have to ask their governments for permission.

Thus we will probably have to up our game. The mandate of the weather services is not enough, we need to make clear to the governments of this world that sharing climate data is of huge benefit to every single country. Compared to the costs of climate change this is a no-brainer. Don Henry writes that "[The G7] also said they would continue efforts to provide US$100 billion a year by 2020 to support developing countries' own climate actions." The revenues from selling climate data are irrelevant compared to that number.

There is just a large political climate summit coming up, the COP21 in Paris in December. This week there was a preparatory meeting in Bonn to work on the text of the climate treaty. This proposal already has an optional text about climate research:

[Industrialised countries] and those Parties [nations] in a position to do so shall support the [Least Developed Countries] in the implementation of national adaptation plans and the development of additional activities under the [Least Developed Countries] work programme, including the development of institutional capacity by establishing regional institutions to respond to adaptation needs and strengthen climate-related research and systematic observation for climate data collection, archiving, analysis and modelling.

An earlier climate treaty (COP4 from 1998) already speaks about the exchange of climate data (FCCC/CP/1998/16/Add.1):

Urges Parties to undertake free and unrestricted exchange of data to meet the needs of the Convention, recognizing the various policies on data exchange of relevant international and intergovernmental organizations;

"Urges" is not enough, but that is a basis that could be reinforced. With the kind of money COP21 is dealing with it should be easy to support weather services of less wealthy countries to improve their observation systems and make the data freely available. That would be an enormous win-win situation.

To make this happen, we probably need to show that the climate science community stands behind this. We would need a group of distinguished climate scientists from as much countries as possible to support a "petition" requesting better measurements in data sparse regions and free and unrestricted data sharing.

To get heard we would probably also need to write articles for the national newspapers, to get published they would again have to be written by well-known scientists. To get attention it would also be great if many climate blogs would write about the action on the same day.

Maybe we could make this work. My impression was already that basically everyone in the climate science community finds the free exchange of climate data very important and the current situation a major impediment for better climate research. After last weeks article on data sharing the response was enormous and only positive. This may have been the first time that a blog post of mine that did not respond to something in the press got over 1000 views. It was certainly my first tweet that got over 13 thousand views and 100 retweets:

Please RT if you want the World Meteorological Organization to share climate data. #meteoworld http://t.co/C8qD0p6YXx pic.twitter.com/X4zymbIYJ6

— Victor Venema (@VariabilityBlog) June 3, 2015

This action of my little homogenization blog was even in the top of the twitter page on the Congress of the WMO (#MeteoWorld), right next to the photo of the newly elected WMO Secretary-General Petteri Taalas.



With all this internet enthusiasm and the dedication of the people fighting for free data at the WMO and likely many more outside of the WMO, we may be able to make this work. If you would like to stay informed please fill in the form below or write to me. If enough people show interest, I feel we should try. I also do not have the time, but this is important.

Tweet

Related reading

Congress of the World Meteorological Organization, free our climate data

Why raw temperatures show too little global warming

Everything you need to know about the Paris climate summit and UN talks

Bonn climate summit brings us slowly closer to a global deal by Don Henry (Public Policy Fellow, Melbourne Sustainable Society Institute at University of Melbourne) at The Conversation.

Free climate data action promoted in Italian. Thank you Sylvie Coyaud.

If my Italian is good enough, that is Google Translate, this post wants the Pope to put the sharing of climate data in his encyclical. Weather data is a common good.


* Photo at the top: By Royal Air Force official photographer [Public domain], via Wikimedia Commons

Congress of the World Meteorological Organization, free our climate data

A small revolution happens at the World Meteorological Organization (WMO). Its main governing body (WMO congress) is discussing a draft resolution that national weather services shall provide free and unrestricted access to climate data. The problem is the fine print. The fine print makes it possible to keep on refusing to share important climate data with each other.

The data situation is getting better, more and more countries are freeing their climate data. The USA, Canada and Australia have a long traditions. Germany, The Netherlands, Finland, Sweden, Norway, Slovenia, Brazil and Israel have just freed their data. China and Russia are pretty good with sharing data. Switzerland has concrete plans to free their data. I probably forgot many countries and for Israel you currently still have to be able to read Hebrew, but things are definitely improving.

That there are large differences between countries is illustrated by this map of data availability for daily mean temperature data in the ECA&D database, a dataset that is used to study changes in severe weather. The green dots are data where you can download and work with the station data, the red dots are data that ECA&D are only allowed to use internally to make maps. In the number of stations available you can clearly see many national boundaries; that is not just the number of real stations, but to a large part national policies on data sharing.

Sharing data is important

We need this data to see what is happening to the climate. We already had almost a degree of global warming and are likely in for at least another one. This will change the sea level, the circulation, precipitation patterns. This will change extreme and severe weather. We will need to adapt to these climatic changes and to know how to protect our communities we need climate data.

Many countries have set up Climate Service Centres or are in the process of doing so to provide their populations with the information they need to adapt. Here companies, (local) governments, non-governmental organisation and citizens can get advice on how to prepare themselves for climate change.

It makes a large difference how often we will see heat waves like the one in [[Europe in 2003]] (70 thousand additional deaths; Robine et al., 2008), in [[Russia in 2010]] (a death toll of 55,000, a crop failure of ~25% and an economic loss of about 1% of the GBP; Barriopedro et al., 2011) or now in India. It makes a large difference how often a [[winter flood like in the UK in 2013-2014]] or [[the flood now in Texas and Oklahoma]] will occur. Once every 10, 100 or 1000 years? If it is 10 years, expensive infrastructural changes will be needed, if it is 1000 years, we will probably decide to life with that. It makes a difference how long droughts like the ones in California or in Chile will last and being able to make regional climate prediction requires high-quality historical climate data.

One of the main outcomes of the current 17th WMO congress will be the adoption of the Global Framework on Climate Services (GFCS). A great initiative to make sure that everyone benefits from climate services, but how will the GFCS framework succeed in helping humanity cope with climate change if there is almost no data to work with?

In their own resolution (8.1) on GFCS, the Congress recognizes this themselves:

Congress noted that EC-66 had adopted a value proposition for the international exchange of climate data and products to support the implementation of the GFCS and recommended a draft resolution on this topic for consideration by Congress.

To understand climate, we need a global overview. National studies are not enough. To understand changes in circulation, interactions with mountains and vegetation, to understand changes in extremes, we need spatially resolved information and not just a few stations.

Homogenization

To reduce the influence of measurement errors and non-climatic changes (inhomogeneities) on our (trend) assessments we need dense networks. These errors are detected and corrected by comparing one station to its neighbours. The closer the neighbours are, the more accurate we can assess the real climatic changes. This is especially important when it comes to changes in severe and extreme weather, where the removal of non-climatic changes is very challenging.

For the global mean land temperature the non-climatic changes already represent 25% of the change: After homogenization (to reduce non-climatic changes) in GHCNv3 the trend is 0.8°C per century since 1880 (Lawrimore et al., 2011; table 4). In the raw data this trend is only 0.6°C per century. That makes a large difference for our assessment how far climate change has progressed, while for large parts of the world we currently do not have enough data to remove such non-climatic changes well. This results in large uncertainties.

This 25% is global, but when in comes to the impacts of climate change, we need reliable local information. Locally the (trend) biases are much larger; on a global scale many biases cancel each ohter. For (decadal) climate prediction we need accurate variability on annual time scales, not "just" secular trends, this is again harder and has larger uncertainties. In the German climate prediction project MiKlip it was shown that a well-homogenized radiosonde dataset was able to distinguish much better between prediction systems and thus to better guide the development. Based on the physics of the non-climatic change we expect that (trend) biases are much stronger for extremes than they are for the mean. For example, errors due to insolation are worst on hot, sunny and calm days, while they are much less a problem on normal cloudy and windy days and thus less of a problem for the average. For the best possible data to protect our communities, we need dense networks, we need all the data there is.

WMO resolution

Theoretically the data exchange resolution will free everything you ever dreamed of. A large number of datasets is mentioned from satellites to sea and lake level, from greenhouse gases to snow cover and river ice. But exactly for the historical climate station that is so important to put climate change into a perspective a limitation is made. Here the international exchange is limited to the [[GCOS]] Stations. The total number of GCOS Stations is 1017 (March 01, 2014). For comparison, Berkeley Earth and the International Surface Temperature Initiative have records with more than 30 thousand stations. And most GCOS stations are likely already included in that. Thus in the end, this resolution will free almost no new climate station data.

The resolution proposes to share “all available data”. But they define that basically as data that is currently already open:

“All available” means that the originators of the data can make them available under this resolution. The term recognizes the rights of Members to choose the manner by, and the extent to, which they make their climate relevant data and products available domestically and for international exchange, taking into consideration relevant international instruments and national policies and legislation.

I have not heard of cases where national weather services denied access to data just for the fun of it. Normally they say it is due to "national policies and legislation". Thus this resolution will not change much.

No idea where these counterproductive national policies come from. For new instruments, for expensive satellites, for the [[Argo system]] to measure the ocean heat content, it is normally specified that the data should be open to all so that society maximally benefits from the investment. In America they naturally see the data as free for all because the tax payer has already paid for it.

In the past there may have been strategic (military) concerns. Climate and weather information can determine wars. However, nowadays weather and climate models are so good that the military benefit of observations is limited. Had Napoleon had a climate model, his troops would have been given warmer cloths before leaving for Russia. To prepare for war you do not need it more accurate than that.

The ministers of finance seems to like the revenues from selling climate data, but I cannot imagine them making much money that way. It is nothing in comparison to the impacts of climate changes or the costs of maladaptation. It will be much less than the money society invested in the climate observations. An investment that is devalued by sitting on the data and not sharing it.

All that while the WMO theoretically recognises how important sharing data is. In another resolution (9.1), ironically on big data, they write:

With increasing acceptance that the climate is changing, Congress noted that Members are again faced with coming to agreement with respect to the international exchange of data of importance of free and unrestricted access to climate-related information at both global and regional levels.

UN and Data Revolution. In August 2014 UN Secretary-General Ban Ki-moon asked an Independent Expert Advisory Group to make concrete recommendations on bringing about a data revolution in sustainable development (). The report indicates that too often existing data remain unused because they are released too late or not at all, not well-documented and harmonized, or not available at the level of detail needed for decision-making. More diverse, integrated, timely and trustworthy information can lead to better decision-making and real-time citizen feedback.

All that while citizen scientists are building up huge meteorological networks in Japan and North America. The citizen scientists are happy to share their data and the weather services should fear that their closed datasets will soon become a laughing stock.

Free our climate data

My apologies when this post sounds angry. I am angry. If that is reason to fire me as chair of the Task Team on Homogenization of the WMO Commission for Climatology, so be it. I cannot keep my mouth shut while this is happing.

Even if this resolution is a step forward and I am grateful for the people who made this happen, it is impossible that in these times the weather services of the world do not do everything they can to protect the communities they work for and freely share all climate data internationally. I really cannot understand how the limited revenues from selling data can seriously be seen as a reason to accept huge societal losses from climate change impacts and maladaptation.

Don't ask me how to solve this deadlock, but WMO Congress it is your job to solve this. You have until Friday next week the 12th of June.

[UPDATE. It might not be visible here because there are only little comments, but this post is read a lot for one without a connection to the mass media. That often happens below science posts that do not say something controversial. (All the scientists I know see data exchange as holding climate science back.) Also the tweet to this post is popular, never had one like this before, please retweet it to show your support for the free exchange of climate data.

Please RT if you want the World Meteorological Organization to share climate data. #meteoworld http://t.co/C8qD0p6YXx pic.twitter.com/X4zymbIYJ6

— Victor Venema (@VariabilityBlog) June 3, 2015

]

[UPDATE. Wow, the above tweet has now been seen over 7,000 times (4th June; 19h). Not bad for a small station data blog. Never seen anything like this. Also Sylvie Coyaud blogs at La Repubblica now reports about freeing climate data (in Italian). If there are journalists in Geneva, do ask the delegates about sharing data, especially when they present the Global Framework for Climate Services as the prime outcome of this WMO Congress.]

Related reading

Nature published a column by Martin Bobrow of the Expert Advisory Group on Data Access, which just wrote a report on the governance of scientific data access: Funders must encourage scientists to share.

Why raw temperatures show too little global warming

Just the facts, homogenization adjustments reduce global warming

New article: Benchmarking homogenisation algorithms for monthly data

Statistical homogenisation for dummies

New article: Benchmarking homogenisation algorithms for monthly data

A framework for benchmarking of homogenisation algorithm performance on the global scale - Paper now published