tag:blogger.com,1999:blog-9093436161326155359.post3639445947907913050..comments2024-03-28T06:43:02.954+00:00Comments on Variable Variability: Sad that for Lamar Smith the "hiatus" has far-reaching policy implications Victor Venemahttp://www.blogger.com/profile/02842816166712285801noreply@blogger.comBlogger4125tag:blogger.com,1999:blog-9093436161326155359.post-57320664617876418382015-11-29T20:11:19.463+00:002015-11-29T20:11:19.463+00:00Frank, you need to estimate the right value of the...Frank, you need to estimate the right value of the your biases b1, b2, and b3. That was exactly what NOAA now did in their updated dataset. They took the bias between buoys and ship inlet measurements into account, which they had not done before, but which previous work by the UK Hadley Centre showed to be important. <br /><br />Not sure if the term "methodologically homogeneous" is clearly defined. However the satellite temperatures are surely not homogeneous and need large adjustments. The earliest dataset that did not have these yet even erroneously showed a cooling trend. The new update of UAHv5 to the unpublished UAHv6 again makes rather large adjustments. I would not call that a-priory homogeneous and would not assume that the m (of your mt) would be the same of the other datasets or the m of reality. <br /><br />I agree that we need error estimate for remaining inhomogeneities, not just for interpolation. The surface temperature dataset HadCRUT partially does so. I do not know of such an estimates for the satellite temperatures; possibly my lack of expertise of these estimates, if you know of any please let me know. <br /><br />Estimating the error introduced by unknown unknowns is hard. For the land station temperatures this may be possible because you have so much redundant information (many stations measuring the same regional climate signal), for the satellite or SST temperatures the unknown unknowns are a challenge to say the least.Victor Venemahttps://www.blogger.com/profile/02842816166712285801noreply@blogger.comtag:blogger.com,1999:blog-9093436161326155359.post-8913464549026990922015-11-29T18:18:40.148+00:002015-11-29T18:18:40.148+00:00Victor: Thanks for your reply. I refuse to accep...Victor: Thanks for your reply. I refuse to accept comparisons to ERSST v3 or v4 - they contain a variable amount of data from buoys. Given the simple linear problem I presented above, why would you overlay any of the y1, y2, y3 records with a composite containing them (especially a somewhat mysterious composite you hadn't created. The hypothesis we need to test concerns b1, b2 and b3. <br /><br />There are METHODOLOGICALLY homogeneous data sets. Satellite microwave temperature data is a homogeneous data set, but I don't know if it is useful because of interference from changing aerosols. The change across a long period with similar aerosols at both ends could be useful. For the periods of when buoys were being introduced, I suspect there is enough metadata about engine intake temperatures to create a more homogeneous data set. <br /><br />One viable interpretation of this episode is that we need to add uncertainty to the central estimate of the trend in for warming that represents uncertainty that the right values for bias correction (b1, b2 .. bn) have been identified. The existing error bars reflect random noise common to all composites; the difference in central value represents uncertainty in bias correction - until someone unambiguously proves which is best. <br /><br />FrankAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-9093436161326155359.post-29047849428444094922015-11-28T21:44:46.345+00:002015-11-28T21:44:46.345+00:00Frank, the comparison of the trend of all ERSST da...Frank, the comparison of the trend of all ERSST data with the trend from buoys is a good sanity check. I do not work on SST, but I would not a-priory call the buoy dataset homogeneous, just likely more homogeneous than the full dataset.<br /><br />The problem is there are nearly no homogeneous datasets. So much has happened economically, technically and socially in the last 150 years. It is very difficult to keep observation methods constant over such a long time. I hope we can do this in future, the US climate reference network is hopefully something that will spread, it was certainly not the case in the past.<br /><br />Under the assumption that there are no homogeneous datasets, what we do in statistical homogenization is similar to what you propose. We first detect breaks and then combine the data to one regional climate signal with an equation that is very similar to yours.<br /><br />We assume that the observations are given by:<br />1) A regional climate signal, which is the same for all stations (in your case: mt, we compute one value per year and compute trend at the end).<br />2) A step function with the biases (b1, b2, b3, ...)<br />3) Noise due to measurement noise or local weather.<br /><br />Then we minimize the noise to get the regional climate signal. Method described in Caussinus and Mestre (rather short, though). <br /><br />I hope that is a satisfactory answer.Victor Venemahttps://www.blogger.com/profile/02842816166712285801noreply@blogger.comtag:blogger.com,1999:blog-9093436161326155359.post-21851710545916804262015-11-28T21:23:10.412+00:002015-11-28T21:23:10.412+00:00Victor: Consider a series of linear relationship...Victor: Consider a series of linear relationships:<br /><br />y1 = mt + b1 <br />y2 = mt + b2<br />y3 = mt + b3<br />etc<br /><br />where y1, y2, y3 ... yn are difference sources of temperature data, t is time, m is the rate at which temperature is rising and b1, b2, b3 ... bn are systematic errors associated with each measurement method. We wish to prepare a composite temperature record from all yi when the amount of data coming from each measurement is changing with time. It is trivial to see that the slope of any composite temperature record prepared from yi will vary with the accuracy with which the relative biases are corrected: b1-b2, b1-b3, b1-bn. One can easily show that the slope of the composite record depends on bias correction. And there are a wide variety of estimates available about these biases. Question: How can an outsider tell if appropriate choices were made? <br /><br />Answer: Transparency comes from plotting data from homogeneous sources. For simplified data like this, one only need to plot y1 and y2 vs time on the same graph; followed by y1 and y2' vs time, where y2' is y2-(b1-b2). With the proper correction, the two lines superimpose. <br /><br />This is obviously much harder to do with real temperature data. In the early days of a new measurement technique, the coverage often isn't global. One is forced to do comparisons within grid cells. And we expect m to vary with time and latitude. <br /><br />At Judy's, Zeke and Kevin compare a buoy only record with ERSST v3 and ERSST v4, but these are composite records with potentially dubious bias corrections, not homogeneous records. In the most recent years, both composites are dominated by buoy records. So I would like to see as many homogeneous records of SST overlaid on the buoy record. And it would be useful to include satellite records of SST (which are perturbed by aerosols) and near-surface records from ARGO. Even UAH/RSS tropospheric records might have some value if their greater variability (due to low heat capacity) could be taken into account. <br /><br />Are you aware of anywhere that reliable comparisons between homogenous records have been made? <br /><br />http://judithcurry.com/2015/11/22/a-buoy-only-sea-surface-temperature-record/ <br /><br />FrankAnonymousnoreply@blogger.com