SST validation test
UPDATE* looks like the error difference in method is theirs. Not sure what led to it, but the value they have for the month is -.17C and unweighted average is around -.45C. If you area weight the temps, you get -.47C. Month 222 from 1850
UPDATE: the do the NH and SH separately and take the mean
I suppose I should look at the other months, but reported the issue to them
The first pass at a validation test is complete. Basically, we read in the HADSST2 dataset which is a 5×5 SST dataset from 1850 to present, by month. The early years are somewhat sparse in terms of temperature data. The process of validation consisted of very simply reading in the netCDF file that hadley make available, then calculating an average for each month and comparing against their published monthly numbers.
A Positive error indicates that for that month Hadley is warmer than I calculate. For the most part the errors are rather small but early in the period, there are some puzzling differences. The differences should not be that large for such a simple calculation, so, some double checking is in order. When I get a chance. The overall mean difference between them and me is on the order of LT .01C. Given that trig is involved and area on the sphere, there is no assurance of an exact match. Since they don’t have published code I cannot see how they did their area weighting. Anyway, I can investigate how raster calculates area ( did that) and I can do a simple test or two to see the issue. In the grand scheme of things, its not that big.
Finally, I’m going to switch out my Land/Water mask for a Land/Ocean Mask. Then I will create an SST file of area weighted anomalies. There is no need to do this math more than once. I will play around with writing out a netCDF.
In the End, we should end up with a big old raster brick for the land data by month and another raster brick of the SST data. Both area weighted based on land/ocean percentages in each cell.
Then you add the bricks. (sum each layer from one with the same layer in the other). And you have a monthly series.
Thinking that I might want to build such a thing at a high resolution..and then one can just use raster aggregate. There is a 1 degree SST datafile. Building a 1 degree land file is simple.
Other thoughts are to look closely at coastal locations.