Home > Uncategorized > SST validation test

SST validation test

UPDATE* looks like the error difference in method is theirs. Not sure what led to it, but the value they have for the month is -.17C and unweighted average is around -.45C. If you area weight the temps, you get -.47C. Month 222 from 1850

UPDATE: the do the NH and SH separately and take the mean

“1868-06-03 12:00:00”

I suppose I should look at the other months, but reported the issue to them

The first pass at a validation test is complete. Basically, we read in the HADSST2 dataset which is a 5×5 SST dataset from 1850 to present, by month. The early years are somewhat sparse in terms of temperature data. The process of validation consisted of very simply reading in the netCDF file that hadley make available, then calculating an average for each month and comparing against their published monthly numbers.

A Positive error indicates that for that month Hadley is warmer than I calculate. For the most part the errors are rather small but early in the period, there are some puzzling differences. The differences should not be that large for such a simple calculation, so, some double checking is in order. When I get  a chance. The overall mean difference between them and me is on the order of LT .01C. Given that trig is involved and area on the sphere, there is no assurance of an exact match. Since they don’t have published code I cannot see how they did their area weighting. Anyway, I can investigate how raster calculates area ( did that) and I can do a simple test or two to see the issue. In the grand scheme of things, its not that big.

Finally, I’m going to switch out my Land/Water mask for a Land/Ocean Mask. Then I will create an SST file of area weighted anomalies. There is no need to do this math more than once.  I will play around with writing out a netCDF.

In the End, we should end up with a big old raster brick for the land data by month and another raster brick of the SST data. Both area weighted based on land/ocean percentages in each cell.

Then you add the bricks. (sum each layer from one with the same layer in the other). And you have a monthly series.

Thinking that I might want to build such a thing at a high resolution..and then one can just use raster aggregate. There is a 1 degree SST datafile. Building a 1 degree land file is simple.

Other thoughts are to look closely at coastal locations.

Categories: Uncategorized
  1. August 25, 2010 at 6:49 AM


    I thought they published perl code. Two questions, on a previous thread, you showed your area weighting by land. Land/ocean grids had less weighting due to less land area I presume. How was the land/ocean area determined – yes I prefer spoon feeding sometimes. Sorry if I missed it in the post.

    Second question, are you planning on making this a long term effort or just a temp reconstruction?

    I’ve read nearly all the posts BTW, I just comment a lot less than people realize.

  2. August 25, 2010 at 6:50 AM

    Sorry, I can’t understand myself –Second question, are you planning on making this blog into a long term effort or just a temp reconstruction?

  3. Steven Mosher
    August 25, 2010 at 8:30 AM

    On the perl code I recall that was land only, but will have to check.

    The land ocean grid. When area weighting the land I used a land water grid. That’s a grid with % of land per grid. There is a flag you can set: land.mask. When it is set to true we use the LAND AREA in a cell. so a cell that has 100Ksqkm of land ( 100%) gets a land area of 100sqkm. If a similar sized cell has 50% land, then its land area is 50Ksq km. If you set the flag to FALSE, then the land area is assumed to be equal to the cell area. That is, every cell is calculated as a 100% land.

    With the sea I plan on doing the same thing. Calcualte it both ways. the SST mask is just
    1-LAND. The first calculation I did, did not include the percent of water in each grid. Once I get the whole algorithm worked out and tested, then flipping the “percent” water is a snap.

    So in the end you have a grid that is SST area weighted, percent of water in the grid correct.
    The you have a matching land grid, area weighted, percent land correct. You simply average those two. In that case the coastal cells are the only one with two values.

    One last thing. The Land/water mask I used will be swapped out for a land/ocean. The difference being that the former has inland water. Minor difference.

  4. Steven Mosher
    August 25, 2010 at 8:34 AM

    Jeff Id :

    Sorry, I can’t understand myself –Second question, are you planning on making this blog into a long term effort or just a temp reconstruction?

    This blog I think I will just dedicate to the R work. So keeping it light and on the programming issues. BTW, I’m pretty sure I can just plumb your work and Roman’s work in.

  5. August 25, 2010 at 3:03 PM

    Brohen et al 2006 Section 4, “Blending Land and Marine Data”, is worth a re-read if you are coding the land/ocean blend. They used to use a (clamped) area weighting, but now they minimise uncertainty.

    • August 25, 2010 at 3:29 PM

      This was supposed to be with reference to comment #3 rather than the main article. Must have tea!

    • Steven Mosher
      August 25, 2010 at 7:41 PM


      The paragraph on land/ocean is one that RomanM and I have discussed for a couple years. I did more listening than contributing.
      Anyway, the coastal situation is interesting, not just for the math, but for looking at the ‘mismatch’ between rates of warming seen in the SST versus those seen on the land. The liminal is often illuminating. I think Zeke may look at coastal issues as well.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: