Adding NaN thresholding

[kjdoore]

It would be cool to add the ability to set the NaN thresholding similar to what is done in xESMF. Currently most_common and conservative have a NaN thresholding of 1, but it would be nice if a user could specify this value.

[BSchilperoort]

I am not sure how easy this will be for the conservative method. To make computations fast, we reduce the coordinates one at a time. So we can only define a nan threshold per reduction.

For example, a 3x3 grid is reduced to 1x1:

0, 1, 0
0, 0, 0
0, 0, NaN

First reduce first dim:

0.33,
0,
NaN (or 0)

Then reduce second dim:

NaN (or 0.16)

I guess the NaN threshold here can be 1/9. Which means that for 2 dimensions being reduced the NaN threshold is 1/3 per dimension 🤔

This also means that the for the second reduction the weights have to be adjusted. Currently if you'd not mask the NaNs out, the result would be 0.11 (as the NaN counts for value of 0).

[kjdoore]

Yeah, I thought it may complicate the process, especially given the current methodology. I am having trouble thinking of a way to implement this efficiently. Tracking the NaN fraction and adjusting the threshold as you move across dimensions is likely no more efficient that utilizing the traditional methodology of conservative regridding...

[BSchilperoort]

I think it is possible, but not easy to do, especially if NaNs appear over time on one lat/lon location. There would be additional overhead in the computation too.

If there are just a few NaN values in your data then interpolating those before regridding is probably the best solution.