Difference in TrendAnalysis filtering behavior between power=0 and power=nan

[kandersolar]

Changing TrendAnalysis(pv=power, ...) to TrendAnalysis(pv=power.replace(0, np.nan), ...) can change the degradation output. There are at least two differences in behavior between power=0 and power=np.nan, both of which occur in the pre-aggregation filtering steps:

1. In rdtools.clipping.quantile_clip_filter the clipping threshold is calculated as the 98th percentile of power_ac. This ignores power_ac=np.nan but not power_ac=0. With power_ac=0 the threshold tends to be lower and the filter excludes more points.
2. In TrendAnalysis._pvwatts_norm, energy_normalized is optionally renormalized by dividing by its 95th percentile. Times with pv_energy=0 and nonzero irradiance affect this 95th percentile, but times where pv_energy=np.nan do not. By changing this renormalization constant, the entire normalized_energy series is scaled up or down and points get moved into or out of the allowed range specified in rdtools.filtering.normalized_filter.

Note that the clipping effect applies to any power=0 while the normalization effect only applies to power=0 during daylight hours (i.e., when expected production is nonzero).

Here's a modified version of the PVDAQ4 TrendAnalysis notebook showing how the clipping threshold varies (856 to 819 in this case): https://gist.github.com/kanderso-nrel/ea025bf4ea94397b16865e3fcacd9d88

I don't yet have a good example for the second effect.

Marking this as a bug since it seems to me that both of these are undesirable effects. Thanks @williamhobbs for noticing the difference in output!

[mdeceglie]

We should be careful about changing answers for users relying on the default behavior. Let's add a kwarg, whose default value triggers a deprecation warning and a suggestion. The preferred option for that kwarg would use some minimum as a threshold for inclusion in the quantile calculations. We'll use the preferred option in updated examples.

[mdeceglie]

Thinking this through a little more, for the minimum threshold calculation let's first replace all NaN with 0, then use a fraction of a high percentile to set the minimum.

[mdeceglie]

New plan: address this bug in the examples for now, then implement a more robust but result-changing fix in RdTools 3