Fix forecasting lead times and improve forecasting functionality #132
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR improves forecasting functionality in DeepSensor:
TaskLoader
withtarget_delta_t
containing values >0 would havemodel.predict
only return the longest lead time (if there ar multiple lead times).time
dim inmodel.predict
forecasts would only correspond to the initialisation time, which is confusing. This PR adds aninit_time
andlead_time
dim alongside atime
coordinate (which is the valid time of the forecast, which is more intuitive and enables comparison with non-forecast ground truth with a singletime
dim).deepsensor.eval.metrics.compute_errors
method which takes thePrediction
returned bymodel.predict
and computes the differences between the"mean"
entry of model predictions with those in a target dataset for each variable in thePrediction
, returning anxarray.Dataset
of errors from which mean statistics can be computed.Google Colab demonstrator here.
Notes:
model.predict
only works if the target variables are the same for each target set. If this isn't the case,model.predict
raises an error. This is primarily for implementation simplicity.target_delta_t > 0
, and then does some slightly ugly conditional logic to return predictions with forecasting dimensions. I prioritised getting unit tests working than doing something more elegant.Closes #130
Addresses one of the desiderata of #30