Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix forecasting lead times and improve forecasting functionality #132

Merged
merged 8 commits into from
Oct 20, 2024

Conversation

tom-andersson
Copy link
Collaborator

@tom-andersson tom-andersson commented Oct 20, 2024

This PR improves forecasting functionality in DeepSensor:

  1. Fixes a bug where models set up with a TaskLoader with target_delta_t containing values >0 would have model.predict only return the longest lead time (if there ar multiple lead times).
  2. Previously, the time dim in model.predict forecasts would only correspond to the initialisation time, which is confusing. This PR adds an init_time and lead_time dim alongside a time coordinate (which is the valid time of the forecast, which is more intuitive and enables comparison with non-forecast ground truth with a single time dim).
  3. Adds a convenient deepsensor.eval.metrics.compute_errors method which takes the Prediction returned by model.predict and computes the differences between the "mean" entry of model predictions with those in a target dataset for each variable in the Prediction, returning an xarray.Dataset of errors from which mean statistics can be computed.

Google Colab demonstrator here.

Notes:

  • Forecasting model.predict only works if the target variables are the same for each target set. If this isn't the case, model.predict raises an error. This is primarily for implementation simplicity.
  • This PR is a bit hacky. It detects if any target_delta_t > 0, and then does some slightly ugly conditional logic to return predictions with forecasting dimensions. I prioritised getting unit tests working than doing something more elegant.

Closes #130
Addresses one of the desiderata of #30

@tom-andersson tom-andersson merged commit fe696a9 into main Oct 20, 2024
12 checks passed
@tom-andersson
Copy link
Collaborator Author

Google Colab demonstrator here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Potential bug: Forecasting with model.predict does not return predictions of the future
1 participant