Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quantiles without Sampling #1445

Closed
FunkyFabe opened this issue Dec 26, 2022 · 2 comments · Fixed by #1811
Closed

Quantiles without Sampling #1445

FunkyFabe opened this issue Dec 26, 2022 · 2 comments · Fixed by #1811
Labels
triage Issue waiting for triaging

Comments

@FunkyFabe
Copy link

Problem:
Calculating the quantiles of a probabilistic forecast requires sampling.
When working with parameterised likelihood models such das the gaussian likelihood, one must use the num_samples parameter in the predict method to obtain samples. Then one can calculate the quantiles from the probabilistic time series.
In case of parameterised likelihood models one could directly calculate the the quantiles and avoid da calculation for sampling.
What is the advantage of sampling?

Proposed Solution:
Extend the predict API to calculate quantiles using the internal estimated parameters of the distribution.

Alternative Solution:
Make raw outputs of the neural network accessible.

@FunkyFabe FunkyFabe added the triage Issue waiting for triaging label Dec 26, 2022
@hrzn
Copy link
Contributor

hrzn commented Dec 26, 2022

Storing samples allows TimeSeries to be completely distribution-agnostic and work for any kind of distribution, parametric or not. Furthermore, storing parameters (e.g. of a Gaussian) would not scale as soon as the dimension becomes large (in time and/or number of components), due to the covariance matrix requiring O(N^2) parameters. Finally, samples usually incur only a very small performance penalty (their computation is vectorized), and no significant loss of accuracy for a distribution like the Gaussian.

@TueLindhart
Copy link

In my use case, I specifically need to predict exact quantiles. When I use sampling I also gain obviously wrong quantiles where the quantiles are drifting to values that are obviously wrong. I tried to use three models where I used 0.14, 0.5 and 0.86 quantiles. However, I was not allowed to do this either. I think you should seriously consider to allow gaining the raw quantiles. Otherwise, you should at least make sampling work properly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
triage Issue waiting for triaging
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants