Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DOC] Model callouts and descriptions #359

Merged
merged 6 commits into from
May 20, 2024
Merged

[DOC] Model callouts and descriptions #359

merged 6 commits into from
May 20, 2024

Conversation

MMenchero
Copy link
Contributor

Description

  • Added callouts for tutorials 09-20.
  • Extended descriptions for the introductory pages on training, validation, uncertainty quantification, and special topics.

@review-notebook-app Review Notebook App
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

@github-actions GitHub Actions
Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 11.4565 6.3255 0.0079 0.0043

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 2.5777 2.0988 0.0051 0.0043

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 178.293 196.363 269.23 1331.02
mape 0.0234 0.0234 0.0304 0.1692
mse 121588 123119 213677 4.68961e+06
total_time 3.3356 5.5434 0.0072 0.0064

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 465.532 353.528 398.956 1119.26
mape 0.062 0.0454 0.0512 0.1583
mse 835120 422332 656723 3.17316e+06
total_time 2.9454 2.2291 0.0072 0.0063

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 558.649 361.033 602.926 1340.95
mape 0.0697 0.046 0.0787 0.17
mse 1.22721e+06 441118 1.61572e+06 6.04619e+06
total_time 4.2792 2.9211 0.007 0.0065

Plot:

@github-actions GitHub Actions
Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.935 199.132 2571.33 10604.2
total_time 2.5147 1.9302 0.0079 0.0042

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 2.0376 2.1839 0.0052 0.0043

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 178.293 196.363 269.23 1331.02
mape 0.0234 0.0234 0.0304 0.1692
mse 121588 123119 213677 4.68961e+06
total_time 3.2393 2.2711 0.007 0.0063

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 465.532 353.528 398.956 1119.26
mape 0.062 0.0454 0.0512 0.1583
mse 835120 422332 656723 3.17316e+06
total_time 4.1726 2.5377 0.0068 0.0062

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 558.649 361.033 602.926 1340.95
mape 0.0697 0.046 0.0787 0.17
mse 1.22721e+06 441118 1.61572e+06 6.04619e+06
total_time 5.8042 4.8135 0.007 0.0063

Plot:

AzulGarza
AzulGarza requested changes May 15, 2024
View reviewed changes
Copy link
Member

@AzulGarza AzulGarza left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @MMenchero! i deployed the readme dot com docs using this branch and apparently there is a strange behavior with

``` python
...

i think we can use the style in #357 (review). wdyt?

@github-actions GitHub Actions
Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 4.964 5.1404 0.0079 0.0044

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 3.3617 3.255 0.0051 0.0043

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 178.293 268.121 269.23 1331.02
mape 0.0234 0.0311 0.0304 0.1692
mse 121588 219457 213677 4.68961e+06
total_time 3.3294 3.6069 0.0072 0.0063

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 465.532 346.984 398.956 1119.26
mape 0.062 0.0437 0.0512 0.1583
mse 835121 403787 656723 3.17316e+06
total_time 4.3433 4.0274 0.007 0.0064

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 558.649 459.769 602.926 1340.95
mape 0.0697 0.0566 0.0787 0.17
mse 1.22721e+06 739135 1.61572e+06 6.04619e+06
total_time 6.9628 4.7964 0.0071 0.0066

Plot:

@MMenchero
Copy link
Contributor Author

Ready new format @AzulGarza.

@github-actions GitHub Actions
Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.935 199.132 2571.33 10604.2
total_time 3.0561 2.0175 0.0079 0.0043

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.22 4110.79 5928.17 18859.2
total_time 2.0995 1.9161 0.0049 0.0042

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 178.293 268.121 269.23 1331.02
mape 0.0234 0.0311 0.0304 0.1692
mse 121588 219457 213677 4.68961e+06
total_time 1.951 3.5829 0.0073 0.0063

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 465.532 346.984 398.956 1119.26
mape 0.062 0.0437 0.0512 0.1583
mse 835120 403787 656723 3.17316e+06
total_time 3.8154 2.0847 0.0067 0.0063

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 558.649 459.769 602.926 1340.95
mape 0.0697 0.0566 0.0787 0.17
mse 1.22721e+06 739135 1.61572e+06 6.04619e+06
total_time 6.2749 2.5635 0.007 0.0064

Plot:

@AzulGarza AzulGarza self-requested a review May 17, 2024 06:56
AzulGarza
AzulGarza approved these changes May 17, 2024
View reviewed changes
Copy link
Member

@AzulGarza AzulGarza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you @MMenchero!

could we use @mergenthaler's suggestion:

> 👍 Use an Azure AI endpoint
>
> To use an Azure AI endpoint, set the `base_url` argument:
>
> `nixtla_client = NixtlaClient(base_url="you azure ai endpoint", api_key="your api_key")`

this simplifies the callout and makes the code more readable.

@AzulGarza AzulGarza self-requested a review May 17, 2024 07:00
AzulGarza
AzulGarza requested changes May 17, 2024
View reviewed changes
Copy link
Member

@AzulGarza AzulGarza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, approved by error

@MMenchero MMenchero force-pushed the hotfix/model-callouts branch from f19de72 to 218e712 Compare May 17, 2024 17:54
@github-actions GitHub Actions
Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 2.0169 1.732 0.0077 0.0042

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 1.435 1.6047 0.005 0.0044

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 178.293 268.121 269.23 1331.02
mape 0.0234 0.0311 0.0304 0.1692
mse 121588 219457 213677 4.68961e+06
total_time 1.5382 1.9199 0.0072 0.0063

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 465.532 346.984 398.956 1119.26
mape 0.062 0.0437 0.0512 0.1583
mse 835120 403787 656723 3.17316e+06
total_time 2.3917 1.781 0.0068 0.0064

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 558.649 459.769 602.926 1340.95
mape 0.0697 0.0566 0.0787 0.17
mse 1.22721e+06 739135 1.61572e+06 6.04619e+06
total_time 3.9213 1.7525 0.007 0.0064

Plot:

@github-actions GitHub Actions
Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 1.859 3.1556 0.0082 0.0044

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.22 4110.79 5928.17 18859.2
total_time 4.5163 2.7896 0.0052 0.0045

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 178.293 268.121 269.23 1331.02
mape 0.0234 0.0311 0.0304 0.1692
mse 121588 219457 213677 4.68961e+06
total_time 3.3135 3.1362 0.0075 0.0065

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 465.532 346.984 398.956 1119.26
mape 0.062 0.0437 0.0512 0.1583
mse 835120 403787 656723 3.17316e+06
total_time 4.0966 2.6318 0.0081 0.0087

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 558.649 459.769 602.926 1340.95
mape 0.0697 0.0566 0.0787 0.17
mse 1.22721e+06 739135 1.61572e+06 6.04619e+06
total_time 6.3491 2.7408 0.007 0.0065

Plot:

@MMenchero
Copy link
Contributor Author

Ready requested changes @AzulGarza

@AzulGarza AzulGarza self-requested a review May 20, 2024 19:49
@AzulGarza AzulGarza merged commit db6aa64 into main May 20, 2024
14 checks passed
@AzulGarza AzulGarza deleted the hotfix/model-callouts branch May 20, 2024 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@AzulGarza AzulGarza AzulGarza approved these changes

@mergenthaler mergenthaler Awaiting requested review from mergenthaler

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@MMenchero @AzulGarza