Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Algorithm] TD3 fast #2389

Open
wants to merge 13 commits into
base: gh/vmoens/17/base
Choose a base branch
from
Open

[Algorithm] TD3 fast #2389

wants to merge 13 commits into from

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Aug 10, 2024 •
edited
Loading

Stack from ghstack (oldest at bottom):

@vmoens
Update
0b1a592 
[ghstack-poisoned]
This was referenced Aug 10, 2024
Merged
Merged
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Aug 10, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2389

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 4 Unrelated Failures

As of commit 5222c24 with merge base 2b975da (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@vmoens vmoens mentioned this pull request Aug 10, 2024
Merged
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 10, 2024
vmoens added a commit that referenced this pull request Aug 10, 2024
@vmoens
[Algorithm] TD3 fast
d83e976 
ghstack-source-id: e1a6c874a6794afd650de2bf44bb483a3f814c12
Pull Request resolved: #2389
@vmoens
Update
be2559b 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Aug 10, 2024
@vmoens
[Algorithm] TD3 fast
3a45e93 
ghstack-source-id: e97238cc93c58c9d61fa42c7432e18aa8ad01957
Pull Request resolved: #2389
@vmoens
Update
2f98caf 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Aug 10, 2024
@vmoens
[Algorithm] TD3 fast
18ff0bb 
ghstack-source-id: 9dcb1cb4286c984f60d4bb126e8aaa17e75c2a75
Pull Request resolved: #2389
vmoens added a commit that referenced this pull request Aug 10, 2024
@vmoens
[Algorithm] TD3 fast
ae6f4b2 
ghstack-source-id: 9dcb1cb4286c984f60d4bb126e8aaa17e75c2a75
Pull Request resolved: #2389
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 10, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 58.9850ms 58.4849ms 17.0984 Ops/s 16.8666 Ops/s $\color{#35bf28}+1.37\%$
test_sync 34.8829ms 32.9676ms 30.3329 Ops/s 27.7583 Ops/s $\textbf{\color{#35bf28}+9.27\%}$
test_async 68.2330ms 31.4476ms 31.7990 Ops/s 31.5159 Ops/s $\color{#35bf28}+0.90\%$
test_simple 0.5086s 0.4258s 2.3485 Ops/s 2.2862 Ops/s $\color{#35bf28}+2.72\%$
test_transformed 0.6652s 0.5866s 1.7047 Ops/s 1.6933 Ops/s $\color{#35bf28}+0.67\%$
test_serial 1.3708s 1.2844s 0.7785 Ops/s 0.7751 Ops/s $\color{#35bf28}+0.45\%$
test_parallel 1.2736s 1.1503s 0.8693 Ops/s 0.8849 Ops/s $\color{#d91a1a}-1.76\%$
test_step_mdp_speed[True-True-True-True-True] 0.2421ms 24.8969μs 40.1656 KOps/s 39.3139 KOps/s $\color{#35bf28}+2.17\%$
test_step_mdp_speed[True-True-True-True-False] 87.2050μs 14.4551μs 69.1796 KOps/s 67.9819 KOps/s $\color{#35bf28}+1.76\%$
test_step_mdp_speed[True-True-True-False-True] 34.4060μs 14.3378μs 69.7455 KOps/s 68.3255 KOps/s $\color{#35bf28}+2.08\%$
test_step_mdp_speed[True-True-True-False-False] 60.3240μs 8.3033μs 120.4339 KOps/s 118.1025 KOps/s $\color{#35bf28}+1.97\%$
test_step_mdp_speed[True-True-False-True-True] 56.6270μs 26.3853μs 37.8998 KOps/s 37.1004 KOps/s $\color{#35bf28}+2.15\%$
test_step_mdp_speed[True-True-False-True-False] 68.7800μs 15.8222μs 63.2024 KOps/s 61.5396 KOps/s $\color{#35bf28}+2.70\%$
test_step_mdp_speed[True-True-False-False-True] 44.4250μs 15.8429μs 63.1199 KOps/s 62.4260 KOps/s $\color{#35bf28}+1.11\%$
test_step_mdp_speed[True-True-False-False-False] 67.9510μs 9.7878μs 102.1676 KOps/s 99.9707 KOps/s $\color{#35bf28}+2.20\%$
test_step_mdp_speed[True-False-True-True-True] 98.9460μs 28.0360μs 35.6684 KOps/s 34.2462 KOps/s $\color{#35bf28}+4.15\%$
test_step_mdp_speed[True-False-True-True-False] 72.4340μs 17.4903μs 57.1744 KOps/s 56.1374 KOps/s $\color{#35bf28}+1.85\%$
test_step_mdp_speed[True-False-True-False-True] 85.4810μs 15.7574μs 63.4623 KOps/s 61.8678 KOps/s $\color{#35bf28}+2.58\%$
test_step_mdp_speed[True-False-True-False-False] 61.4790μs 9.8013μs 102.0274 KOps/s 100.2578 KOps/s $\color{#35bf28}+1.77\%$
test_step_mdp_speed[True-False-False-True-True] 63.4290μs 29.7753μs 33.5849 KOps/s 33.1599 KOps/s $\color{#35bf28}+1.28\%$
test_step_mdp_speed[True-False-False-True-False] 69.8910μs 19.2118μs 52.0514 KOps/s 52.0714 KOps/s $\color{#d91a1a}-0.04\%$
test_step_mdp_speed[True-False-False-False-True] 63.5900μs 17.3406μs 57.6682 KOps/s 56.8524 KOps/s $\color{#35bf28}+1.44\%$
test_step_mdp_speed[True-False-False-False-False] 40.6260μs 11.3247μs 88.3026 KOps/s 86.8562 KOps/s $\color{#35bf28}+1.67\%$
test_step_mdp_speed[False-True-True-True-True] 90.8980μs 28.1703μs 35.4984 KOps/s 34.0855 KOps/s $\color{#35bf28}+4.15\%$
test_step_mdp_speed[False-True-True-True-False] 54.4020μs 17.6544μs 56.6431 KOps/s 54.8640 KOps/s $\color{#35bf28}+3.24\%$
test_step_mdp_speed[False-True-True-False-True] 79.7090μs 18.4987μs 54.0577 KOps/s 53.9772 KOps/s $\color{#35bf28}+0.15\%$
test_step_mdp_speed[False-True-True-False-False] 74.3390μs 11.0371μs 90.6038 KOps/s 88.4622 KOps/s $\color{#35bf28}+2.42\%$
test_step_mdp_speed[False-True-False-True-True] 0.1095ms 29.6356μs 33.7432 KOps/s 33.4072 KOps/s $\color{#35bf28}+1.01\%$
test_step_mdp_speed[False-True-False-True-False] 61.2150μs 19.0094μs 52.6056 KOps/s 52.0046 KOps/s $\color{#35bf28}+1.16\%$
test_step_mdp_speed[False-True-False-False-True] 0.1031ms 19.7150μs 50.7228 KOps/s 49.4940 KOps/s $\color{#35bf28}+2.48\%$
test_step_mdp_speed[False-True-False-False-False] 54.4320μs 12.5293μs 79.8130 KOps/s 78.1550 KOps/s $\color{#35bf28}+2.12\%$
test_step_mdp_speed[False-False-True-True-True] 3.4416ms 31.5475μs 31.6982 KOps/s 31.3359 KOps/s $\color{#35bf28}+1.16\%$
test_step_mdp_speed[False-False-True-True-False] 82.0740μs 20.7208μs 48.2607 KOps/s 47.9052 KOps/s $\color{#35bf28}+0.74\%$
test_step_mdp_speed[False-False-True-False-True] 54.4620μs 19.8806μs 50.3002 KOps/s 49.4713 KOps/s $\color{#35bf28}+1.68\%$
test_step_mdp_speed[False-False-True-False-False] 72.5460μs 12.5737μs 79.5312 KOps/s 78.3567 KOps/s $\color{#35bf28}+1.50\%$
test_step_mdp_speed[False-False-False-True-True] 81.6030μs 32.4487μs 30.8179 KOps/s 30.1226 KOps/s $\color{#35bf28}+2.31\%$
test_step_mdp_speed[False-False-False-True-False] 84.3680μs 21.9748μs 45.5066 KOps/s 44.8951 KOps/s $\color{#35bf28}+1.36\%$
test_step_mdp_speed[False-False-False-False-True] 50.2550μs 20.7951μs 48.0882 KOps/s 46.5738 KOps/s $\color{#35bf28}+3.25\%$
test_step_mdp_speed[False-False-False-False-False] 68.8790μs 13.7104μs 72.9373 KOps/s 70.5107 KOps/s $\color{#35bf28}+3.44\%$
test_values[generalized_advantage_estimate-True-True] 12.7701ms 9.9486ms 100.5170 Ops/s 100.2965 Ops/s $\color{#35bf28}+0.22\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.7334ms 35.7027ms 28.0091 Ops/s 27.7147 Ops/s $\color{#35bf28}+1.06\%$
test_values[td0_return_estimate-False-False] 0.2646ms 0.1873ms 5.3384 KOps/s 5.4294 KOps/s $\color{#d91a1a}-1.68\%$
test_values[td1_return_estimate-False-False] 24.3905ms 24.1039ms 41.4870 Ops/s 40.8651 Ops/s $\color{#35bf28}+1.52\%$
test_values[vec_td1_return_estimate-False-False] 40.8671ms 36.1517ms 27.6612 Ops/s 27.8117 Ops/s $\color{#d91a1a}-0.54\%$
test_values[td_lambda_return_estimate-True-False] 45.3965ms 35.2345ms 28.3813 Ops/s 28.4165 Ops/s $\color{#d91a1a}-0.12\%$
test_values[vec_td_lambda_return_estimate-True-False] 40.9167ms 36.3754ms 27.4911 Ops/s 27.6379 Ops/s $\color{#d91a1a}-0.53\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.5587ms 8.3324ms 120.0130 Ops/s 119.1332 Ops/s $\color{#35bf28}+0.74\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.3076ms 1.9930ms 501.7624 Ops/s 513.3529 Ops/s $\color{#d91a1a}-2.26\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6371ms 0.3675ms 2.7207 KOps/s 2.7592 KOps/s $\color{#d91a1a}-1.39\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 53.8551ms 48.6012ms 20.5756 Ops/s 21.7105 Ops/s $\textbf{\color{#d91a1a}-5.23\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.7631ms 3.1780ms 314.6603 Ops/s 322.0772 Ops/s $\color{#d91a1a}-2.30\%$
test_dqn_speed 6.5343ms 1.2846ms 778.4443 Ops/s 769.1306 Ops/s $\color{#35bf28}+1.21\%$
test_ddpg_speed 3.5546ms 2.6897ms 371.7868 Ops/s 371.0890 Ops/s $\color{#35bf28}+0.19\%$
test_sac_speed 9.0210ms 8.2356ms 121.4237 Ops/s 118.6983 Ops/s $\color{#35bf28}+2.30\%$
test_redq_speed 14.7110ms 13.0513ms 76.6206 Ops/s 75.9926 Ops/s $\color{#35bf28}+0.83\%$
test_redq_deprec_speed 15.7086ms 14.1841ms 70.5017 Ops/s 73.6790 Ops/s $\color{#d91a1a}-4.31\%$
test_td3_speed 9.9997ms 8.6215ms 115.9892 Ops/s 120.7115 Ops/s $\color{#d91a1a}-3.91\%$
test_cql_speed 40.1541ms 37.3109ms 26.8018 Ops/s 25.1092 Ops/s $\textbf{\color{#35bf28}+6.74\%}$
test_a2c_speed 10.8564ms 7.8332ms 127.6617 Ops/s 129.6596 Ops/s $\color{#d91a1a}-1.54\%$
test_ppo_speed 9.2395ms 8.5131ms 117.4665 Ops/s 124.2124 Ops/s $\textbf{\color{#d91a1a}-5.43\%}$
test_reinforce_speed 11.7801ms 6.9010ms 144.9068 Ops/s 145.7036 Ops/s $\color{#d91a1a}-0.55\%$
test_iql_speed 34.7618ms 33.0746ms 30.2347 Ops/s 30.5623 Ops/s $\color{#d91a1a}-1.07\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.4055ms 5.1174ms 195.4107 Ops/s 194.8221 Ops/s $\color{#35bf28}+0.30\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.1117s 0.5723ms 1.7473 KOps/s 1.9977 KOps/s $\textbf{\color{#d91a1a}-12.53\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7353ms 0.4719ms 2.1192 KOps/s 2.1312 KOps/s $\color{#d91a1a}-0.56\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.9399ms 5.1900ms 192.6785 Ops/s 197.3196 Ops/s $\color{#d91a1a}-2.35\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.4094ms 0.5014ms 1.9944 KOps/s 2.0514 KOps/s $\color{#d91a1a}-2.78\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6773ms 0.4805ms 2.0813 KOps/s 2.1167 KOps/s $\color{#d91a1a}-1.67\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 3.0980ms 1.7988ms 555.9314 Ops/s 579.6809 Ops/s $\color{#d91a1a}-4.10\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.2061ms 1.6588ms 602.8326 Ops/s 615.4158 Ops/s $\color{#d91a1a}-2.04\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.4617ms 5.7968ms 172.5084 Ops/s 190.0850 Ops/s $\textbf{\color{#d91a1a}-9.25\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.4606ms 0.6571ms 1.5217 KOps/s 1.5837 KOps/s $\color{#d91a1a}-3.91\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9208ms 0.6225ms 1.6065 KOps/s 1.6313 KOps/s $\color{#d91a1a}-1.52\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 8.3557ms 5.6893ms 175.7678 Ops/s 194.1930 Ops/s $\textbf{\color{#d91a1a}-9.49\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.7076ms 0.5203ms 1.9219 KOps/s 1.9985 KOps/s $\color{#d91a1a}-3.84\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6773ms 0.4787ms 2.0890 KOps/s 2.1280 KOps/s $\color{#d91a1a}-1.83\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.8883ms 5.2093ms 191.9656 Ops/s 195.7010 Ops/s $\color{#d91a1a}-1.91\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.8249ms 0.5016ms 1.9937 KOps/s 2.0390 KOps/s $\color{#d91a1a}-2.22\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7107ms 0.4870ms 2.0534 KOps/s 2.0983 KOps/s $\color{#d91a1a}-2.14\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.6674ms 5.4443ms 183.6777 Ops/s 189.4703 Ops/s $\color{#d91a1a}-3.06\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8294ms 0.6412ms 1.5595 KOps/s 1.5663 KOps/s $\color{#d91a1a}-0.44\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 7.7998ms 0.6287ms 1.5906 KOps/s 1.6223 KOps/s $\color{#d91a1a}-1.95\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1385s 9.1891ms 108.8250 Ops/s 150.8078 Ops/s $\textbf{\color{#d91a1a}-27.84\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 19.0556ms 13.8602ms 72.1489 Ops/s 72.5339 Ops/s $\color{#d91a1a}-0.53\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.4677ms 1.3864ms 721.3163 Ops/s 724.4627 Ops/s $\color{#d91a1a}-0.43\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1281s 6.5735ms 152.1268 Ops/s 112.6598 Ops/s $\textbf{\color{#35bf28}+35.03\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 19.2379ms 14.3049ms 69.9061 Ops/s 71.7588 Ops/s $\color{#d91a1a}-2.58\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.1930ms 1.3725ms 728.5902 Ops/s 764.5426 Ops/s $\color{#d91a1a}-4.70\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1282s 6.7505ms 148.1370 Ops/s 148.0206 Ops/s $\color{#35bf28}+0.08\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1364s 16.5910ms 60.2736 Ops/s 71.7802 Ops/s $\textbf{\color{#d91a1a}-16.03\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.0537ms 1.5263ms 655.1757 Ops/s 639.4935 Ops/s $\color{#35bf28}+2.45\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 10, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1067s 0.1058s 9.4512 Ops/s 9.5051 Ops/s $\color{#d91a1a}-0.57\%$
test_sync 97.9573ms 93.5944ms 10.6844 Ops/s 11.2472 Ops/s $\textbf{\color{#d91a1a}-5.00\%}$
test_async 0.2582s 88.8182ms 11.2590 Ops/s 11.8146 Ops/s $\color{#d91a1a}-4.70\%$
test_single_pixels 0.1166s 0.1159s 8.6258 Ops/s 8.5034 Ops/s $\color{#35bf28}+1.44\%$
test_sync_pixels 77.1107ms 75.5225ms 13.2411 Ops/s 13.4257 Ops/s $\color{#d91a1a}-1.37\%$
test_async_pixels 0.1395s 70.1450ms 14.2562 Ops/s 14.0635 Ops/s $\color{#35bf28}+1.37\%$
test_simple 0.7475s 0.7467s 1.3391 Ops/s 1.3079 Ops/s $\color{#35bf28}+2.39\%$
test_transformed 1.0745s 0.9997s 1.0003 Ops/s 1.0216 Ops/s $\color{#d91a1a}-2.09\%$
test_serial 2.2107s 2.1367s 0.4680 Ops/s 0.4748 Ops/s $\color{#d91a1a}-1.43\%$
test_parallel 1.9362s 1.8693s 0.5350 Ops/s 0.5354 Ops/s $\color{#d91a1a}-0.09\%$
test_step_mdp_speed[True-True-True-True-True] 0.2041ms 37.6209μs 26.5809 KOps/s 26.4082 KOps/s $\color{#35bf28}+0.65\%$
test_step_mdp_speed[True-True-True-True-False] 0.2200ms 21.5275μs 46.4521 KOps/s 47.4946 KOps/s $\color{#d91a1a}-2.20\%$
test_step_mdp_speed[True-True-True-False-True] 37.0320μs 21.0667μs 47.4684 KOps/s 47.5812 KOps/s $\color{#d91a1a}-0.24\%$
test_step_mdp_speed[True-True-True-False-False] 26.5410μs 12.0588μs 82.9269 KOps/s 82.6642 KOps/s $\color{#35bf28}+0.32\%$
test_step_mdp_speed[True-True-False-True-True] 67.6640μs 39.7469μs 25.1592 KOps/s 24.8797 KOps/s $\color{#35bf28}+1.12\%$
test_step_mdp_speed[True-True-False-True-False] 41.7030μs 23.7027μs 42.1892 KOps/s 42.6481 KOps/s $\color{#d91a1a}-1.08\%$
test_step_mdp_speed[True-True-False-False-True] 37.4220μs 23.5393μs 42.4822 KOps/s 43.0851 KOps/s $\color{#d91a1a}-1.40\%$
test_step_mdp_speed[True-True-False-False-False] 31.1920μs 14.3332μs 69.7682 KOps/s 69.7841 KOps/s $\color{#d91a1a}-0.02\%$
test_step_mdp_speed[True-False-True-True-True] 0.1948ms 42.7659μs 23.3831 KOps/s 23.6476 KOps/s $\color{#d91a1a}-1.12\%$
test_step_mdp_speed[True-False-True-True-False] 41.6430μs 26.3225μs 37.9904 KOps/s 38.6894 KOps/s $\color{#d91a1a}-1.81\%$
test_step_mdp_speed[True-False-True-False-True] 41.4820μs 23.2739μs 42.9667 KOps/s 42.9174 KOps/s $\color{#35bf28}+0.11\%$
test_step_mdp_speed[True-False-True-False-False] 37.6630μs 14.4014μs 69.4375 KOps/s 69.4722 KOps/s $\color{#d91a1a}-0.05\%$
test_step_mdp_speed[True-False-False-True-True] 0.2432ms 44.9153μs 22.2641 KOps/s 22.4500 KOps/s $\color{#d91a1a}-0.83\%$
test_step_mdp_speed[True-False-False-True-False] 46.5030μs 28.3041μs 35.3305 KOps/s 35.7879 KOps/s $\color{#d91a1a}-1.28\%$
test_step_mdp_speed[True-False-False-False-True] 54.3230μs 25.4812μs 39.2446 KOps/s 39.4372 KOps/s $\color{#d91a1a}-0.49\%$
test_step_mdp_speed[True-False-False-False-False] 39.1230μs 16.5038μs 60.5921 KOps/s 60.6644 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[False-True-True-True-True] 62.0840μs 42.5175μs 23.5197 KOps/s 23.5617 KOps/s $\color{#d91a1a}-0.18\%$
test_step_mdp_speed[False-True-True-True-False] 41.2620μs 26.0483μs 38.3902 KOps/s 38.8105 KOps/s $\color{#d91a1a}-1.08\%$
test_step_mdp_speed[False-True-True-False-True] 51.9630μs 27.7686μs 36.0120 KOps/s 35.7942 KOps/s $\color{#35bf28}+0.61\%$
test_step_mdp_speed[False-True-True-False-False] 32.8220μs 16.2859μs 61.4029 KOps/s 61.7780 KOps/s $\color{#d91a1a}-0.61\%$
test_step_mdp_speed[False-True-False-True-True] 91.7860μs 44.3881μs 22.5285 KOps/s 22.5745 KOps/s $\color{#d91a1a}-0.20\%$
test_step_mdp_speed[False-True-False-True-False] 43.0630μs 28.1419μs 35.5342 KOps/s 35.6360 KOps/s $\color{#d91a1a}-0.29\%$
test_step_mdp_speed[False-True-False-False-True] 50.0730μs 29.6540μs 33.7222 KOps/s 33.4813 KOps/s $\color{#35bf28}+0.72\%$
test_step_mdp_speed[False-True-False-False-False] 98.4560μs 18.4564μs 54.1818 KOps/s 54.4693 KOps/s $\color{#d91a1a}-0.53\%$
test_step_mdp_speed[False-False-True-True-True] 3.9616ms 47.7763μs 20.9309 KOps/s 21.2124 KOps/s $\color{#d91a1a}-1.33\%$
test_step_mdp_speed[False-False-True-True-False] 74.9050μs 30.8486μs 32.4164 KOps/s 32.8300 KOps/s $\color{#d91a1a}-1.26\%$
test_step_mdp_speed[False-False-True-False-True] 54.2330μs 29.9693μs 33.3675 KOps/s 34.0209 KOps/s $\color{#d91a1a}-1.92\%$
test_step_mdp_speed[False-False-True-False-False] 36.2720μs 18.4386μs 54.2340 KOps/s 54.3973 KOps/s $\color{#d91a1a}-0.30\%$
test_step_mdp_speed[False-False-False-True-True] 66.3950μs 49.4661μs 20.2159 KOps/s 20.5281 KOps/s $\color{#d91a1a}-1.52\%$
test_step_mdp_speed[False-False-False-True-False] 48.4630μs 32.6755μs 30.6040 KOps/s 30.6757 KOps/s $\color{#d91a1a}-0.23\%$
test_step_mdp_speed[False-False-False-False-True] 55.4230μs 31.7542μs 31.4919 KOps/s 31.4918 KOps/s $+0.00\%$
test_step_mdp_speed[False-False-False-False-False] 40.4830μs 20.3662μs 49.1009 KOps/s 48.9102 KOps/s $\color{#35bf28}+0.39\%$
test_values[generalized_advantage_estimate-True-True] 23.5657ms 22.8292ms 43.8036 Ops/s 44.0249 Ops/s $\color{#d91a1a}-0.50\%$
test_values[vec_generalized_advantage_estimate-True-True] 0.1030s 2.9442ms 339.6565 Ops/s 353.8742 Ops/s $\color{#d91a1a}-4.02\%$
test_values[td0_return_estimate-False-False] 94.1760μs 63.8607μs 15.6591 KOps/s 15.7235 KOps/s $\color{#d91a1a}-0.41\%$
test_values[td1_return_estimate-False-False] 54.0922ms 51.9907ms 19.2342 Ops/s 19.0698 Ops/s $\color{#35bf28}+0.86\%$
test_values[vec_td1_return_estimate-False-False] 1.3821ms 1.0677ms 936.5530 Ops/s 942.5769 Ops/s $\color{#d91a1a}-0.64\%$
test_values[td_lambda_return_estimate-True-False] 82.7827ms 82.2944ms 12.1515 Ops/s 12.1981 Ops/s $\color{#d91a1a}-0.38\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.4741ms 1.0581ms 945.1252 Ops/s 943.5911 Ops/s $\color{#35bf28}+0.16\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 23.0915ms 22.7849ms 43.8887 Ops/s 44.0203 Ops/s $\color{#d91a1a}-0.30\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9174ms 0.6967ms 1.4354 KOps/s 1.3965 KOps/s $\color{#35bf28}+2.78\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8020ms 0.6432ms 1.5547 KOps/s 1.5625 KOps/s $\color{#d91a1a}-0.50\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6010ms 1.4465ms 691.3227 Ops/s 693.4972 Ops/s $\color{#d91a1a}-0.31\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8084ms 0.6572ms 1.5215 KOps/s 1.5298 KOps/s $\color{#d91a1a}-0.54\%$
test_dqn_speed 7.7923ms 1.3468ms 742.5170 Ops/s 696.3938 Ops/s $\textbf{\color{#35bf28}+6.62\%}$
test_ddpg_speed 3.0168ms 2.7519ms 363.3817 Ops/s 352.8005 Ops/s $\color{#35bf28}+3.00\%$
test_sac_speed 0.1027s 8.6612ms 115.4579 Ops/s 123.4165 Ops/s $\textbf{\color{#d91a1a}-6.45\%}$
test_redq_speed 12.8558ms 10.4046ms 96.1113 Ops/s 93.8884 Ops/s $\color{#35bf28}+2.37\%$
test_redq_deprec_speed 11.9023ms 11.4747ms 87.1484 Ops/s 88.7254 Ops/s $\color{#d91a1a}-1.78\%$
test_td3_speed 7.9110ms 7.8131ms 127.9895 Ops/s 124.8204 Ops/s $\color{#35bf28}+2.54\%$
test_cql_speed 26.7238ms 25.5739ms 39.1023 Ops/s 38.3196 Ops/s $\color{#35bf28}+2.04\%$
test_a2c_speed 5.9590ms 5.6807ms 176.0344 Ops/s 171.2051 Ops/s $\color{#35bf28}+2.82\%$
test_ppo_speed 6.4412ms 6.0185ms 166.1553 Ops/s 161.9298 Ops/s $\color{#35bf28}+2.61\%$
test_reinforce_speed 6.1882ms 4.6490ms 215.0987 Ops/s 211.0409 Ops/s $\color{#35bf28}+1.92\%$
test_iql_speed 20.3684ms 19.8223ms 50.4482 Ops/s 49.4048 Ops/s $\color{#35bf28}+2.11\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.0292ms 6.8098ms 146.8469 Ops/s 148.1347 Ops/s $\color{#d91a1a}-0.87\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9560ms 0.5244ms 1.9071 KOps/s 1.8633 KOps/s $\color{#35bf28}+2.35\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7305ms 0.5021ms 1.9917 KOps/s 1.9437 KOps/s $\color{#35bf28}+2.47\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.9573ms 6.5994ms 151.5294 Ops/s 149.8325 Ops/s $\color{#35bf28}+1.13\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.9198ms 0.5130ms 1.9492 KOps/s 1.8915 KOps/s $\color{#35bf28}+3.05\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6656ms 0.4916ms 2.0342 KOps/s 1.9482 KOps/s $\color{#35bf28}+4.42\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.1267ms 1.9555ms 511.3669 Ops/s 502.1151 Ops/s $\color{#35bf28}+1.84\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 5.8264ms 1.8651ms 536.1544 Ops/s 522.1968 Ops/s $\color{#35bf28}+2.67\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.0551ms 6.8757ms 145.4400 Ops/s 145.8854 Ops/s $\color{#d91a1a}-0.31\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1346s 0.7849ms 1.2741 KOps/s 1.4591 KOps/s $\textbf{\color{#d91a1a}-12.68\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8725ms 0.6521ms 1.5336 KOps/s 1.5087 KOps/s $\color{#35bf28}+1.65\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.8722ms 6.7006ms 149.2412 Ops/s 149.4498 Ops/s $\color{#d91a1a}-0.14\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.6367ms 0.5293ms 1.8895 KOps/s 1.8709 KOps/s $\color{#35bf28}+0.99\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6783ms 0.5039ms 1.9845 KOps/s 1.9454 KOps/s $\color{#35bf28}+2.01\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.9870ms 6.6166ms 151.1349 Ops/s 148.7085 Ops/s $\color{#35bf28}+1.63\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7258ms 0.5203ms 1.9219 KOps/s 1.8862 KOps/s $\color{#35bf28}+1.90\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7317ms 0.4975ms 2.0099 KOps/s 1.9452 KOps/s $\color{#35bf28}+3.33\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.0623ms 6.8422ms 146.1514 Ops/s 145.7052 Ops/s $\color{#35bf28}+0.31\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1468ms 0.6731ms 1.4856 KOps/s 1.4668 KOps/s $\color{#35bf28}+1.28\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8203ms 0.6460ms 1.5480 KOps/s 1.5008 KOps/s $\color{#35bf28}+3.14\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1386s 8.0082ms 124.8724 Ops/s 93.3137 Ops/s $\textbf{\color{#35bf28}+33.82\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 18.8396ms 16.2853ms 61.4050 Ops/s 61.6603 Ops/s $\color{#d91a1a}-0.41\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.2676ms 1.2630ms 791.7588 Ops/s 639.3422 Ops/s $\textbf{\color{#35bf28}+23.84\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1317s 10.4072ms 96.0877 Ops/s 122.6531 Ops/s $\textbf{\color{#d91a1a}-21.66\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 18.5952ms 16.2034ms 61.7155 Ops/s 61.2399 Ops/s $\color{#35bf28}+0.78\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.5627ms 1.3586ms 736.0379 Ops/s 721.0929 Ops/s $\color{#35bf28}+2.07\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1304s 8.0029ms 124.9540 Ops/s 123.4616 Ops/s $\color{#35bf28}+1.21\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 19.1096ms 16.5077ms 60.5778 Ops/s 60.6562 Ops/s $\color{#d91a1a}-0.13\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.7599ms 1.5919ms 628.1612 Ops/s 609.3727 Ops/s $\color{#35bf28}+3.08\%$

@vmoens
Update
990fdbf 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Aug 11, 2024
@vmoens
[Algorithm] TD3 fast
cbb8a54 
ghstack-source-id: b31466e061d8ac79c5507edb05f48b9de21ecb80
Pull Request resolved: #2389
vmoens added a commit that referenced this pull request Aug 11, 2024
@vmoens
[Algorithm] TD3 fast
1c60f5e 
ghstack-source-id: b31466e061d8ac79c5507edb05f48b9de21ecb80
Pull Request resolved: #2389
@vmoens
Update
32935fc 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Aug 11, 2024
@vmoens
[Algorithm] TD3 fast
771c1e0 
ghstack-source-id: ffe06895bd4fa3a7e0c567a1bd6d09d05cb815b2
Pull Request resolved: #2389
vmoens added a commit that referenced this pull request Aug 11, 2024
@vmoens
[Algorithm] TD3 fast
2340537 
ghstack-source-id: ffe06895bd4fa3a7e0c567a1bd6d09d05cb815b2
Pull Request resolved: #2389
@vmoens
Update
b17b1b7 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Aug 11, 2024
@vmoens
[Algorithm] TD3 fast
9b9be22 
ghstack-source-id: 81318d0195e86eeef23fd5d3f97371537e1ded13
Pull Request resolved: #2389
@vmoens
Update
6234035 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Aug 11, 2024
@vmoens
[Algorithm] TD3 fast
632a4ae 
ghstack-source-id: f9e083333792bfc7730468128f533d9f3806764e
Pull Request resolved: #2389
@vmoens
Update
650e0ec 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Aug 12, 2024
@vmoens
[Algorithm] TD3 fast
6d4ea6d 
ghstack-source-id: 6385b70f732f00bd2aebf24a37e402cbaa30b6d5
Pull Request resolved: #2389
@vmoens
Update
00bf70f 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Aug 12, 2024
@vmoens
[Algorithm] TD3 fast
17f0b44 
ghstack-source-id: 16037b74edf8c66efec9c5aad1d6713ff2635762
Pull Request resolved: #2389
@vmoens
Update
1f8e35b 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Aug 12, 2024
@vmoens
[Algorithm] TD3 fast
ce585e6 
ghstack-source-id: 0da09f43487593d500f1cd1937529e27f21c2e03
Pull Request resolved: #2389
@vmoens
Update
72d30d9 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Aug 13, 2024
@vmoens
[Algorithm] TD3 fast
e296582 
ghstack-source-id: b2bd14c1338929c3c28be5085da7f7b33ae4c4ef
Pull Request resolved: #2389
@vmoens
Update
6bd52f6 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Aug 13, 2024
@vmoens
[Algorithm] TD3 fast
88a3fa7 
ghstack-source-id: f5eb65b604a37d971c6464eacfb5534dbc7fcc5c
Pull Request resolved: #2389
vmoens added a commit that referenced this pull request Aug 13, 2024
@vmoens
[Algorithm] TD3 fast
72fd931 
ghstack-source-id: f5eb65b604a37d971c6464eacfb5534dbc7fcc5c
Pull Request resolved: #2389
@vmoens
Update
5222c24 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Aug 13, 2024
@vmoens
[Algorithm] TD3 fast
aa5f624 
ghstack-source-id: 00ba39b75e7570f64c4572fe8a7e6a03548d8d46
Pull Request resolved: #2389
vmoens added a commit that referenced this pull request Aug 13, 2024
@vmoens
[Algorithm] TD3 fast
517e3fc 
ghstack-source-id: 00ba39b75e7570f64c4572fe8a7e6a03548d8d46
Pull Request resolved: #2389
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot