Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] compatibility to new Composite dist log_prob/entropy APIs #2435

Merged
merged 1 commit into from
Sep 12, 2024
Merged

[BugFix] compatibility to new Composite dist log_prob/entropy APIs #2435

merged 1 commit into from
Sep 12, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Sep 12, 2024 •
edited
Loading

@vmoens
Update
040ea58 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Sep 12, 2024
@vmoens
[BugFix] compatibility to new Composite dist log_prob/entropy APIs
eafb342 
ghstack-source-id: a09b6c34000f57a66736bb9811ca3656c861ec0c
Pull Request resolved: #2435
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Sep 12, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2435

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 9 Unrelated Failures

As of commit 040ea58 with merge base d40fa4f (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 12, 2024
@vmoens
Copy link
Contributor Author

vmoens commented Sep 12, 2024

Fixes related to pytorch/tensordict#981

@vmoens vmoens added the bug Something isn't working label Sep 12, 2024
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 59.9873ms 59.7312ms 16.7417 Ops/s 16.7328 Ops/s $\color{#35bf28}+0.05\%$
test_sync 39.5321ms 32.3163ms 30.9442 Ops/s 30.9314 Ops/s $\color{#35bf28}+0.04\%$
test_async 59.8121ms 30.6716ms 32.6035 Ops/s 32.6629 Ops/s $\color{#d91a1a}-0.18\%$
test_simple 0.4988s 0.4227s 2.3659 Ops/s 2.4478 Ops/s $\color{#d91a1a}-3.35\%$
test_transformed 0.6537s 0.5855s 1.7081 Ops/s 1.7391 Ops/s $\color{#d91a1a}-1.78\%$
test_serial 1.3573s 1.2879s 0.7764 Ops/s 0.7523 Ops/s $\color{#35bf28}+3.21\%$
test_parallel 1.1923s 1.1275s 0.8869 Ops/s 0.8882 Ops/s $\color{#d91a1a}-0.15\%$
test_step_mdp_speed[True-True-True-True-True] 0.1689ms 27.6618μs 36.1509 KOps/s 37.2797 KOps/s $\color{#d91a1a}-3.03\%$
test_step_mdp_speed[True-True-True-True-False] 53.0790μs 16.0692μs 62.2307 KOps/s 61.3372 KOps/s $\color{#35bf28}+1.46\%$
test_step_mdp_speed[True-True-True-False-True] 61.3460μs 15.9368μs 62.7477 KOps/s 64.0849 KOps/s $\color{#d91a1a}-2.09\%$
test_step_mdp_speed[True-True-True-False-False] 51.3570μs 9.1790μs 108.9447 KOps/s 108.6275 KOps/s $\color{#35bf28}+0.29\%$
test_step_mdp_speed[True-True-False-True-True] 66.8950μs 29.6045μs 33.7786 KOps/s 34.4835 KOps/s $\color{#d91a1a}-2.04\%$
test_step_mdp_speed[True-True-False-True-False] 47.1980μs 17.7256μs 56.4157 KOps/s 56.7815 KOps/s $\color{#d91a1a}-0.64\%$
test_step_mdp_speed[True-True-False-False-True] 52.9090μs 17.5724μs 56.9073 KOps/s 56.8351 KOps/s $\color{#35bf28}+0.13\%$
test_step_mdp_speed[True-True-False-False-False] 28.1520μs 10.8090μs 92.5156 KOps/s 92.1710 KOps/s $\color{#35bf28}+0.37\%$
test_step_mdp_speed[True-False-True-True-True] 63.4490μs 31.1188μs 32.1350 KOps/s 32.5064 KOps/s $\color{#d91a1a}-1.14\%$
test_step_mdp_speed[True-False-True-True-False] 64.3400μs 19.5941μs 51.0357 KOps/s 51.7888 KOps/s $\color{#d91a1a}-1.45\%$
test_step_mdp_speed[True-False-True-False-True] 50.0940μs 17.3601μs 57.6035 KOps/s 58.2099 KOps/s $\color{#d91a1a}-1.04\%$
test_step_mdp_speed[True-False-True-False-False] 48.3100μs 10.8795μs 91.9163 KOps/s 92.3436 KOps/s $\color{#d91a1a}-0.46\%$
test_step_mdp_speed[True-False-False-True-True] 83.3360μs 32.6362μs 30.6408 KOps/s 31.0490 KOps/s $\color{#d91a1a}-1.31\%$
test_step_mdp_speed[True-False-False-True-False] 78.1970μs 20.8887μs 47.8729 KOps/s 47.9318 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[True-False-False-False-True] 69.1490μs 19.1427μs 52.2391 KOps/s 53.2726 KOps/s $\color{#d91a1a}-1.94\%$
test_step_mdp_speed[True-False-False-False-False] 62.9480μs 12.3578μs 80.9203 KOps/s 79.8390 KOps/s $\color{#35bf28}+1.35\%$
test_step_mdp_speed[False-True-True-True-True] 79.4390μs 31.1490μs 32.1038 KOps/s 32.7270 KOps/s $\color{#d91a1a}-1.90\%$
test_step_mdp_speed[False-True-True-True-False] 48.0100μs 19.5547μs 51.1385 KOps/s 51.9921 KOps/s $\color{#d91a1a}-1.64\%$
test_step_mdp_speed[False-True-True-False-True] 69.1700μs 20.1174μs 49.7082 KOps/s 50.3969 KOps/s $\color{#d91a1a}-1.37\%$
test_step_mdp_speed[False-True-True-False-False] 35.9470μs 12.1483μs 82.3158 KOps/s 82.9043 KOps/s $\color{#d91a1a}-0.71\%$
test_step_mdp_speed[False-True-False-True-True] 89.8380μs 32.5508μs 30.7212 KOps/s 31.0767 KOps/s $\color{#d91a1a}-1.14\%$
test_step_mdp_speed[False-True-False-True-False] 72.4150μs 21.0726μs 47.4550 KOps/s 48.2094 KOps/s $\color{#d91a1a}-1.56\%$
test_step_mdp_speed[False-True-False-False-True] 3.1792ms 21.4640μs 46.5896 KOps/s 45.7629 KOps/s $\color{#35bf28}+1.81\%$
test_step_mdp_speed[False-True-False-False-False] 65.7040μs 13.7923μs 72.5042 KOps/s 74.3787 KOps/s $\color{#d91a1a}-2.52\%$
test_step_mdp_speed[False-False-True-True-True] 73.8580μs 34.1354μs 29.2951 KOps/s 29.5360 KOps/s $\color{#d91a1a}-0.82\%$
test_step_mdp_speed[False-False-True-True-False] 70.8930μs 22.5310μs 44.3833 KOps/s 44.7103 KOps/s $\color{#d91a1a}-0.73\%$
test_step_mdp_speed[False-False-True-False-True] 71.2740μs 21.6730μs 46.1405 KOps/s 45.5381 KOps/s $\color{#35bf28}+1.32\%$
test_step_mdp_speed[False-False-True-False-False] 33.1630μs 13.7726μs 72.6077 KOps/s 74.1067 KOps/s $\color{#d91a1a}-2.02\%$
test_step_mdp_speed[False-False-False-True-True] 97.9030μs 35.9229μs 27.8374 KOps/s 28.3827 KOps/s $\color{#d91a1a}-1.92\%$
test_step_mdp_speed[False-False-False-True-False] 73.6780μs 23.8854μs 41.8666 KOps/s 41.9560 KOps/s $\color{#d91a1a}-0.21\%$
test_step_mdp_speed[False-False-False-False-True] 57.6280μs 22.8163μs 43.8282 KOps/s 43.2859 KOps/s $\color{#35bf28}+1.25\%$
test_step_mdp_speed[False-False-False-False-False] 0.1002ms 15.0938μs 66.2522 KOps/s 66.3564 KOps/s $\color{#d91a1a}-0.16\%$
test_values[generalized_advantage_estimate-True-True] 10.5615ms 9.4995ms 105.2688 Ops/s 105.6650 Ops/s $\color{#d91a1a}-0.37\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.5187ms 35.5986ms 28.0910 Ops/s 29.8068 Ops/s $\textbf{\color{#d91a1a}-5.76\%}$
test_values[td0_return_estimate-False-False] 0.2219ms 0.1684ms 5.9390 KOps/s 5.6324 KOps/s $\textbf{\color{#35bf28}+5.44\%}$
test_values[td1_return_estimate-False-False] 27.1066ms 23.6959ms 42.2014 Ops/s 41.8888 Ops/s $\color{#35bf28}+0.75\%$
test_values[vec_td1_return_estimate-False-False] 39.1946ms 35.5982ms 28.0913 Ops/s 29.7930 Ops/s $\textbf{\color{#d91a1a}-5.71\%}$
test_values[td_lambda_return_estimate-True-False] 50.8808ms 34.8137ms 28.7243 Ops/s 28.6770 Ops/s $\color{#35bf28}+0.16\%$
test_values[vec_td_lambda_return_estimate-True-False] 38.4988ms 35.7242ms 27.9922 Ops/s 29.7651 Ops/s $\textbf{\color{#d91a1a}-5.96\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.4921ms 8.2921ms 120.5969 Ops/s 120.3164 Ops/s $\color{#35bf28}+0.23\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.4113ms 1.9675ms 508.2521 Ops/s 499.6837 Ops/s $\color{#35bf28}+1.71\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5793ms 0.3572ms 2.7999 KOps/s 2.7618 KOps/s $\color{#35bf28}+1.38\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 50.0161ms 47.3753ms 21.1080 Ops/s 24.8474 Ops/s $\textbf{\color{#d91a1a}-15.05\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.7694ms 3.0360ms 329.3811 Ops/s 330.7590 Ops/s $\color{#d91a1a}-0.42\%$
test_dqn_speed 6.4428ms 1.3033ms 767.3116 Ops/s 759.8775 Ops/s $\color{#35bf28}+0.98\%$
test_ddpg_speed 3.3098ms 2.7219ms 367.3956 Ops/s 366.6985 Ops/s $\color{#35bf28}+0.19\%$
test_sac_speed 9.4608ms 7.9919ms 125.1273 Ops/s 125.7999 Ops/s $\color{#d91a1a}-0.53\%$
test_redq_speed 13.9122ms 12.9029ms 77.5021 Ops/s 78.9403 Ops/s $\color{#d91a1a}-1.82\%$
test_redq_deprec_speed 14.4173ms 12.7560ms 78.3944 Ops/s 79.1582 Ops/s $\color{#d91a1a}-0.96\%$
test_td3_speed 8.2653ms 8.0108ms 124.8316 Ops/s 123.5191 Ops/s $\color{#35bf28}+1.06\%$
test_cql_speed 37.6609ms 36.1086ms 27.6942 Ops/s 28.1666 Ops/s $\color{#d91a1a}-1.68\%$
test_a2c_speed 8.7182ms 7.2502ms 137.9273 Ops/s 136.8614 Ops/s $\color{#35bf28}+0.78\%$
test_ppo_speed 8.0189ms 7.5045ms 133.2526 Ops/s 131.8381 Ops/s $\color{#35bf28}+1.07\%$
test_reinforce_speed 8.0924ms 6.5374ms 152.9652 Ops/s 152.3788 Ops/s $\color{#35bf28}+0.38\%$
test_iql_speed 33.0219ms 31.8327ms 31.4142 Ops/s 31.3654 Ops/s $\color{#35bf28}+0.16\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.5040ms 4.7494ms 210.5535 Ops/s 204.2973 Ops/s $\color{#35bf28}+3.06\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.1076s 0.5383ms 1.8577 KOps/s 2.0829 KOps/s $\textbf{\color{#d91a1a}-10.81\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7700ms 0.4446ms 2.2491 KOps/s 2.2161 KOps/s $\color{#35bf28}+1.49\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.0083ms 4.7209ms 211.8259 Ops/s 203.6401 Ops/s $\color{#35bf28}+4.02\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.2736ms 0.4680ms 2.1369 KOps/s 2.1137 KOps/s $\color{#35bf28}+1.10\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7345ms 0.4425ms 2.2601 KOps/s 2.2599 KOps/s $+0.01\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.7357ms 1.5818ms 632.1892 Ops/s 625.6812 Ops/s $\color{#35bf28}+1.04\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.3125ms 1.4986ms 667.3084 Ops/s 662.9987 Ops/s $\color{#35bf28}+0.65\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.7197ms 4.8966ms 204.2215 Ops/s 196.9663 Ops/s $\color{#35bf28}+3.68\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9522ms 0.6061ms 1.6500 KOps/s 1.6392 KOps/s $\color{#35bf28}+0.66\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9394ms 0.5775ms 1.7315 KOps/s 1.7127 KOps/s $\color{#35bf28}+1.09\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.9535ms 4.7046ms 212.5557 Ops/s 203.2728 Ops/s $\color{#35bf28}+4.57\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.1974ms 0.4720ms 2.1186 KOps/s 2.0836 KOps/s $\color{#35bf28}+1.68\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7982ms 0.4526ms 2.2096 KOps/s 2.2359 KOps/s $\color{#d91a1a}-1.17\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.1206ms 4.6709ms 214.0913 Ops/s 204.7006 Ops/s $\color{#35bf28}+4.59\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.5962ms 0.4648ms 2.1514 KOps/s 2.0953 KOps/s $\color{#35bf28}+2.68\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 7.3278ms 0.4482ms 2.2313 KOps/s 2.2429 KOps/s $\color{#d91a1a}-0.52\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.7737ms 4.8915ms 204.4367 Ops/s 198.1831 Ops/s $\color{#35bf28}+3.16\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1083ms 0.6157ms 1.6241 KOps/s 1.5892 KOps/s $\color{#35bf28}+2.19\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7338ms 0.5767ms 1.7341 KOps/s 1.6763 KOps/s $\color{#35bf28}+3.45\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1364s 6.6687ms 149.9542 Ops/s 156.3362 Ops/s $\color{#d91a1a}-4.08\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.5182ms 12.9144ms 77.4328 Ops/s 76.8272 Ops/s $\color{#35bf28}+0.79\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.2673ms 1.3237ms 755.4382 Ops/s 789.9698 Ops/s $\color{#d91a1a}-4.37\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1301s 8.9868ms 111.2748 Ops/s 161.0518 Ops/s $\textbf{\color{#d91a1a}-30.91\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.8779ms 12.8273ms 77.9590 Ops/s 76.7867 Ops/s $\color{#35bf28}+1.53\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.9273ms 1.2357ms 809.2897 Ops/s 803.0640 Ops/s $\color{#35bf28}+0.78\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1179s 6.4288ms 155.5507 Ops/s 111.6886 Ops/s $\textbf{\color{#35bf28}+39.27\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.0870ms 13.0620ms 76.5582 Ops/s 75.1014 Ops/s $\color{#35bf28}+1.94\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.9001ms 1.3630ms 733.6533 Ops/s 730.9399 Ops/s $\color{#35bf28}+0.37\%$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1015s 0.1006s 9.9389 Ops/s 8.9856 Ops/s $\textbf{\color{#35bf28}+10.61\%}$
test_sync 92.0130ms 88.3514ms 11.3184 Ops/s 11.0925 Ops/s $\color{#35bf28}+2.04\%$
test_async 0.1662s 83.8183ms 11.9306 Ops/s 12.0440 Ops/s $\color{#d91a1a}-0.94\%$
test_single_pixels 0.1083s 0.1076s 9.2910 Ops/s 9.2590 Ops/s $\color{#35bf28}+0.35\%$
test_sync_pixels 71.9365ms 71.4961ms 13.9868 Ops/s 14.0453 Ops/s $\color{#d91a1a}-0.42\%$
test_async_pixels 0.1103s 66.2978ms 15.0834 Ops/s 14.8552 Ops/s $\color{#35bf28}+1.54\%$
test_simple 0.7139s 0.7130s 1.4025 Ops/s 1.3432 Ops/s $\color{#35bf28}+4.42\%$
test_transformed 0.9427s 0.9378s 1.0664 Ops/s 1.0517 Ops/s $\color{#35bf28}+1.39\%$
test_serial 2.0217s 2.0205s 0.4949 Ops/s 0.4878 Ops/s $\color{#35bf28}+1.45\%$
test_parallel 1.8528s 1.8260s 0.5476 Ops/s 0.5404 Ops/s $\color{#35bf28}+1.34\%$
test_step_mdp_speed[True-True-True-True-True] 0.2284ms 37.5106μs 26.6592 KOps/s 27.5996 KOps/s $\color{#d91a1a}-3.41\%$
test_step_mdp_speed[True-True-True-True-False] 59.1210μs 21.0032μs 47.6118 KOps/s 47.2454 KOps/s $\color{#35bf28}+0.78\%$
test_step_mdp_speed[True-True-True-False-True] 55.9600μs 20.9123μs 47.8187 KOps/s 47.0981 KOps/s $\color{#35bf28}+1.53\%$
test_step_mdp_speed[True-True-True-False-False] 40.7900μs 11.9400μs 83.7519 KOps/s 82.4086 KOps/s $\color{#35bf28}+1.63\%$
test_step_mdp_speed[True-True-False-True-True] 72.3910μs 39.5573μs 25.2798 KOps/s 25.4545 KOps/s $\color{#d91a1a}-0.69\%$
test_step_mdp_speed[True-True-False-True-False] 59.0200μs 22.7979μs 43.8637 KOps/s 42.4133 KOps/s $\color{#35bf28}+3.42\%$
test_step_mdp_speed[True-True-False-False-True] 60.5510μs 22.7488μs 43.9584 KOps/s 42.1538 KOps/s $\color{#35bf28}+4.28\%$
test_step_mdp_speed[True-True-False-False-False] 43.4100μs 14.0172μs 71.3410 KOps/s 69.1666 KOps/s $\color{#35bf28}+3.14\%$
test_step_mdp_speed[True-False-True-True-True] 71.2810μs 41.6938μs 23.9844 KOps/s 23.8172 KOps/s $\color{#35bf28}+0.70\%$
test_step_mdp_speed[True-False-True-True-False] 58.6300μs 25.3078μs 39.5135 KOps/s 38.8623 KOps/s $\color{#35bf28}+1.68\%$
test_step_mdp_speed[True-False-True-False-True] 52.9010μs 23.3080μs 42.9036 KOps/s 42.5916 KOps/s $\color{#35bf28}+0.73\%$
test_step_mdp_speed[True-False-True-False-False] 46.1310μs 14.1148μs 70.8478 KOps/s 69.9996 KOps/s $\color{#35bf28}+1.21\%$
test_step_mdp_speed[True-False-False-True-True] 86.1110μs 43.6111μs 22.9299 KOps/s 22.6862 KOps/s $\color{#35bf28}+1.07\%$
test_step_mdp_speed[True-False-False-True-False] 63.2610μs 27.0558μs 36.9606 KOps/s 35.7844 KOps/s $\color{#35bf28}+3.29\%$
test_step_mdp_speed[True-False-False-False-True] 73.6900μs 24.6862μs 40.5085 KOps/s 39.0819 KOps/s $\color{#35bf28}+3.65\%$
test_step_mdp_speed[True-False-False-False-False] 47.0500μs 15.9948μs 62.5205 KOps/s 60.9821 KOps/s $\color{#35bf28}+2.52\%$
test_step_mdp_speed[False-True-True-True-True] 88.5310μs 41.4153μs 24.1457 KOps/s 23.7997 KOps/s $\color{#35bf28}+1.45\%$
test_step_mdp_speed[False-True-True-True-False] 54.1610μs 25.3007μs 39.5245 KOps/s 38.7212 KOps/s $\color{#35bf28}+2.07\%$
test_step_mdp_speed[False-True-True-False-True] 61.6810μs 25.9933μs 38.4714 KOps/s 37.8332 KOps/s $\color{#35bf28}+1.69\%$
test_step_mdp_speed[False-True-True-False-False] 50.1700μs 15.7786μs 63.3769 KOps/s 62.3467 KOps/s $\color{#35bf28}+1.65\%$
test_step_mdp_speed[False-True-False-True-True] 0.1081ms 43.0130μs 23.2488 KOps/s 22.5115 KOps/s $\color{#35bf28}+3.28\%$
test_step_mdp_speed[False-True-False-True-False] 0.4066ms 27.2411μs 36.7093 KOps/s 35.8413 KOps/s $\color{#35bf28}+2.42\%$
test_step_mdp_speed[False-True-False-False-True] 3.4012ms 28.5399μs 35.0387 KOps/s 34.0911 KOps/s $\color{#35bf28}+2.78\%$
test_step_mdp_speed[False-True-False-False-False] 0.4036ms 17.7903μs 56.2104 KOps/s 54.7236 KOps/s $\color{#35bf28}+2.72\%$
test_step_mdp_speed[False-False-True-True-True] 0.4246ms 45.3315μs 22.0597 KOps/s 21.5774 KOps/s $\color{#35bf28}+2.24\%$
test_step_mdp_speed[False-False-True-True-False] 60.0710μs 29.6601μs 33.7154 KOps/s 33.1275 KOps/s $\color{#35bf28}+1.77\%$
test_step_mdp_speed[False-False-True-False-True] 59.1910μs 28.2221μs 35.4332 KOps/s 34.9611 KOps/s $\color{#35bf28}+1.35\%$
test_step_mdp_speed[False-False-True-False-False] 0.3976ms 17.7131μs 56.4555 KOps/s 55.2353 KOps/s $\color{#35bf28}+2.21\%$
test_step_mdp_speed[False-False-False-True-True] 0.4208ms 47.4658μs 21.0678 KOps/s 20.6739 KOps/s $\color{#35bf28}+1.91\%$
test_step_mdp_speed[False-False-False-True-False] 0.4151ms 31.5011μs 31.7449 KOps/s 31.2852 KOps/s $\color{#35bf28}+1.47\%$
test_step_mdp_speed[False-False-False-False-True] 0.4030ms 29.4127μs 33.9989 KOps/s 33.1774 KOps/s $\color{#35bf28}+2.48\%$
test_step_mdp_speed[False-False-False-False-False] 54.3610μs 19.1552μs 52.2050 KOps/s 50.0115 KOps/s $\color{#35bf28}+4.39\%$
test_values[generalized_advantage_estimate-True-True] 24.5276ms 23.6749ms 42.2388 Ops/s 41.6700 Ops/s $\color{#35bf28}+1.37\%$
test_values[vec_generalized_advantage_estimate-True-True] 0.1233s 3.3369ms 299.6780 Ops/s 329.7763 Ops/s $\textbf{\color{#d91a1a}-9.13\%}$
test_values[td0_return_estimate-False-False] 87.7810μs 62.5341μs 15.9913 KOps/s 15.5045 KOps/s $\color{#35bf28}+3.14\%$
test_values[td1_return_estimate-False-False] 54.3306ms 53.1640ms 18.8097 Ops/s 18.5936 Ops/s $\color{#35bf28}+1.16\%$
test_values[vec_td1_return_estimate-False-False] 1.2578ms 1.0461ms 955.8952 Ops/s 949.1620 Ops/s $\color{#35bf28}+0.71\%$
test_values[td_lambda_return_estimate-True-False] 85.7291ms 84.7033ms 11.8059 Ops/s 11.7210 Ops/s $\color{#35bf28}+0.72\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.2733ms 1.0507ms 951.7679 Ops/s 949.7560 Ops/s $\color{#35bf28}+0.21\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.5013ms 23.6083ms 42.3580 Ops/s 41.7290 Ops/s $\color{#35bf28}+1.51\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9139ms 0.6867ms 1.4562 KOps/s 1.4325 KOps/s $\color{#35bf28}+1.65\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7494ms 0.6335ms 1.5784 KOps/s 1.5587 KOps/s $\color{#35bf28}+1.27\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5104ms 1.4382ms 695.3287 Ops/s 692.5281 Ops/s $\color{#35bf28}+0.40\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.7221ms 0.6570ms 1.5222 KOps/s 1.5191 KOps/s $\color{#35bf28}+0.20\%$
test_dqn_speed 6.8704ms 1.2742ms 784.7768 Ops/s 776.4627 Ops/s $\color{#35bf28}+1.07\%$
test_ddpg_speed 3.1186ms 2.5903ms 386.0516 Ops/s 382.6635 Ops/s $\color{#35bf28}+0.89\%$
test_sac_speed 7.8570ms 7.4399ms 134.4099 Ops/s 133.4880 Ops/s $\color{#35bf28}+0.69\%$
test_redq_speed 10.5757ms 9.7891ms 102.1540 Ops/s 102.0591 Ops/s $\color{#35bf28}+0.09\%$
test_redq_deprec_speed 10.7432ms 10.2414ms 97.6426 Ops/s 97.0958 Ops/s $\color{#35bf28}+0.56\%$
test_td3_speed 7.7576ms 7.5638ms 132.2079 Ops/s 132.3465 Ops/s $\color{#d91a1a}-0.10\%$
test_cql_speed 28.0052ms 24.6815ms 40.5162 Ops/s 40.2076 Ops/s $\color{#35bf28}+0.77\%$
test_a2c_speed 5.8081ms 5.3583ms 186.6262 Ops/s 184.8982 Ops/s $\color{#35bf28}+0.93\%$
test_ppo_speed 6.0460ms 5.7060ms 175.2541 Ops/s 174.8446 Ops/s $\color{#35bf28}+0.23\%$
test_reinforce_speed 5.9701ms 4.5211ms 221.1853 Ops/s 226.1438 Ops/s $\color{#d91a1a}-2.19\%$
test_iql_speed 22.7949ms 18.9612ms 52.7393 Ops/s 54.1472 Ops/s $\color{#d91a1a}-2.60\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.7610ms 6.3255ms 158.0898 Ops/s 156.2951 Ops/s $\color{#35bf28}+1.15\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2738ms 0.2416ms 4.1393 KOps/s 2.7850 KOps/s $\textbf{\color{#35bf28}+48.63\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.3890ms 0.2164ms 4.6202 KOps/s 4.6839 KOps/s $\color{#d91a1a}-1.36\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.8760ms 6.3281ms 158.0262 Ops/s 158.5168 Ops/s $\color{#d91a1a}-0.31\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.2729ms 0.3233ms 3.0926 KOps/s 4.3036 KOps/s $\textbf{\color{#d91a1a}-28.14\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6293ms 0.2993ms 3.3410 KOps/s 4.7821 KOps/s $\textbf{\color{#d91a1a}-30.13\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.5982ms 1.4008ms 713.8816 Ops/s 829.3653 Ops/s $\textbf{\color{#d91a1a}-13.92\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.6231ms 1.2858ms 777.7159 Ops/s 882.3694 Ops/s $\textbf{\color{#d91a1a}-11.86\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.9221ms 6.5112ms 153.5824 Ops/s 152.9634 Ops/s $\color{#35bf28}+0.40\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.2268ms 0.4514ms 2.2156 KOps/s 2.1950 KOps/s $\color{#35bf28}+0.94\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7985ms 0.4299ms 2.3261 KOps/s 2.3595 KOps/s $\color{#d91a1a}-1.42\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.7227ms 6.3885ms 156.5322 Ops/s 152.6151 Ops/s $\color{#35bf28}+2.57\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2447ms 0.3292ms 3.0375 KOps/s 3.1055 KOps/s $\color{#d91a1a}-2.19\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.4590ms 0.3051ms 3.2781 KOps/s 3.3412 KOps/s $\color{#d91a1a}-1.89\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.8177ms 6.3100ms 158.4786 Ops/s 158.3635 Ops/s $\color{#35bf28}+0.07\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.2516ms 0.3118ms 3.2071 KOps/s 2.1725 KOps/s $\textbf{\color{#35bf28}+47.62\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.4908ms 0.2960ms 3.3780 KOps/s 3.3640 KOps/s $\color{#35bf28}+0.42\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.6421ms 6.4846ms 154.2110 Ops/s 152.5663 Ops/s $\color{#35bf28}+1.08\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9922ms 0.4446ms 2.2491 KOps/s 2.2545 KOps/s $\color{#d91a1a}-0.24\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.5799ms 0.4196ms 2.3833 KOps/s 2.3486 KOps/s $\color{#35bf28}+1.48\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1477s 7.9804ms 125.3073 Ops/s 125.2660 Ops/s $\color{#35bf28}+0.03\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 30.5237ms 16.2983ms 61.3560 Ops/s 62.0165 Ops/s $\color{#d91a1a}-1.07\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.2488ms 0.9986ms 1.0014 KOps/s 964.9490 Ops/s $\color{#35bf28}+3.78\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1322s 7.6560ms 130.6173 Ops/s 98.2334 Ops/s $\textbf{\color{#35bf28}+32.97\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 21.8080ms 16.4072ms 60.9488 Ops/s 62.6206 Ops/s $\color{#d91a1a}-2.67\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.1093ms 1.0015ms 998.5240 Ops/s 889.6444 Ops/s $\textbf{\color{#35bf28}+12.24\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1349s 10.3843ms 96.2990 Ops/s 128.2037 Ops/s $\textbf{\color{#d91a1a}-24.89\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 21.8012ms 16.2827ms 61.4147 Ops/s 61.9344 Ops/s $\color{#d91a1a}-0.84\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 8.1912ms 1.2324ms 811.4070 Ops/s 796.9420 Ops/s $\color{#35bf28}+1.82\%$

@vmoens vmoens merged commit 040ea58 into gh/vmoens/26/base Sep 12, 2024
62 of 69 checks passed
vmoens added a commit that referenced this pull request Sep 12, 2024
@vmoens
[BugFix] compatibility to new Composite dist log_prob/entropy APIs
36545af 
ghstack-source-id: a09b6c34000f57a66736bb9811ca3656c861ec0c
Pull Request resolved: #2435
@vmoens vmoens deleted the gh/vmoens/26/head branch September 12, 2024 10:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot