Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix _reset data passing in parallel env #1880

Merged
merged 5 commits into from
Feb 6, 2024
Merged

[BugFix] Fix _reset data passing in parallel env #1880

merged 5 commits into from
Feb 6, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Feb 6, 2024 •
edited
Loading

Fixes #1865

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Feb 6, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1880

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (2 Unrelated Failures)

As of commit 7e95ffc with merge base 62d977b (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 6, 2024
@vmoens vmoens added the bug Something isn't working label Feb 6, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 6, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1413s 68.4213ms 14.6153 Ops/s 15.6778 Ops/s $\textbf{\color{#d91a1a}-6.78\%}$
test_sync 47.6170ms 35.6864ms 28.0219 Ops/s 27.8441 Ops/s $\color{#35bf28}+0.64\%$
test_async 0.1273s 33.3363ms 29.9974 Ops/s 29.5656 Ops/s $\color{#35bf28}+1.46\%$
test_simple 0.5204s 0.4475s 2.2346 Ops/s 2.2499 Ops/s $\color{#d91a1a}-0.68\%$
test_transformed 0.6760s 0.6122s 1.6334 Ops/s 1.6357 Ops/s $\color{#d91a1a}-0.14\%$
test_serial 1.5794s 1.4787s 0.6763 Ops/s 0.6930 Ops/s $\color{#d91a1a}-2.41\%$
test_parallel 1.5500s 1.4381s 0.6953 Ops/s 0.6981 Ops/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[True-True-True-True-True] 0.1484ms 20.8984μs 47.8506 KOps/s 47.0289 KOps/s $\color{#35bf28}+1.75\%$
test_step_mdp_speed[True-True-True-True-False] 42.3990μs 12.8401μs 77.8808 KOps/s 76.7696 KOps/s $\color{#35bf28}+1.45\%$
test_step_mdp_speed[True-True-True-False-True] 46.5170μs 12.2510μs 81.6262 KOps/s 80.7821 KOps/s $\color{#35bf28}+1.04\%$
test_step_mdp_speed[True-True-True-False-False] 32.5410μs 7.4960μs 133.4045 KOps/s 133.3594 KOps/s $\color{#35bf28}+0.03\%$
test_step_mdp_speed[True-True-False-True-True] 64.5100μs 22.3776μs 44.6875 KOps/s 44.2407 KOps/s $\color{#35bf28}+1.01\%$
test_step_mdp_speed[True-True-False-True-False] 50.0940μs 14.2725μs 70.0650 KOps/s 69.8979 KOps/s $\color{#35bf28}+0.24\%$
test_step_mdp_speed[True-True-False-False-True] 59.7820μs 13.6464μs 73.2795 KOps/s 74.0439 KOps/s $\color{#d91a1a}-1.03\%$
test_step_mdp_speed[True-True-False-False-False] 47.6490μs 8.7144μs 114.7524 KOps/s 113.3742 KOps/s $\color{#35bf28}+1.22\%$
test_step_mdp_speed[True-False-True-True-True] 50.1640μs 23.8276μs 41.9682 KOps/s 41.5402 KOps/s $\color{#35bf28}+1.03\%$
test_step_mdp_speed[True-False-True-True-False] 43.6010μs 15.6475μs 63.9081 KOps/s 63.7566 KOps/s $\color{#35bf28}+0.24\%$
test_step_mdp_speed[True-False-True-False-True] 63.5090μs 13.5624μs 73.7331 KOps/s 73.1970 KOps/s $\color{#35bf28}+0.73\%$
test_step_mdp_speed[True-False-True-False-False] 33.0710μs 8.8024μs 113.6056 KOps/s 115.4597 KOps/s $\color{#d91a1a}-1.61\%$
test_step_mdp_speed[True-False-False-True-True] 71.9040μs 25.1361μs 39.7833 KOps/s 39.6909 KOps/s $\color{#35bf28}+0.23\%$
test_step_mdp_speed[True-False-False-True-False] 50.0840μs 16.8235μs 59.4408 KOps/s 59.0401 KOps/s $\color{#35bf28}+0.68\%$
test_step_mdp_speed[True-False-False-False-True] 58.9400μs 14.9326μs 66.9674 KOps/s 67.3881 KOps/s $\color{#d91a1a}-0.62\%$
test_step_mdp_speed[True-False-False-False-False] 50.2640μs 9.9029μs 100.9805 KOps/s 101.0441 KOps/s $\color{#d91a1a}-0.06\%$
test_step_mdp_speed[False-True-True-True-True] 0.1293ms 24.5757μs 40.6906 KOps/s 41.6594 KOps/s $\color{#d91a1a}-2.33\%$
test_step_mdp_speed[False-True-True-True-False] 45.4250μs 15.5601μs 64.2670 KOps/s 64.0927 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[False-True-True-False-True] 44.6030μs 15.7363μs 63.5473 KOps/s 63.2955 KOps/s $\color{#35bf28}+0.40\%$
test_step_mdp_speed[False-True-True-False-False] 40.3260μs 9.9264μs 100.7411 KOps/s 99.4257 KOps/s $\color{#35bf28}+1.32\%$
test_step_mdp_speed[False-True-False-True-True] 36.9490μs 25.2527μs 39.5997 KOps/s 38.9505 KOps/s $\color{#35bf28}+1.67\%$
test_step_mdp_speed[False-True-False-True-False] 57.1080μs 16.8041μs 59.5094 KOps/s 59.2623 KOps/s $\color{#35bf28}+0.42\%$
test_step_mdp_speed[False-True-False-False-True] 46.6680μs 16.9709μs 58.9243 KOps/s 58.9813 KOps/s $\color{#d91a1a}-0.10\%$
test_step_mdp_speed[False-True-False-False-False] 46.4970μs 11.2225μs 89.1069 KOps/s 89.3098 KOps/s $\color{#d91a1a}-0.23\%$
test_step_mdp_speed[False-False-True-True-True] 63.0180μs 26.3727μs 37.9180 KOps/s 37.7040 KOps/s $\color{#35bf28}+0.57\%$
test_step_mdp_speed[False-False-True-True-False] 61.7560μs 18.1521μs 55.0899 KOps/s 55.2967 KOps/s $\color{#d91a1a}-0.37\%$
test_step_mdp_speed[False-False-True-False-True] 52.7790μs 17.0160μs 58.7682 KOps/s 58.9252 KOps/s $\color{#d91a1a}-0.27\%$
test_step_mdp_speed[False-False-True-False-False] 33.3520μs 11.2211μs 89.1177 KOps/s 89.0481 KOps/s $\color{#35bf28}+0.08\%$
test_step_mdp_speed[False-False-False-True-True] 57.0770μs 27.2538μs 36.6921 KOps/s 36.4408 KOps/s $\color{#35bf28}+0.69\%$
test_step_mdp_speed[False-False-False-True-False] 47.2380μs 19.1517μs 52.2147 KOps/s 51.9324 KOps/s $\color{#35bf28}+0.54\%$
test_step_mdp_speed[False-False-False-False-True] 65.9230μs 17.9507μs 55.7081 KOps/s 55.8708 KOps/s $\color{#d91a1a}-0.29\%$
test_step_mdp_speed[False-False-False-False-False] 48.7910μs 12.3621μs 80.8923 KOps/s 81.4519 KOps/s $\color{#d91a1a}-0.69\%$
test_values[generalized_advantage_estimate-True-True] 9.6967ms 9.4591ms 105.7178 Ops/s 103.5695 Ops/s $\color{#35bf28}+2.07\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.6406ms 35.3724ms 28.2706 Ops/s 28.0247 Ops/s $\color{#35bf28}+0.88\%$
test_values[td0_return_estimate-False-False] 0.2255ms 0.1694ms 5.9023 KOps/s 5.5616 KOps/s $\textbf{\color{#35bf28}+6.13\%}$
test_values[td1_return_estimate-False-False] 24.5906ms 23.6863ms 42.2186 Ops/s 41.8386 Ops/s $\color{#35bf28}+0.91\%$
test_values[vec_td1_return_estimate-False-False] 41.4184ms 35.7819ms 27.9471 Ops/s 28.2623 Ops/s $\color{#d91a1a}-1.12\%$
test_values[td_lambda_return_estimate-True-False] 36.9706ms 34.3903ms 29.0779 Ops/s 29.1504 Ops/s $\color{#d91a1a}-0.25\%$
test_values[vec_td_lambda_return_estimate-True-False] 36.6814ms 35.4094ms 28.2411 Ops/s 28.1415 Ops/s $\color{#35bf28}+0.35\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.2901ms 8.2031ms 121.9052 Ops/s 121.8518 Ops/s $\color{#35bf28}+0.04\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.3139ms 2.0008ms 499.7930 Ops/s 493.1464 Ops/s $\color{#35bf28}+1.35\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4328ms 0.3536ms 2.8284 KOps/s 2.8899 KOps/s $\color{#d91a1a}-2.13\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 47.1942ms 45.1974ms 22.1252 Ops/s 23.2781 Ops/s $\color{#d91a1a}-4.95\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.6054ms 3.0728ms 325.4349 Ops/s 323.6675 Ops/s $\color{#35bf28}+0.55\%$
test_dqn_speed 12.0253ms 6.0898ms 164.2081 Ops/s 163.3896 Ops/s $\color{#35bf28}+0.50\%$
test_ddpg_speed 14.2148ms 11.9722ms 83.5271 Ops/s 81.4634 Ops/s $\color{#35bf28}+2.53\%$
test_sac_speed 28.5060ms 24.3541ms 41.0608 Ops/s 39.2952 Ops/s $\color{#35bf28}+4.49\%$
test_redq_speed 15.1535ms 14.2184ms 70.3313 Ops/s 67.5487 Ops/s $\color{#35bf28}+4.12\%$
test_redq_deprec_speed 27.5368ms 23.0704ms 43.3455 Ops/s 43.0848 Ops/s $\color{#35bf28}+0.61\%$
test_td3_speed 28.9925ms 17.3779ms 57.5444 Ops/s 56.9882 Ops/s $\color{#35bf28}+0.98\%$
test_cql_speed 76.9176ms 75.8878ms 13.1773 Ops/s 13.1076 Ops/s $\color{#35bf28}+0.53\%$
test_a2c_speed 24.3099ms 22.6503ms 44.1494 Ops/s 43.1841 Ops/s $\color{#35bf28}+2.24\%$
test_ppo_speed 25.1372ms 23.8218ms 41.9783 Ops/s 43.3297 Ops/s $\color{#d91a1a}-3.12\%$
test_reinforce_speed 25.1143ms 22.3411ms 44.7606 Ops/s 37.9440 Ops/s $\textbf{\color{#35bf28}+17.96\%}$
test_iql_speed 58.8402ms 57.9053ms 17.2696 Ops/s 17.2607 Ops/s $\color{#35bf28}+0.05\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.2846ms 2.9051ms 344.2189 Ops/s 340.6115 Ops/s $\color{#35bf28}+1.06\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8341ms 0.5123ms 1.9520 KOps/s 1.8047 KOps/s $\textbf{\color{#35bf28}+8.16\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6859ms 0.4809ms 2.0793 KOps/s 2.0709 KOps/s $\color{#35bf28}+0.40\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.3441ms 2.9128ms 343.3177 Ops/s 350.4267 Ops/s $\color{#d91a1a}-2.03\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8957ms 0.5135ms 1.9474 KOps/s 1.9664 KOps/s $\color{#d91a1a}-0.96\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.8140ms 0.4955ms 2.0180 KOps/s 2.0770 KOps/s $\color{#d91a1a}-2.84\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1697ms 2.7087ms 369.1793 Ops/s 370.7127 Ops/s $\color{#d91a1a}-0.41\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9638ms 0.6315ms 1.5835 KOps/s 1.5981 KOps/s $\color{#d91a1a}-0.91\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9434ms 0.6066ms 1.6486 KOps/s 1.6608 KOps/s $\color{#d91a1a}-0.73\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.9616ms 2.8941ms 345.5305 Ops/s 344.0713 Ops/s $\color{#35bf28}+0.42\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7986ms 0.5172ms 1.9336 KOps/s 1.9520 KOps/s $\color{#d91a1a}-0.94\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8665ms 0.4922ms 2.0317 KOps/s 2.0715 KOps/s $\color{#d91a1a}-1.92\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.4234ms 3.0166ms 331.4982 Ops/s 331.5921 Ops/s $\color{#d91a1a}-0.03\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8427ms 0.5137ms 1.9466 KOps/s 1.9010 KOps/s $\color{#35bf28}+2.40\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6333ms 0.4863ms 2.0563 KOps/s 2.0287 KOps/s $\color{#35bf28}+1.36\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.8753ms 2.7952ms 357.7589 Ops/s 339.2631 Ops/s $\textbf{\color{#35bf28}+5.45\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9007ms 0.6298ms 1.5877 KOps/s 1.5470 KOps/s $\color{#35bf28}+2.63\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.0910ms 0.6115ms 1.6352 KOps/s 1.6105 KOps/s $\color{#35bf28}+1.53\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2331s 8.7327ms 114.5124 Ops/s 108.9229 Ops/s $\textbf{\color{#35bf28}+5.13\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 18.6507ms 13.5894ms 73.5867 Ops/s 72.8677 Ops/s $\color{#35bf28}+0.99\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 3.8821ms 2.5753ms 388.3026 Ops/s 374.7500 Ops/s $\color{#35bf28}+3.62\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.2191s 12.6621ms 78.9761 Ops/s 117.8959 Ops/s $\textbf{\color{#d91a1a}-33.01\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 15.7460ms 13.5072ms 74.0348 Ops/s 73.6366 Ops/s $\color{#35bf28}+0.54\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 4.0357ms 2.5371ms 394.1562 Ops/s 394.9411 Ops/s $\color{#d91a1a}-0.20\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.2195s 8.7444ms 114.3590 Ops/s 116.0106 Ops/s $\color{#d91a1a}-1.42\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 16.2261ms 13.6943ms 73.0232 Ops/s 72.4777 Ops/s $\color{#35bf28}+0.75\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 4.7583ms 2.7507ms 363.5487 Ops/s 358.6372 Ops/s $\color{#35bf28}+1.37\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 6, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1167s 0.1158s 8.6390 Ops/s 8.5917 Ops/s $\color{#35bf28}+0.55\%$
test_sync 95.9836ms 95.7665ms 10.4421 Ops/s 10.4107 Ops/s $\color{#35bf28}+0.30\%$
test_async 0.1811s 92.3045ms 10.8337 Ops/s 10.8863 Ops/s $\color{#d91a1a}-0.48\%$
test_single_pixels 0.1289s 0.1286s 7.7787 Ops/s 7.3536 Ops/s $\textbf{\color{#35bf28}+5.78\%}$
test_sync_pixels 80.7941ms 78.5619ms 12.7288 Ops/s 13.1883 Ops/s $\color{#d91a1a}-3.48\%$
test_async_pixels 0.1486s 74.1169ms 13.4922 Ops/s 13.8542 Ops/s $\color{#d91a1a}-2.61\%$
test_simple 0.8851s 0.8150s 1.2271 Ops/s 1.1791 Ops/s $\color{#35bf28}+4.07\%$
test_transformed 1.1254s 1.0621s 0.9415 Ops/s 0.9043 Ops/s $\color{#35bf28}+4.11\%$
test_serial 2.3539s 2.3499s 0.4256 Ops/s 0.4026 Ops/s $\textbf{\color{#35bf28}+5.71\%}$
test_parallel 2.2550s 2.0988s 0.4765 Ops/s 0.4716 Ops/s $\color{#35bf28}+1.04\%$
test_step_mdp_speed[True-True-True-True-True] 87.2410μs 32.2968μs 30.9628 KOps/s 29.7091 KOps/s $\color{#35bf28}+4.22\%$
test_step_mdp_speed[True-True-True-True-False] 45.9010μs 19.2065μs 52.0658 KOps/s 50.0488 KOps/s $\color{#35bf28}+4.03\%$
test_step_mdp_speed[True-True-True-False-True] 34.5110μs 18.1961μs 54.9568 KOps/s 53.3399 KOps/s $\color{#35bf28}+3.03\%$
test_step_mdp_speed[True-True-True-False-False] 27.5300μs 10.9019μs 91.7270 KOps/s 87.9398 KOps/s $\color{#35bf28}+4.31\%$
test_step_mdp_speed[True-True-False-True-True] 55.4600μs 33.9656μs 29.4416 KOps/s 28.3606 KOps/s $\color{#35bf28}+3.81\%$
test_step_mdp_speed[True-True-False-True-False] 36.8500μs 21.0472μs 47.5122 KOps/s 46.1663 KOps/s $\color{#35bf28}+2.92\%$
test_step_mdp_speed[True-True-False-False-True] 47.8810μs 20.0756μs 49.8116 KOps/s 48.1273 KOps/s $\color{#35bf28}+3.50\%$
test_step_mdp_speed[True-True-False-False-False] 41.1710μs 12.8650μs 77.7305 KOps/s 75.0477 KOps/s $\color{#35bf28}+3.57\%$
test_step_mdp_speed[True-False-True-True-True] 60.2600μs 35.9552μs 27.8124 KOps/s 26.8706 KOps/s $\color{#35bf28}+3.50\%$
test_step_mdp_speed[True-False-True-True-False] 47.6400μs 23.2326μs 43.0429 KOps/s 42.3982 KOps/s $\color{#35bf28}+1.52\%$
test_step_mdp_speed[True-False-True-False-True] 46.6910μs 20.0022μs 49.9945 KOps/s 48.5310 KOps/s $\color{#35bf28}+3.02\%$
test_step_mdp_speed[True-False-True-False-False] 28.8100μs 12.8441μs 77.8569 KOps/s 76.0155 KOps/s $\color{#35bf28}+2.42\%$
test_step_mdp_speed[True-False-False-True-True] 67.9910μs 37.8397μs 26.4273 KOps/s 25.7618 KOps/s $\color{#35bf28}+2.58\%$
test_step_mdp_speed[True-False-False-True-False] 54.6810μs 24.7501μs 40.4039 KOps/s 39.1067 KOps/s $\color{#35bf28}+3.32\%$
test_step_mdp_speed[True-False-False-False-True] 43.3700μs 21.7964μs 45.8792 KOps/s 44.8358 KOps/s $\color{#35bf28}+2.33\%$
test_step_mdp_speed[True-False-False-False-False] 35.1110μs 14.5286μs 68.8300 KOps/s 66.4977 KOps/s $\color{#35bf28}+3.51\%$
test_step_mdp_speed[False-True-True-True-True] 63.9110μs 35.7044μs 28.0077 KOps/s 26.8573 KOps/s $\color{#35bf28}+4.28\%$
test_step_mdp_speed[False-True-True-True-False] 41.2910μs 22.9656μs 43.5433 KOps/s 41.9543 KOps/s $\color{#35bf28}+3.79\%$
test_step_mdp_speed[False-True-True-False-True] 53.0700μs 23.6487μs 42.2857 KOps/s 40.4715 KOps/s $\color{#35bf28}+4.48\%$
test_step_mdp_speed[False-True-True-False-False] 35.4910μs 14.6735μs 68.1502 KOps/s 66.1835 KOps/s $\color{#35bf28}+2.97\%$
test_step_mdp_speed[False-True-False-True-True] 69.3710μs 37.5255μs 26.6486 KOps/s 25.5929 KOps/s $\color{#35bf28}+4.12\%$
test_step_mdp_speed[False-True-False-True-False] 50.1110μs 24.6820μs 40.5154 KOps/s 38.8925 KOps/s $\color{#35bf28}+4.17\%$
test_step_mdp_speed[False-True-False-False-True] 52.1700μs 25.8303μs 38.7142 KOps/s 37.9841 KOps/s $\color{#35bf28}+1.92\%$
test_step_mdp_speed[False-True-False-False-False] 37.3200μs 16.2490μs 61.5422 KOps/s 59.4281 KOps/s $\color{#35bf28}+3.56\%$
test_step_mdp_speed[False-False-True-True-True] 67.8410μs 39.4392μs 25.3555 KOps/s 24.8843 KOps/s $\color{#35bf28}+1.89\%$
test_step_mdp_speed[False-False-True-True-False] 47.9810μs 26.5094μs 37.7224 KOps/s 36.0918 KOps/s $\color{#35bf28}+4.52\%$
test_step_mdp_speed[False-False-True-False-True] 47.5800μs 25.7661μs 38.8107 KOps/s 38.4101 KOps/s $\color{#35bf28}+1.04\%$
test_step_mdp_speed[False-False-True-False-False] 34.7110μs 16.4911μs 60.6387 KOps/s 59.2858 KOps/s $\color{#35bf28}+2.28\%$
test_step_mdp_speed[False-False-False-True-True] 78.3410μs 41.0151μs 24.3813 KOps/s 24.1585 KOps/s $\color{#35bf28}+0.92\%$
test_step_mdp_speed[False-False-False-True-False] 50.4300μs 28.2984μs 35.3377 KOps/s 34.0122 KOps/s $\color{#35bf28}+3.90\%$
test_step_mdp_speed[False-False-False-False-True] 54.6810μs 27.1057μs 36.8926 KOps/s 35.7567 KOps/s $\color{#35bf28}+3.18\%$
test_step_mdp_speed[False-False-False-False-False] 34.1900μs 18.2921μs 54.6684 KOps/s 54.2197 KOps/s $\color{#35bf28}+0.83\%$
test_values[generalized_advantage_estimate-True-True] 25.3142ms 24.2868ms 41.1747 Ops/s 42.0712 Ops/s $\color{#d91a1a}-2.13\%$
test_values[vec_generalized_advantage_estimate-True-True] 89.3977ms 3.3468ms 298.7922 Ops/s 309.2621 Ops/s $\color{#d91a1a}-3.39\%$
test_values[td0_return_estimate-False-False] 94.2910μs 60.2256μs 16.6042 KOps/s 16.6810 KOps/s $\color{#d91a1a}-0.46\%$
test_values[td1_return_estimate-False-False] 54.4613ms 51.5226ms 19.4089 Ops/s 18.5530 Ops/s $\color{#35bf28}+4.61\%$
test_values[vec_td1_return_estimate-False-False] 1.9930ms 1.7586ms 568.6196 Ops/s 569.1993 Ops/s $\color{#d91a1a}-0.10\%$
test_values[td_lambda_return_estimate-True-False] 86.4931ms 81.7195ms 12.2370 Ops/s 11.8468 Ops/s $\color{#35bf28}+3.29\%$
test_values[vec_td_lambda_return_estimate-True-False] 4.0566ms 1.8013ms 555.1691 Ops/s 558.9520 Ops/s $\color{#d91a1a}-0.68\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 22.5672ms 22.3607ms 44.7213 Ops/s 42.3331 Ops/s $\textbf{\color{#35bf28}+5.64\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9037ms 0.6935ms 1.4420 KOps/s 1.4504 KOps/s $\color{#d91a1a}-0.58\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7178ms 0.6445ms 1.5517 KOps/s 1.5515 KOps/s $+0.01\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5216ms 1.4438ms 692.5995 Ops/s 691.0066 Ops/s $\color{#35bf28}+0.23\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9203ms 0.6637ms 1.5067 KOps/s 1.5105 KOps/s $\color{#d91a1a}-0.26\%$
test_dqn_speed 14.5642ms 7.6252ms 131.1447 Ops/s 127.2655 Ops/s $\color{#35bf28}+3.05\%$
test_ddpg_speed 0.1072s 15.8533ms 63.0783 Ops/s 66.9312 Ops/s $\textbf{\color{#d91a1a}-5.76\%}$
test_sac_speed 29.2474ms 28.3367ms 35.2899 Ops/s 31.3390 Ops/s $\textbf{\color{#35bf28}+12.61\%}$
test_redq_speed 12.6206ms 11.6057ms 86.1648 Ops/s 85.3796 Ops/s $\color{#35bf28}+0.92\%$
test_redq_deprec_speed 23.7508ms 22.8543ms 43.7555 Ops/s 43.1114 Ops/s $\color{#35bf28}+1.49\%$
test_td3_speed 31.2003ms 19.1137ms 52.3184 Ops/s 51.4082 Ops/s $\color{#35bf28}+1.77\%$
test_cql_speed 77.9222ms 76.0387ms 13.1512 Ops/s 12.8640 Ops/s $\color{#35bf28}+2.23\%$
test_a2c_speed 27.1332ms 25.3371ms 39.4678 Ops/s 38.7445 Ops/s $\color{#35bf28}+1.87\%$
test_ppo_speed 0.1813s 29.6680ms 33.7064 Ops/s 38.1225 Ops/s $\textbf{\color{#d91a1a}-11.58\%}$
test_reinforce_speed 25.8096ms 24.2194ms 41.2892 Ops/s 39.9341 Ops/s $\color{#35bf28}+3.39\%$
test_iql_speed 55.0565ms 51.2381ms 19.5167 Ops/s 19.1329 Ops/s $\color{#35bf28}+2.01\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.7940ms 3.6459ms 274.2793 Ops/s 277.7094 Ops/s $\color{#d91a1a}-1.24\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7588ms 0.5523ms 1.8105 KOps/s 1.7755 KOps/s $\color{#35bf28}+1.97\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7111ms 0.5265ms 1.8993 KOps/s 1.8616 KOps/s $\color{#35bf28}+2.02\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.0436ms 3.6913ms 270.9101 Ops/s 277.1600 Ops/s $\color{#d91a1a}-2.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7287ms 0.5498ms 1.8188 KOps/s 1.7829 KOps/s $\color{#35bf28}+2.01\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6764ms 0.5220ms 1.9157 KOps/s 1.8718 KOps/s $\color{#35bf28}+2.34\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.5165ms 3.3525ms 298.2839 Ops/s 299.7697 Ops/s $\color{#d91a1a}-0.50\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7841ms 0.6801ms 1.4704 KOps/s 1.4293 KOps/s $\color{#35bf28}+2.88\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8364ms 0.6517ms 1.5344 KOps/s 1.5111 KOps/s $\color{#35bf28}+1.55\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.8079ms 3.6695ms 272.5146 Ops/s 276.5360 Ops/s $\color{#d91a1a}-1.45\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7553ms 0.5556ms 1.7999 KOps/s 1.7538 KOps/s $\color{#35bf28}+2.63\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7123ms 0.5309ms 1.8837 KOps/s 1.8230 KOps/s $\color{#35bf28}+3.33\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.8652ms 3.6332ms 275.2367 Ops/s 275.8640 Ops/s $\color{#d91a1a}-0.23\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6771ms 0.5494ms 1.8203 KOps/s 1.8038 KOps/s $\color{#35bf28}+0.91\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7092ms 0.5240ms 1.9083 KOps/s 1.8978 KOps/s $\color{#35bf28}+0.55\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.5415ms 3.4185ms 292.5257 Ops/s 303.6891 Ops/s $\color{#d91a1a}-3.68\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8107ms 0.6751ms 1.4813 KOps/s 1.4627 KOps/s $\color{#35bf28}+1.27\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7893ms 0.6536ms 1.5299 KOps/s 1.5136 KOps/s $\color{#35bf28}+1.07\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2123s 9.7149ms 102.9344 Ops/s 73.7371 Ops/s $\textbf{\color{#35bf28}+39.60\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 18.4632ms 16.0143ms 62.4442 Ops/s 59.4563 Ops/s $\textbf{\color{#35bf28}+5.03\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.7615ms 3.0472ms 328.1675 Ops/s 327.0244 Ops/s $\color{#35bf28}+0.35\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.2121s 9.7797ms 102.2529 Ops/s 105.2212 Ops/s $\color{#d91a1a}-2.82\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 18.5132ms 15.9469ms 62.7083 Ops/s 59.9546 Ops/s $\color{#35bf28}+4.59\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.8132ms 3.0395ms 329.0061 Ops/s 318.3502 Ops/s $\color{#35bf28}+3.35\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.2120s 9.9456ms 100.5466 Ops/s 102.8135 Ops/s $\color{#d91a1a}-2.20\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.2249s 20.3918ms 49.0394 Ops/s 59.1447 Ops/s $\textbf{\color{#d91a1a}-17.09\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.7095ms 3.2604ms 306.7119 Ops/s 303.5593 Ops/s $\color{#35bf28}+1.04\%$

@vmoens vmoens added the Suitable for minor Suitable to be integrated in minor release (no new feature) label Feb 6, 2024
@vmoens vmoens merged commit e53eb73 into main Feb 6, 2024
66 of 68 checks passed
@vmoens vmoens deleted the fix-update-reset-penv branch February 6, 2024 17:43
vmoens added a commit that referenced this pull request Feb 28, 2024
@vmoens
[BugFix] Fix _reset data passing in parallel env (#1880)
ca1cca9
@vmoens vmoens mentioned this pull request Feb 28, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Suitable for minor Suitable to be integrated in minor release (no new feature)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG] Truncated key is on different devices with BatchedEnv on different device than single envs.
2 participants
@vmoens @facebook-github-bot