Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Pass replay buffers to MultiSyncDataCollector #2386

Closed
wants to merge 1 commit into from
Closed

[Feature] Pass replay buffers to MultiSyncDataCollector #2386

wants to merge 1 commit into from

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Aug 10, 2024 •
edited
Loading

@vmoens
Update
81ee73b 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Aug 10, 2024
Merged
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Aug 10, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2386

Note: Links to docs will display an error until the docs builds have been completed.

❌ 15 New Failures, 2 Cancelled Jobs, 5 Unrelated Failures

As of commit 81ee73b with merge base a6310ae (image):

NEW FAILURES - The following jobs have failed:

CANCELLED JOBS - The following jobs were cancelled. Please retry:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens added a commit that referenced this pull request Aug 10, 2024
@vmoens
[Feature] Pass replay buffers to MultiSyncDataCollector
797df27 
ghstack-source-id: 29dec41c7b3c448574e872293d1aab6a873a530f
Pull Request resolved: #2386
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 10, 2024
@vmoens vmoens added the enhancement New feature or request label Aug 10, 2024
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}13$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 59.4510ms 58.5041ms 17.0928 Ops/s 17.1977 Ops/s $\color{#d91a1a}-0.61\%$
test_sync 48.1457ms 36.6845ms 27.2595 Ops/s 30.8801 Ops/s $\textbf{\color{#d91a1a}-11.72\%}$
test_async 78.8012ms 31.1731ms 32.0790 Ops/s 32.6321 Ops/s $\color{#d91a1a}-1.70\%$
test_simple 0.4877s 0.4180s 2.3926 Ops/s 2.4174 Ops/s $\color{#d91a1a}-1.03\%$
test_transformed 0.6324s 0.5774s 1.7320 Ops/s 1.7677 Ops/s $\color{#d91a1a}-2.02\%$
test_serial 1.3633s 1.2811s 0.7806 Ops/s 0.8000 Ops/s $\color{#d91a1a}-2.42\%$
test_parallel 1.1553s 1.0977s 0.9110 Ops/s 0.9061 Ops/s $\color{#35bf28}+0.54\%$
test_step_mdp_speed[True-True-True-True-True] 0.1909ms 24.8548μs 40.2337 KOps/s 39.5129 KOps/s $\color{#35bf28}+1.82\%$
test_step_mdp_speed[True-True-True-True-False] 40.3850μs 14.4682μs 69.1170 KOps/s 68.7692 KOps/s $\color{#35bf28}+0.51\%$
test_step_mdp_speed[True-True-True-False-True] 45.5360μs 14.3975μs 69.4563 KOps/s 68.9108 KOps/s $\color{#35bf28}+0.79\%$
test_step_mdp_speed[True-True-True-False-False] 35.3570μs 8.3104μs 120.3311 KOps/s 118.6824 KOps/s $\color{#35bf28}+1.39\%$
test_step_mdp_speed[True-True-False-True-True] 74.0790μs 26.7338μs 37.4059 KOps/s 37.2387 KOps/s $\color{#35bf28}+0.45\%$
test_step_mdp_speed[True-True-False-True-False] 57.0360μs 15.8516μs 63.0852 KOps/s 62.6709 KOps/s $\color{#35bf28}+0.66\%$
test_step_mdp_speed[True-True-False-False-True] 40.5150μs 15.8278μs 63.1802 KOps/s 62.0828 KOps/s $\color{#35bf28}+1.77\%$
test_step_mdp_speed[True-True-False-False-False] 33.3620μs 9.8842μs 101.1716 KOps/s 100.8106 KOps/s $\color{#35bf28}+0.36\%$
test_step_mdp_speed[True-False-True-True-True] 57.7580μs 28.1566μs 35.5156 KOps/s 34.9957 KOps/s $\color{#35bf28}+1.49\%$
test_step_mdp_speed[True-False-True-True-False] 50.6050μs 17.4688μs 57.2448 KOps/s 56.3065 KOps/s $\color{#35bf28}+1.67\%$
test_step_mdp_speed[True-False-True-False-True] 47.4280μs 15.9026μs 62.8827 KOps/s 63.1476 KOps/s $\color{#d91a1a}-0.42\%$
test_step_mdp_speed[True-False-True-False-False] 42.7790μs 9.8466μs 101.5580 KOps/s 100.3375 KOps/s $\color{#35bf28}+1.22\%$
test_step_mdp_speed[True-False-False-True-True] 67.0650μs 29.4681μs 33.9350 KOps/s 33.4188 KOps/s $\color{#35bf28}+1.54\%$
test_step_mdp_speed[True-False-False-True-False] 56.9660μs 18.9500μs 52.7705 KOps/s 51.8479 KOps/s $\color{#35bf28}+1.78\%$
test_step_mdp_speed[True-False-False-False-True] 47.4490μs 17.2776μs 57.8785 KOps/s 56.6649 KOps/s $\color{#35bf28}+2.14\%$
test_step_mdp_speed[True-False-False-False-False] 44.6030μs 11.2497μs 88.8911 KOps/s 87.6103 KOps/s $\color{#35bf28}+1.46\%$
test_step_mdp_speed[False-True-True-True-True] 73.8480μs 28.2026μs 35.4577 KOps/s 35.0514 KOps/s $\color{#35bf28}+1.16\%$
test_step_mdp_speed[False-True-True-True-False] 66.5650μs 17.6289μs 56.7251 KOps/s 56.1810 KOps/s $\color{#35bf28}+0.97\%$
test_step_mdp_speed[False-True-True-False-True] 58.9260μs 18.6168μs 53.7149 KOps/s 54.0622 KOps/s $\color{#d91a1a}-0.64\%$
test_step_mdp_speed[False-True-True-False-False] 46.3270μs 11.1017μs 90.0762 KOps/s 89.9414 KOps/s $\color{#35bf28}+0.15\%$
test_step_mdp_speed[False-True-False-True-True] 63.5190μs 29.6794μs 33.6934 KOps/s 33.3596 KOps/s $\color{#35bf28}+1.00\%$
test_step_mdp_speed[False-True-False-True-False] 47.6390μs 18.9627μs 52.7352 KOps/s 51.2076 KOps/s $\color{#35bf28}+2.98\%$
test_step_mdp_speed[False-True-False-False-True] 66.2940μs 19.8815μs 50.2981 KOps/s 50.2393 KOps/s $\color{#35bf28}+0.12\%$
test_step_mdp_speed[False-True-False-False-False] 45.4360μs 12.4923μs 80.0491 KOps/s 78.6219 KOps/s $\color{#35bf28}+1.82\%$
test_step_mdp_speed[False-False-True-True-True] 3.5001ms 32.2386μs 31.0187 KOps/s 31.1575 KOps/s $\color{#d91a1a}-0.45\%$
test_step_mdp_speed[False-False-True-True-False] 60.8940μs 20.4699μs 48.8521 KOps/s 47.6630 KOps/s $\color{#35bf28}+2.49\%$
test_step_mdp_speed[False-False-True-False-True] 50.4050μs 19.9760μs 50.0602 KOps/s 50.2002 KOps/s $\color{#d91a1a}-0.28\%$
test_step_mdp_speed[False-False-True-False-False] 39.6350μs 12.5918μs 79.4170 KOps/s 78.7870 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[False-False-False-True-True] 67.1250μs 32.2926μs 30.9668 KOps/s 30.2906 KOps/s $\color{#35bf28}+2.23\%$
test_step_mdp_speed[False-False-False-True-False] 49.4630μs 21.7652μs 45.9449 KOps/s 45.1956 KOps/s $\color{#35bf28}+1.66\%$
test_step_mdp_speed[False-False-False-False-True] 49.2820μs 21.0610μs 47.4812 KOps/s 47.3120 KOps/s $\color{#35bf28}+0.36\%$
test_step_mdp_speed[False-False-False-False-False] 45.8860μs 13.8085μs 72.4193 KOps/s 70.9299 KOps/s $\color{#35bf28}+2.10\%$
test_values[generalized_advantage_estimate-True-True] 14.0585ms 9.7289ms 102.7865 Ops/s 109.2091 Ops/s $\textbf{\color{#d91a1a}-5.88\%}$
test_values[vec_generalized_advantage_estimate-True-True] 40.7892ms 33.3581ms 29.9777 Ops/s 28.1553 Ops/s $\textbf{\color{#35bf28}+6.47\%}$
test_values[td0_return_estimate-False-False] 0.2214ms 0.1755ms 5.6970 KOps/s 6.0158 KOps/s $\textbf{\color{#d91a1a}-5.30\%}$
test_values[td1_return_estimate-False-False] 29.3342ms 24.2558ms 41.2272 Ops/s 43.1364 Ops/s $\color{#d91a1a}-4.43\%$
test_values[vec_td1_return_estimate-False-False] 40.9685ms 34.1328ms 29.2973 Ops/s 28.1483 Ops/s $\color{#35bf28}+4.08\%$
test_values[td_lambda_return_estimate-True-False] 35.8706ms 34.0830ms 29.3401 Ops/s 29.7660 Ops/s $\color{#d91a1a}-1.43\%$
test_values[vec_td_lambda_return_estimate-True-False] 36.1219ms 33.7163ms 29.6593 Ops/s 27.7181 Ops/s $\textbf{\color{#35bf28}+7.00\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 9.0533ms 8.3231ms 120.1473 Ops/s 123.5279 Ops/s $\color{#d91a1a}-2.74\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2635ms 1.8481ms 541.0885 Ops/s 488.4671 Ops/s $\textbf{\color{#35bf28}+10.77\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5467ms 0.3626ms 2.7582 KOps/s 2.8092 KOps/s $\color{#d91a1a}-1.82\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 50.0917ms 49.4661ms 20.2158 Ops/s 20.4104 Ops/s $\color{#d91a1a}-0.95\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.1239ms 3.0500ms 327.8735 Ops/s 332.4317 Ops/s $\color{#d91a1a}-1.37\%$
test_dqn_speed 6.6898ms 1.3070ms 765.1091 Ops/s 788.2615 Ops/s $\color{#d91a1a}-2.94\%$
test_ddpg_speed 3.6051ms 2.7252ms 366.9515 Ops/s 379.2825 Ops/s $\color{#d91a1a}-3.25\%$
test_sac_speed 10.1827ms 8.1001ms 123.4548 Ops/s 127.8142 Ops/s $\color{#d91a1a}-3.41\%$
test_redq_speed 17.7545ms 13.1816ms 75.8632 Ops/s 79.7683 Ops/s $\color{#d91a1a}-4.90\%$
test_redq_deprec_speed 15.0644ms 12.7644ms 78.3431 Ops/s 79.9915 Ops/s $\color{#d91a1a}-2.06\%$
test_td3_speed 8.5851ms 8.1505ms 122.6924 Ops/s 128.9850 Ops/s $\color{#d91a1a}-4.88\%$
test_cql_speed 38.2596ms 35.7233ms 27.9929 Ops/s 27.9998 Ops/s $\color{#d91a1a}-0.02\%$
test_a2c_speed 11.2645ms 7.5747ms 132.0179 Ops/s 131.1557 Ops/s $\color{#35bf28}+0.66\%$
test_ppo_speed 9.6712ms 7.7769ms 128.5860 Ops/s 128.2285 Ops/s $\color{#35bf28}+0.28\%$
test_reinforce_speed 8.0323ms 6.4948ms 153.9683 Ops/s 151.7986 Ops/s $\color{#35bf28}+1.43\%$
test_iql_speed 33.4182ms 31.8690ms 31.3785 Ops/s 31.3642 Ops/s $\color{#35bf28}+0.05\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.7433ms 4.9738ms 201.0524 Ops/s 212.3185 Ops/s $\textbf{\color{#d91a1a}-5.31\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8800ms 0.4821ms 2.0744 KOps/s 2.1197 KOps/s $\color{#d91a1a}-2.14\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7807ms 0.4626ms 2.1618 KOps/s 2.2311 KOps/s $\color{#d91a1a}-3.10\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.8138ms 4.9439ms 202.2691 Ops/s 214.8624 Ops/s $\textbf{\color{#d91a1a}-5.86\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8334ms 0.4722ms 2.1178 KOps/s 2.1524 KOps/s $\color{#d91a1a}-1.61\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6540ms 0.4543ms 2.2010 KOps/s 2.2849 KOps/s $\color{#d91a1a}-3.67\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.4448ms 1.6935ms 590.4961 Ops/s 598.3984 Ops/s $\color{#d91a1a}-1.32\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.9818ms 1.6003ms 624.8836 Ops/s 550.2654 Ops/s $\textbf{\color{#35bf28}+13.56\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.7553ms 5.0845ms 196.6775 Ops/s 207.5602 Ops/s $\textbf{\color{#d91a1a}-5.24\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.5014ms 0.6155ms 1.6246 KOps/s 1.6488 KOps/s $\color{#d91a1a}-1.47\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8659ms 0.5927ms 1.6871 KOps/s 1.7334 KOps/s $\color{#d91a1a}-2.67\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.7845ms 5.0227ms 199.0949 Ops/s 211.2443 Ops/s $\textbf{\color{#d91a1a}-5.75\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7073ms 0.4873ms 2.0523 KOps/s 2.1240 KOps/s $\color{#d91a1a}-3.38\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 7.2447ms 0.4718ms 2.1196 KOps/s 2.2049 KOps/s $\color{#d91a1a}-3.87\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.9100ms 5.1161ms 195.4622 Ops/s 216.3990 Ops/s $\textbf{\color{#d91a1a}-9.68\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.2122ms 0.4796ms 2.0849 KOps/s 2.1460 KOps/s $\color{#d91a1a}-2.85\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6582ms 0.4520ms 2.2125 KOps/s 2.1281 KOps/s $\color{#35bf28}+3.97\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.8946ms 5.1423ms 194.4652 Ops/s 201.5288 Ops/s $\color{#d91a1a}-3.51\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1340s 0.8081ms 1.2375 KOps/s 1.6297 KOps/s $\textbf{\color{#d91a1a}-24.06\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7598ms 0.5954ms 1.6796 KOps/s 1.7151 KOps/s $\color{#d91a1a}-2.07\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1305s 6.3576ms 157.2920 Ops/s 167.5312 Ops/s $\textbf{\color{#d91a1a}-6.11\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.7159ms 13.3648ms 74.8236 Ops/s 79.4751 Ops/s $\textbf{\color{#d91a1a}-5.85\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.2411ms 1.0557ms 947.2566 Ops/s 856.2168 Ops/s $\textbf{\color{#35bf28}+10.63\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1265s 6.3109ms 158.4553 Ops/s 171.4597 Ops/s $\textbf{\color{#d91a1a}-7.58\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.6936ms 13.3216ms 75.0663 Ops/s 68.2790 Ops/s $\textbf{\color{#35bf28}+9.94\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.7838ms 1.0705ms 934.1659 Ops/s 931.2087 Ops/s $\color{#35bf28}+0.32\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1263s 9.0253ms 110.7999 Ops/s 169.7399 Ops/s $\textbf{\color{#d91a1a}-34.72\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.0103ms 13.6017ms 73.5204 Ops/s 77.3197 Ops/s $\color{#d91a1a}-4.91\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 4.8784ms 1.2853ms 778.0215 Ops/s 799.4059 Ops/s $\color{#d91a1a}-2.68\%$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}24$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1065s 0.1062s 9.4155 Ops/s 9.5701 Ops/s $\color{#d91a1a}-1.62\%$
test_sync 93.9924ms 90.6410ms 11.0325 Ops/s 10.9311 Ops/s $\color{#35bf28}+0.93\%$
test_async 0.2579s 89.8333ms 11.1317 Ops/s 11.7323 Ops/s $\textbf{\color{#d91a1a}-5.12\%}$
test_single_pixels 0.1173s 0.1167s 8.5684 Ops/s 8.6946 Ops/s $\color{#d91a1a}-1.45\%$
test_sync_pixels 75.3259ms 74.8047ms 13.3681 Ops/s 13.0247 Ops/s $\color{#35bf28}+2.64\%$
test_async_pixels 0.1432s 71.1383ms 14.0571 Ops/s 14.2391 Ops/s $\color{#d91a1a}-1.28\%$
test_simple 0.7517s 0.7503s 1.3328 Ops/s 1.3095 Ops/s $\color{#35bf28}+1.78\%$
test_transformed 1.0819s 1.0062s 0.9939 Ops/s 1.0236 Ops/s $\color{#d91a1a}-2.90\%$
test_serial 2.2421s 2.1680s 0.4612 Ops/s 0.4655 Ops/s $\color{#d91a1a}-0.91\%$
test_parallel 2.0501s 1.9051s 0.5249 Ops/s 0.5364 Ops/s $\color{#d91a1a}-2.14\%$
test_step_mdp_speed[True-True-True-True-True] 94.6420μs 39.2672μs 25.4665 KOps/s 26.8521 KOps/s $\textbf{\color{#d91a1a}-5.16\%}$
test_step_mdp_speed[True-True-True-True-False] 0.1390ms 22.2951μs 44.8530 KOps/s 47.9971 KOps/s $\textbf{\color{#d91a1a}-6.55\%}$
test_step_mdp_speed[True-True-True-False-True] 77.0810μs 21.9494μs 45.5594 KOps/s 48.3672 KOps/s $\textbf{\color{#d91a1a}-5.81\%}$
test_step_mdp_speed[True-True-True-False-False] 30.5710μs 12.4663μs 80.2160 KOps/s 84.7936 KOps/s $\textbf{\color{#d91a1a}-5.40\%}$
test_step_mdp_speed[True-True-False-True-True] 82.1120μs 41.0877μs 24.3382 KOps/s 25.7101 KOps/s $\textbf{\color{#d91a1a}-5.34\%}$
test_step_mdp_speed[True-True-False-True-False] 49.0610μs 24.1579μs 41.3943 KOps/s 43.5390 KOps/s $\color{#d91a1a}-4.93\%$
test_step_mdp_speed[True-True-False-False-True] 39.4110μs 24.6316μs 40.5982 KOps/s 44.0782 KOps/s $\textbf{\color{#d91a1a}-7.89\%}$
test_step_mdp_speed[True-True-False-False-False] 33.4100μs 14.7874μs 67.6250 KOps/s 71.0848 KOps/s $\color{#d91a1a}-4.87\%$
test_step_mdp_speed[True-False-True-True-True] 0.1450ms 43.9046μs 22.7767 KOps/s 24.2891 KOps/s $\textbf{\color{#d91a1a}-6.23\%}$
test_step_mdp_speed[True-False-True-True-False] 0.2190ms 26.7461μs 37.3886 KOps/s 39.7418 KOps/s $\textbf{\color{#d91a1a}-5.92\%}$
test_step_mdp_speed[True-False-True-False-True] 51.7710μs 24.4384μs 40.9192 KOps/s 43.8891 KOps/s $\textbf{\color{#d91a1a}-6.77\%}$
test_step_mdp_speed[True-False-True-False-False] 0.2076ms 14.8180μs 67.4856 KOps/s 70.6430 KOps/s $\color{#d91a1a}-4.47\%$
test_step_mdp_speed[True-False-False-True-True] 71.1510μs 46.1794μs 21.6547 KOps/s 23.0201 KOps/s $\textbf{\color{#d91a1a}-5.93\%}$
test_step_mdp_speed[True-False-False-True-False] 0.1923ms 28.9621μs 34.5279 KOps/s 36.4661 KOps/s $\textbf{\color{#d91a1a}-5.31\%}$
test_step_mdp_speed[True-False-False-False-True] 44.2310μs 26.8565μs 37.2349 KOps/s 39.9076 KOps/s $\textbf{\color{#d91a1a}-6.70\%}$
test_step_mdp_speed[True-False-False-False-False] 0.1831ms 16.8511μs 59.3434 KOps/s 61.6825 KOps/s $\color{#d91a1a}-3.79\%$
test_step_mdp_speed[False-True-True-True-True] 68.8910μs 43.7905μs 22.8360 KOps/s 24.3139 KOps/s $\textbf{\color{#d91a1a}-6.08\%}$
test_step_mdp_speed[False-True-True-True-False] 47.4600μs 26.5647μs 37.6439 KOps/s 39.4319 KOps/s $\color{#d91a1a}-4.53\%$
test_step_mdp_speed[False-True-True-False-True] 0.2056ms 29.5517μs 33.8390 KOps/s 36.2582 KOps/s $\textbf{\color{#d91a1a}-6.67\%}$
test_step_mdp_speed[False-True-True-False-False] 0.2147ms 17.0934μs 58.5021 KOps/s 62.9414 KOps/s $\textbf{\color{#d91a1a}-7.05\%}$
test_step_mdp_speed[False-True-False-True-True] 78.5820μs 46.1778μs 21.6554 KOps/s 23.0258 KOps/s $\textbf{\color{#d91a1a}-5.95\%}$
test_step_mdp_speed[False-True-False-True-False] 0.2160ms 28.6771μs 34.8710 KOps/s 36.4275 KOps/s $\color{#d91a1a}-4.27\%$
test_step_mdp_speed[False-True-False-False-True] 55.1110μs 31.4815μs 31.7647 KOps/s 33.7595 KOps/s $\textbf{\color{#d91a1a}-5.91\%}$
test_step_mdp_speed[False-True-False-False-False] 0.1173ms 19.0012μs 52.6283 KOps/s 55.5214 KOps/s $\textbf{\color{#d91a1a}-5.21\%}$
test_step_mdp_speed[False-False-True-True-True] 4.1128ms 49.1414μs 20.3495 KOps/s 21.5939 KOps/s $\textbf{\color{#d91a1a}-5.76\%}$
test_step_mdp_speed[False-False-True-True-False] 59.1110μs 31.3785μs 31.8690 KOps/s 33.3942 KOps/s $\color{#d91a1a}-4.57\%$
test_step_mdp_speed[False-False-True-False-True] 58.3610μs 31.6303μs 31.6152 KOps/s 33.5116 KOps/s $\textbf{\color{#d91a1a}-5.66\%}$
test_step_mdp_speed[False-False-True-False-False] 37.0100μs 19.0076μs 52.6106 KOps/s 55.3571 KOps/s $\color{#d91a1a}-4.96\%$
test_step_mdp_speed[False-False-False-True-True] 73.3710μs 50.4378μs 19.8264 KOps/s 20.9911 KOps/s $\textbf{\color{#d91a1a}-5.55\%}$
test_step_mdp_speed[False-False-False-True-False] 48.7710μs 33.3420μs 29.9922 KOps/s 31.2358 KOps/s $\color{#d91a1a}-3.98\%$
test_step_mdp_speed[False-False-False-False-True] 57.2120μs 33.2181μs 30.1041 KOps/s 31.5760 KOps/s $\color{#d91a1a}-4.66\%$
test_step_mdp_speed[False-False-False-False-False] 57.2620μs 21.2471μs 47.0653 KOps/s 49.4744 KOps/s $\color{#d91a1a}-4.87\%$
test_values[generalized_advantage_estimate-True-True] 24.2865ms 23.6601ms 42.2653 Ops/s 42.8778 Ops/s $\color{#d91a1a}-1.43\%$
test_values[vec_generalized_advantage_estimate-True-True] 96.4264ms 2.8207ms 354.5171 Ops/s 350.3025 Ops/s $\color{#35bf28}+1.20\%$
test_values[td0_return_estimate-False-False] 93.7410μs 64.3389μs 15.5427 KOps/s 15.6304 KOps/s $\color{#d91a1a}-0.56\%$
test_values[td1_return_estimate-False-False] 53.9652ms 53.4522ms 18.7083 Ops/s 19.0404 Ops/s $\color{#d91a1a}-1.74\%$
test_values[vec_td1_return_estimate-False-False] 1.4000ms 1.0802ms 925.7775 Ops/s 935.0734 Ops/s $\color{#d91a1a}-0.99\%$
test_values[td_lambda_return_estimate-True-False] 88.5374ms 84.6251ms 11.8168 Ops/s 11.9562 Ops/s $\color{#d91a1a}-1.17\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.3969ms 1.0722ms 932.6869 Ops/s 936.1220 Ops/s $\color{#d91a1a}-0.37\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.0378ms 23.7699ms 42.0701 Ops/s 42.9720 Ops/s $\color{#d91a1a}-2.10\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9523ms 0.7093ms 1.4098 KOps/s 1.4119 KOps/s $\color{#d91a1a}-0.15\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7999ms 0.6605ms 1.5140 KOps/s 1.5262 KOps/s $\color{#d91a1a}-0.80\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6332ms 1.4660ms 682.1384 Ops/s 688.7461 Ops/s $\color{#d91a1a}-0.96\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8556ms 0.6741ms 1.4834 KOps/s 1.5014 KOps/s $\color{#d91a1a}-1.20\%$
test_dqn_speed 1.9981ms 1.4076ms 710.4108 Ops/s 724.7049 Ops/s $\color{#d91a1a}-1.97\%$
test_ddpg_speed 3.2202ms 2.7683ms 361.2359 Ops/s 359.1273 Ops/s $\color{#35bf28}+0.59\%$
test_sac_speed 8.4712ms 7.9634ms 125.5742 Ops/s 124.1257 Ops/s $\color{#35bf28}+1.17\%$
test_redq_speed 11.9820ms 10.4967ms 95.2680 Ops/s 95.9261 Ops/s $\color{#d91a1a}-0.69\%$
test_redq_deprec_speed 0.1102s 12.2508ms 81.6272 Ops/s 89.0813 Ops/s $\textbf{\color{#d91a1a}-8.37\%}$
test_td3_speed 8.1389ms 7.9484ms 125.8113 Ops/s 125.8049 Ops/s $+0.01\%$
test_cql_speed 27.3049ms 25.9161ms 38.5861 Ops/s 39.0963 Ops/s $\color{#d91a1a}-1.30\%$
test_a2c_speed 6.4965ms 5.7550ms 173.7626 Ops/s 174.7982 Ops/s $\color{#d91a1a}-0.59\%$
test_ppo_speed 7.9390ms 5.9935ms 166.8465 Ops/s 169.1807 Ops/s $\color{#d91a1a}-1.38\%$
test_reinforce_speed 5.2579ms 4.6215ms 216.3786 Ops/s 220.1779 Ops/s $\color{#d91a1a}-1.73\%$
test_iql_speed 20.7861ms 20.2162ms 49.4652 Ops/s 50.0640 Ops/s $\color{#d91a1a}-1.20\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.9723ms 6.7336ms 148.5091 Ops/s 153.2008 Ops/s $\color{#d91a1a}-3.06\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0877ms 0.5246ms 1.9060 KOps/s 1.8620 KOps/s $\color{#35bf28}+2.36\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7724ms 0.5077ms 1.9695 KOps/s 1.9557 KOps/s $\color{#35bf28}+0.71\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.0326ms 6.6530ms 150.3076 Ops/s 154.8581 Ops/s $\color{#d91a1a}-2.94\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.0438ms 0.5201ms 1.9227 KOps/s 1.9333 KOps/s $\color{#d91a1a}-0.55\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6971ms 0.5000ms 2.0000 KOps/s 2.0018 KOps/s $\color{#d91a1a}-0.09\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.1521ms 2.0094ms 497.6639 Ops/s 471.5879 Ops/s $\textbf{\color{#35bf28}+5.53\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.1410ms 1.9185ms 521.2509 Ops/s 508.4119 Ops/s $\color{#35bf28}+2.53\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.1610ms 6.8706ms 145.5470 Ops/s 150.0031 Ops/s $\color{#d91a1a}-2.97\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.2971ms 0.6759ms 1.4796 KOps/s 1.4845 KOps/s $\color{#d91a1a}-0.33\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8565ms 0.6600ms 1.5152 KOps/s 1.5343 KOps/s $\color{#d91a1a}-1.24\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.8594ms 6.6953ms 149.3581 Ops/s 152.1046 Ops/s $\color{#d91a1a}-1.81\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2062ms 0.5285ms 1.8921 KOps/s 1.9049 KOps/s $\color{#d91a1a}-0.67\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7194ms 0.5090ms 1.9646 KOps/s 1.9834 KOps/s $\color{#d91a1a}-0.95\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.0463ms 6.6718ms 149.8843 Ops/s 153.4432 Ops/s $\color{#d91a1a}-2.32\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6938ms 0.5220ms 1.9158 KOps/s 1.9299 KOps/s $\color{#d91a1a}-0.73\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6924ms 0.5016ms 1.9938 KOps/s 2.0058 KOps/s $\color{#d91a1a}-0.60\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.1227ms 6.8838ms 145.2691 Ops/s 148.8007 Ops/s $\color{#d91a1a}-2.37\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8558ms 0.6857ms 1.4584 KOps/s 1.4773 KOps/s $\color{#d91a1a}-1.28\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.6646ms 0.6690ms 1.4948 KOps/s 1.5302 KOps/s $\color{#d91a1a}-2.31\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1354s 7.8261ms 127.7774 Ops/s 96.0075 Ops/s $\textbf{\color{#35bf28}+33.09\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 19.0097ms 16.3548ms 61.1440 Ops/s 62.5144 Ops/s $\color{#d91a1a}-2.19\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.2313ms 1.2763ms 783.5136 Ops/s 782.4343 Ops/s $\color{#35bf28}+0.14\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1283s 7.6979ms 129.9055 Ops/s 129.9769 Ops/s $\color{#d91a1a}-0.05\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1429s 18.8418ms 53.0736 Ops/s 63.0225 Ops/s $\textbf{\color{#d91a1a}-15.79\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.2472ms 1.2608ms 793.1293 Ops/s 725.4991 Ops/s $\textbf{\color{#35bf28}+9.32\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1332s 7.9800ms 125.3139 Ops/s 126.9397 Ops/s $\color{#d91a1a}-1.28\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.8721ms 16.5262ms 60.5099 Ops/s 62.4288 Ops/s $\color{#d91a1a}-3.07\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.1079ms 1.3113ms 762.6058 Ops/s 704.8628 Ops/s $\textbf{\color{#35bf28}+8.19\%}$

This was referenced Aug 10, 2024
Merged
Merged
@vmoens
Copy link
Contributor Author

vmoens commented Aug 13, 2024

This commit got lost in the sequence, closing

@vmoens vmoens closed this Aug 13, 2024
@vmoens vmoens deleted the gh/vmoens/14/head branch October 17, 2024 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot