Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] Install ffmpeg in Robohive tests #2063

Merged
merged 2 commits into from
Apr 7, 2024
Merged

[CI] Install ffmpeg in Robohive tests #2063

merged 2 commits into from
Apr 7, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Apr 7, 2024 •
edited
Loading

Still failing but not because of ffmpeg anymore
#2062 will take over

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Apr 7, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2063

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 4 Unrelated Failures

As of commit f0dae68 with merge base f85da4c (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 7, 2024
@vmoens vmoens mentioned this pull request Apr 7, 2024
Merged
@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 7, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 53.0399ms 52.3544ms 19.1006 Ops/s 18.1944 Ops/s $\color{#35bf28}+4.98\%$
test_sync 30.6419ms 28.6747ms 34.8740 Ops/s 33.7929 Ops/s $\color{#35bf28}+3.20\%$
test_async 51.8682ms 27.3216ms 36.6010 Ops/s 38.0330 Ops/s $\color{#d91a1a}-3.77\%$
test_simple 0.3314s 0.3279s 3.0496 Ops/s 2.9308 Ops/s $\color{#35bf28}+4.05\%$
test_transformed 0.4769s 0.4743s 2.1084 Ops/s 2.0930 Ops/s $\color{#35bf28}+0.74\%$
test_serial 1.2497s 1.2034s 0.8310 Ops/s 0.8339 Ops/s $\color{#d91a1a}-0.35\%$
test_parallel 1.0196s 0.9819s 1.0184 Ops/s 1.0055 Ops/s $\color{#35bf28}+1.29\%$
test_step_mdp_speed[True-True-True-True-True] 0.1467ms 21.0801μs 47.4380 KOps/s 46.7602 KOps/s $\color{#35bf28}+1.45\%$
test_step_mdp_speed[True-True-True-True-False] 36.3670μs 13.0587μs 76.5774 KOps/s 76.3707 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[True-True-True-False-True] 38.6320μs 12.5841μs 79.4656 KOps/s 79.8958 KOps/s $\color{#d91a1a}-0.54\%$
test_step_mdp_speed[True-True-True-False-False] 26.4200μs 7.7841μs 128.4665 KOps/s 130.4980 KOps/s $\color{#d91a1a}-1.56\%$
test_step_mdp_speed[True-True-False-True-True] 57.1560μs 22.9345μs 43.6023 KOps/s 43.9531 KOps/s $\color{#d91a1a}-0.80\%$
test_step_mdp_speed[True-True-False-True-False] 36.9090μs 14.3971μs 69.4583 KOps/s 70.3698 KOps/s $\color{#d91a1a}-1.30\%$
test_step_mdp_speed[True-True-False-False-True] 41.2970μs 13.8166μs 72.3768 KOps/s 71.6098 KOps/s $\color{#35bf28}+1.07\%$
test_step_mdp_speed[True-True-False-False-False] 38.9120μs 8.8225μs 113.3465 KOps/s 110.5104 KOps/s $\color{#35bf28}+2.57\%$
test_step_mdp_speed[True-False-True-True-True] 62.1760μs 23.7773μs 42.0569 KOps/s 42.0515 KOps/s $\color{#35bf28}+0.01\%$
test_step_mdp_speed[True-False-True-True-False] 35.8180μs 15.5999μs 64.1030 KOps/s 63.8989 KOps/s $\color{#35bf28}+0.32\%$
test_step_mdp_speed[True-False-True-False-True] 43.2810μs 13.7147μs 72.9147 KOps/s 73.5534 KOps/s $\color{#d91a1a}-0.87\%$
test_step_mdp_speed[True-False-True-False-False] 29.0050μs 8.8551μs 112.9295 KOps/s 113.0284 KOps/s $\color{#d91a1a}-0.09\%$
test_step_mdp_speed[True-False-False-True-True] 57.7680μs 25.0480μs 39.9234 KOps/s 39.9538 KOps/s $\color{#d91a1a}-0.08\%$
test_step_mdp_speed[True-False-False-True-False] 45.4150μs 16.7226μs 59.7993 KOps/s 59.2833 KOps/s $\color{#35bf28}+0.87\%$
test_step_mdp_speed[True-False-False-False-True] 43.0310μs 14.8414μs 67.3792 KOps/s 67.3363 KOps/s $\color{#35bf28}+0.06\%$
test_step_mdp_speed[True-False-False-False-False] 29.9960μs 10.0695μs 99.3099 KOps/s 99.2933 KOps/s $\color{#35bf28}+0.02\%$
test_step_mdp_speed[False-True-True-True-True] 56.0550μs 24.2500μs 41.2371 KOps/s 41.2082 KOps/s $\color{#35bf28}+0.07\%$
test_step_mdp_speed[False-True-True-True-False] 57.8390μs 15.6001μs 64.1020 KOps/s 63.9505 KOps/s $\color{#35bf28}+0.24\%$
test_step_mdp_speed[False-True-True-False-True] 51.1550μs 16.1144μs 62.0564 KOps/s 62.4314 KOps/s $\color{#d91a1a}-0.60\%$
test_step_mdp_speed[False-True-True-False-False] 29.7550μs 10.2454μs 97.6045 KOps/s 98.7387 KOps/s $\color{#d91a1a}-1.15\%$
test_step_mdp_speed[False-True-False-True-True] 36.0880μs 25.4400μs 39.3082 KOps/s 38.8678 KOps/s $\color{#35bf28}+1.13\%$
test_step_mdp_speed[False-True-False-True-False] 48.5010μs 16.9372μs 59.0416 KOps/s 59.0404 KOps/s $+0.00\%$
test_step_mdp_speed[False-True-False-False-True] 46.0460μs 17.4470μs 57.3164 KOps/s 57.6005 KOps/s $\color{#d91a1a}-0.49\%$
test_step_mdp_speed[False-True-False-False-False] 32.2410μs 11.4243μs 87.5324 KOps/s 88.7600 KOps/s $\color{#d91a1a}-1.38\%$
test_step_mdp_speed[False-False-True-True-True] 64.0100μs 26.6331μs 37.5473 KOps/s 37.6573 KOps/s $\color{#d91a1a}-0.29\%$
test_step_mdp_speed[False-False-True-True-False] 49.5760μs 18.2641μs 54.7521 KOps/s 55.2626 KOps/s $\color{#d91a1a}-0.92\%$
test_step_mdp_speed[False-False-True-False-True] 56.0450μs 17.1969μs 58.1500 KOps/s 58.4753 KOps/s $\color{#d91a1a}-0.56\%$
test_step_mdp_speed[False-False-True-False-False] 32.5010μs 11.3948μs 87.7593 KOps/s 88.9578 KOps/s $\color{#d91a1a}-1.35\%$
test_step_mdp_speed[False-False-False-True-True] 0.1132ms 27.3248μs 36.5968 KOps/s 36.3013 KOps/s $\color{#35bf28}+0.81\%$
test_step_mdp_speed[False-False-False-True-False] 54.3510μs 19.0763μs 52.4210 KOps/s 52.0089 KOps/s $\color{#35bf28}+0.79\%$
test_step_mdp_speed[False-False-False-False-True] 44.1930μs 18.2755μs 54.7181 KOps/s 54.5208 KOps/s $\color{#35bf28}+0.36\%$
test_step_mdp_speed[False-False-False-False-False] 44.8340μs 12.3979μs 80.6587 KOps/s 80.9532 KOps/s $\color{#d91a1a}-0.36\%$
test_values[generalized_advantage_estimate-True-True] 9.8219ms 9.2799ms 107.7603 Ops/s 111.3559 Ops/s $\color{#d91a1a}-3.23\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.6600ms 35.2595ms 28.3612 Ops/s 28.4334 Ops/s $\color{#d91a1a}-0.25\%$
test_values[td0_return_estimate-False-False] 0.2245ms 0.1664ms 6.0098 KOps/s 6.2024 KOps/s $\color{#d91a1a}-3.11\%$
test_values[td1_return_estimate-False-False] 26.3732ms 23.5233ms 42.5111 Ops/s 44.0586 Ops/s $\color{#d91a1a}-3.51\%$
test_values[vec_td1_return_estimate-False-False] 36.1765ms 35.1870ms 28.4196 Ops/s 28.3947 Ops/s $\color{#35bf28}+0.09\%$
test_values[td_lambda_return_estimate-True-False] 36.9486ms 33.8244ms 29.5645 Ops/s 30.3632 Ops/s $\color{#d91a1a}-2.63\%$
test_values[vec_td_lambda_return_estimate-True-False] 38.8465ms 35.2856ms 28.3402 Ops/s 28.4146 Ops/s $\color{#d91a1a}-0.26\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.3768ms 8.2239ms 121.5962 Ops/s 125.3681 Ops/s $\color{#d91a1a}-3.01\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.5301ms 1.9925ms 501.8746 Ops/s 558.3756 Ops/s $\textbf{\color{#d91a1a}-10.12\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4800ms 0.3466ms 2.8854 KOps/s 2.8920 KOps/s $\color{#d91a1a}-0.23\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 51.1416ms 46.3990ms 21.5522 Ops/s 21.6830 Ops/s $\color{#d91a1a}-0.60\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.7219ms 3.0154ms 331.6359 Ops/s 330.6449 Ops/s $\color{#35bf28}+0.30\%$
test_dqn_speed 6.7834ms 1.3504ms 740.5164 Ops/s 744.5119 Ops/s $\color{#d91a1a}-0.54\%$
test_ddpg_speed 3.3902ms 2.6578ms 376.2472 Ops/s 373.0951 Ops/s $\color{#35bf28}+0.84\%$
test_sac_speed 8.6016ms 8.1575ms 122.5865 Ops/s 123.0059 Ops/s $\color{#d91a1a}-0.34\%$
test_redq_speed 14.1639ms 12.9659ms 77.1253 Ops/s 77.4945 Ops/s $\color{#d91a1a}-0.48\%$
test_redq_deprec_speed 14.2026ms 12.8458ms 77.8464 Ops/s 77.9758 Ops/s $\color{#d91a1a}-0.17\%$
test_td3_speed 14.8027ms 8.1486ms 122.7210 Ops/s 123.3395 Ops/s $\color{#d91a1a}-0.50\%$
test_cql_speed 38.2660ms 36.9685ms 27.0500 Ops/s 27.1757 Ops/s $\color{#d91a1a}-0.46\%$
test_a2c_speed 8.5565ms 7.3483ms 136.0850 Ops/s 137.6517 Ops/s $\color{#d91a1a}-1.14\%$
test_ppo_speed 8.2766ms 7.5472ms 132.5000 Ops/s 133.2718 Ops/s $\color{#d91a1a}-0.58\%$
test_reinforce_speed 7.4646ms 6.5225ms 153.3145 Ops/s 153.5207 Ops/s $\color{#d91a1a}-0.13\%$
test_iql_speed 33.5091ms 32.3836ms 30.8798 Ops/s 31.2761 Ops/s $\color{#d91a1a}-1.27\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.4236ms 2.1636ms 462.1896 Ops/s 466.7291 Ops/s $\color{#d91a1a}-0.97\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9871ms 0.4950ms 2.0203 KOps/s 1.8239 KOps/s $\textbf{\color{#35bf28}+10.77\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6555ms 0.4702ms 2.1266 KOps/s 2.1283 KOps/s $\color{#d91a1a}-0.08\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.3798ms 2.1193ms 471.8492 Ops/s 468.5209 Ops/s $\color{#35bf28}+0.71\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0794ms 0.4873ms 2.0522 KOps/s 1.9541 KOps/s $\textbf{\color{#35bf28}+5.02\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6888ms 0.4616ms 2.1664 KOps/s 2.1608 KOps/s $\color{#35bf28}+0.26\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.3168ms 1.2149ms 823.1246 Ops/s 821.9419 Ops/s $\color{#35bf28}+0.14\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.3280ms 1.1499ms 869.6501 Ops/s 862.0285 Ops/s $\color{#35bf28}+0.88\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.3861ms 2.2467ms 445.0895 Ops/s 438.8699 Ops/s $\color{#35bf28}+1.42\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1523ms 0.6080ms 1.6448 KOps/s 1.6397 KOps/s $\color{#35bf28}+0.31\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8478ms 0.5876ms 1.7020 KOps/s 1.7057 KOps/s $\color{#d91a1a}-0.22\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.5185ms 2.2132ms 451.8300 Ops/s 465.7679 Ops/s $\color{#d91a1a}-2.99\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.1579ms 0.4985ms 2.0059 KOps/s 2.0166 KOps/s $\color{#d91a1a}-0.53\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7007ms 0.4765ms 2.0988 KOps/s 2.1161 KOps/s $\color{#d91a1a}-0.82\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.5130ms 2.1444ms 466.3392 Ops/s 472.6002 Ops/s $\color{#d91a1a}-1.32\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7916ms 0.4913ms 2.0353 KOps/s 2.0344 KOps/s $\color{#35bf28}+0.04\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.9271ms 0.4677ms 2.1381 KOps/s 2.1598 KOps/s $\color{#d91a1a}-1.00\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.4738ms 2.2549ms 443.4883 Ops/s 439.6895 Ops/s $\color{#35bf28}+0.86\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9950ms 0.6624ms 1.5096 KOps/s 1.6406 KOps/s $\textbf{\color{#d91a1a}-7.99\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7618ms 0.5848ms 1.7101 KOps/s 1.6900 KOps/s $\color{#35bf28}+1.19\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 91.8998ms 7.0001ms 142.8561 Ops/s 139.7871 Ops/s $\color{#35bf28}+2.20\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 14.4805ms 11.9942ms 83.3736 Ops/s 83.4178 Ops/s $\color{#d91a1a}-0.05\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.0752ms 1.0285ms 972.2619 Ops/s 984.9626 Ops/s $\color{#d91a1a}-1.29\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 83.5741ms 6.7627ms 147.8709 Ops/s 146.5289 Ops/s $\color{#35bf28}+0.92\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 14.5293ms 11.9806ms 83.4680 Ops/s 84.5213 Ops/s $\color{#d91a1a}-1.25\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.1615ms 1.0257ms 974.9014 Ops/s 985.2203 Ops/s $\color{#d91a1a}-1.05\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 90.5960ms 5.6674ms 176.4470 Ops/s 182.5774 Ops/s $\color{#d91a1a}-3.36\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 14.7408ms 12.2372ms 81.7182 Ops/s 72.4396 Ops/s $\textbf{\color{#35bf28}+12.81\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.1237ms 1.3782ms 725.5765 Ops/s 721.9515 Ops/s $\color{#35bf28}+0.50\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 7, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1019s 0.1008s 9.9185 Ops/s 9.6469 Ops/s $\color{#35bf28}+2.82\%$
test_sync 92.6036ms 87.4114ms 11.4402 Ops/s 11.5182 Ops/s $\color{#d91a1a}-0.68\%$
test_async 0.1615s 70.5238ms 14.1796 Ops/s 13.9843 Ops/s $\color{#35bf28}+1.40\%$
test_single_pixels 0.1106s 0.1103s 9.0656 Ops/s 9.1204 Ops/s $\color{#d91a1a}-0.60\%$
test_sync_pixels 0.1225s 73.5104ms 13.6035 Ops/s 13.6595 Ops/s $\color{#d91a1a}-0.41\%$
test_async_pixels 0.1227s 61.2537ms 16.3255 Ops/s 16.3416 Ops/s $\color{#d91a1a}-0.10\%$
test_simple 0.6819s 0.6789s 1.4730 Ops/s 1.4773 Ops/s $\color{#d91a1a}-0.29\%$
test_transformed 0.8953s 0.8947s 1.1177 Ops/s 1.1160 Ops/s $\color{#35bf28}+0.15\%$
test_serial 2.1731s 2.1121s 0.4735 Ops/s 0.4813 Ops/s $\color{#d91a1a}-1.64\%$
test_parallel 1.8544s 1.8062s 0.5537 Ops/s 0.5569 Ops/s $\color{#d91a1a}-0.58\%$
test_step_mdp_speed[True-True-True-True-True] 84.7110μs 33.8155μs 29.5723 KOps/s 30.5887 KOps/s $\color{#d91a1a}-3.32\%$
test_step_mdp_speed[True-True-True-True-False] 74.0800μs 19.6834μs 50.8042 KOps/s 51.6763 KOps/s $\color{#d91a1a}-1.69\%$
test_step_mdp_speed[True-True-True-False-True] 88.3010μs 18.8812μs 52.9628 KOps/s 54.8403 KOps/s $\color{#d91a1a}-3.42\%$
test_step_mdp_speed[True-True-True-False-False] 69.4110μs 11.4216μs 87.5531 KOps/s 90.9589 KOps/s $\color{#d91a1a}-3.74\%$
test_step_mdp_speed[True-True-False-True-True] 64.4800μs 35.4170μs 28.2350 KOps/s 29.0421 KOps/s $\color{#d91a1a}-2.78\%$
test_step_mdp_speed[True-True-False-True-False] 46.3510μs 21.8518μs 45.7628 KOps/s 47.1868 KOps/s $\color{#d91a1a}-3.02\%$
test_step_mdp_speed[True-True-False-False-True] 42.4800μs 20.6650μs 48.3911 KOps/s 49.4581 KOps/s $\color{#d91a1a}-2.16\%$
test_step_mdp_speed[True-True-False-False-False] 28.0310μs 13.3879μs 74.6945 KOps/s 77.4382 KOps/s $\color{#d91a1a}-3.54\%$
test_step_mdp_speed[True-False-True-True-True] 58.5900μs 37.8391μs 26.4277 KOps/s 27.3718 KOps/s $\color{#d91a1a}-3.45\%$
test_step_mdp_speed[True-False-True-True-False] 97.6410μs 23.6337μs 42.3124 KOps/s 42.8977 KOps/s $\color{#d91a1a}-1.36\%$
test_step_mdp_speed[True-False-True-False-True] 40.3510μs 20.9051μs 47.8353 KOps/s 49.7002 KOps/s $\color{#d91a1a}-3.75\%$
test_step_mdp_speed[True-False-True-False-False] 34.9000μs 13.4271μs 74.4765 KOps/s 77.4201 KOps/s $\color{#d91a1a}-3.80\%$
test_step_mdp_speed[True-False-False-True-True] 65.4300μs 39.4306μs 25.3610 KOps/s 26.0797 KOps/s $\color{#d91a1a}-2.76\%$
test_step_mdp_speed[True-False-False-True-False] 47.1310μs 25.7325μs 38.8614 KOps/s 39.8628 KOps/s $\color{#d91a1a}-2.51\%$
test_step_mdp_speed[True-False-False-False-True] 0.1006ms 22.2958μs 44.8515 KOps/s 45.8789 KOps/s $\color{#d91a1a}-2.24\%$
test_step_mdp_speed[True-False-False-False-False] 36.9000μs 15.1257μs 66.1125 KOps/s 67.9821 KOps/s $\color{#d91a1a}-2.75\%$
test_step_mdp_speed[False-True-True-True-True] 57.6010μs 37.0747μs 26.9726 KOps/s 27.5713 KOps/s $\color{#d91a1a}-2.17\%$
test_step_mdp_speed[False-True-True-True-False] 48.0210μs 23.8241μs 41.9744 KOps/s 42.7697 KOps/s $\color{#d91a1a}-1.86\%$
test_step_mdp_speed[False-True-True-False-True] 41.1400μs 25.0327μs 39.9477 KOps/s 40.7889 KOps/s $\color{#d91a1a}-2.06\%$
test_step_mdp_speed[False-True-True-False-False] 0.1168ms 15.2696μs 65.4896 KOps/s 68.1361 KOps/s $\color{#d91a1a}-3.88\%$
test_step_mdp_speed[False-True-False-True-True] 67.3010μs 39.6052μs 25.2492 KOps/s 26.2672 KOps/s $\color{#d91a1a}-3.88\%$
test_step_mdp_speed[False-True-False-True-False] 93.8010μs 25.5683μs 39.1109 KOps/s 39.7278 KOps/s $\color{#d91a1a}-1.55\%$
test_step_mdp_speed[False-True-False-False-True] 65.7500μs 26.4810μs 37.7629 KOps/s 38.2262 KOps/s $\color{#d91a1a}-1.21\%$
test_step_mdp_speed[False-True-False-False-False] 80.7810μs 17.1220μs 58.4043 KOps/s 59.8195 KOps/s $\color{#d91a1a}-2.37\%$
test_step_mdp_speed[False-False-True-True-True] 60.6810μs 41.6234μs 24.0249 KOps/s 25.3837 KOps/s $\textbf{\color{#d91a1a}-5.35\%}$
test_step_mdp_speed[False-False-True-True-False] 43.3610μs 27.3931μs 36.5055 KOps/s 37.0148 KOps/s $\color{#d91a1a}-1.38\%$
test_step_mdp_speed[False-False-True-False-True] 56.1100μs 26.7790μs 37.3428 KOps/s 38.9947 KOps/s $\color{#d91a1a}-4.24\%$
test_step_mdp_speed[False-False-True-False-False] 35.0610μs 16.9440μs 59.0180 KOps/s 60.8491 KOps/s $\color{#d91a1a}-3.01\%$
test_step_mdp_speed[False-False-False-True-True] 65.2900μs 42.5588μs 23.4969 KOps/s 23.8586 KOps/s $\color{#d91a1a}-1.52\%$
test_step_mdp_speed[False-False-False-True-False] 57.0510μs 29.2226μs 34.2201 KOps/s 34.7381 KOps/s $\color{#d91a1a}-1.49\%$
test_step_mdp_speed[False-False-False-False-True] 55.8510μs 28.0250μs 35.6824 KOps/s 36.2297 KOps/s $\color{#d91a1a}-1.51\%$
test_step_mdp_speed[False-False-False-False-False] 43.6100μs 18.2970μs 54.6537 KOps/s 54.1851 KOps/s $\color{#35bf28}+0.86\%$
test_values[generalized_advantage_estimate-True-True] 25.3672ms 24.5494ms 40.7341 Ops/s 42.3008 Ops/s $\color{#d91a1a}-3.70\%$
test_values[vec_generalized_advantage_estimate-True-True] 80.4075ms 3.1782ms 314.6434 Ops/s 309.4091 Ops/s $\color{#35bf28}+1.69\%$
test_values[td0_return_estimate-False-False] 0.1110ms 65.2346μs 15.3293 KOps/s 15.6130 KOps/s $\color{#d91a1a}-1.82\%$
test_values[td1_return_estimate-False-False] 54.7162ms 53.0327ms 18.8563 Ops/s 19.4242 Ops/s $\color{#d91a1a}-2.92\%$
test_values[vec_td1_return_estimate-False-False] 2.0961ms 1.7610ms 567.8593 Ops/s 569.6431 Ops/s $\color{#d91a1a}-0.31\%$
test_values[td_lambda_return_estimate-True-False] 86.6134ms 83.6596ms 11.9532 Ops/s 12.2059 Ops/s $\color{#d91a1a}-2.07\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.0812ms 1.7690ms 565.2771 Ops/s 570.8800 Ops/s $\color{#d91a1a}-0.98\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.3923ms 23.6834ms 42.2237 Ops/s 44.3983 Ops/s $\color{#d91a1a}-4.90\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8793ms 0.7022ms 1.4240 KOps/s 1.4400 KOps/s $\color{#d91a1a}-1.11\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7874ms 0.6529ms 1.5316 KOps/s 1.5506 KOps/s $\color{#d91a1a}-1.23\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6415ms 1.4530ms 688.2521 Ops/s 689.8754 Ops/s $\color{#d91a1a}-0.24\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9534ms 0.6757ms 1.4798 KOps/s 1.4956 KOps/s $\color{#d91a1a}-1.06\%$
test_dqn_speed 7.6748ms 1.4546ms 687.4562 Ops/s 698.3252 Ops/s $\color{#d91a1a}-1.56\%$
test_ddpg_speed 2.8559ms 2.7004ms 370.3206 Ops/s 365.8791 Ops/s $\color{#35bf28}+1.21\%$
test_sac_speed 8.4651ms 7.9763ms 125.3716 Ops/s 125.7754 Ops/s $\color{#d91a1a}-0.32\%$
test_redq_speed 11.0129ms 10.2023ms 98.0172 Ops/s 99.2462 Ops/s $\color{#d91a1a}-1.24\%$
test_redq_deprec_speed 11.5823ms 11.1103ms 90.0067 Ops/s 92.2753 Ops/s $\color{#d91a1a}-2.46\%$
test_td3_speed 8.1106ms 7.9169ms 126.3113 Ops/s 124.9419 Ops/s $\color{#35bf28}+1.10\%$
test_cql_speed 25.8796ms 24.6405ms 40.5836 Ops/s 38.9789 Ops/s $\color{#35bf28}+4.12\%$
test_a2c_speed 5.8697ms 5.4250ms 184.3322 Ops/s 180.6286 Ops/s $\color{#35bf28}+2.05\%$
test_ppo_speed 6.1213ms 5.7331ms 174.4260 Ops/s 173.2434 Ops/s $\color{#35bf28}+0.68\%$
test_reinforce_speed 5.4997ms 4.4630ms 224.0654 Ops/s 221.6630 Ops/s $\color{#35bf28}+1.08\%$
test_iql_speed 19.8398ms 19.1442ms 52.2351 Ops/s 51.2827 Ops/s $\color{#35bf28}+1.86\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0943ms 2.9253ms 341.8399 Ops/s 355.4874 Ops/s $\color{#d91a1a}-3.84\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2257ms 0.5391ms 1.8550 KOps/s 1.8385 KOps/s $\color{#35bf28}+0.90\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7351ms 0.5157ms 1.9391 KOps/s 1.8937 KOps/s $\color{#35bf28}+2.40\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1652ms 2.9276ms 341.5786 Ops/s 351.1673 Ops/s $\color{#d91a1a}-2.73\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.3222ms 0.5298ms 1.8874 KOps/s 1.8802 KOps/s $\color{#35bf28}+0.38\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6880ms 0.5076ms 1.9699 KOps/s 1.9533 KOps/s $\color{#35bf28}+0.85\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.5760ms 1.4188ms 704.8080 Ops/s 700.1563 Ops/s $\color{#35bf28}+0.66\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5354ms 1.3487ms 741.4500 Ops/s 737.2941 Ops/s $\color{#35bf28}+0.56\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.2339ms 3.0659ms 326.1686 Ops/s 340.6248 Ops/s $\color{#d91a1a}-4.24\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9365ms 0.7007ms 1.4271 KOps/s 1.5098 KOps/s $\textbf{\color{#d91a1a}-5.48\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.4977ms 0.6461ms 1.5477 KOps/s 1.3678 KOps/s $\textbf{\color{#35bf28}+13.15\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.1211ms 2.9319ms 341.0742 Ops/s 350.0104 Ops/s $\color{#d91a1a}-2.55\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7075ms 0.5404ms 1.8504 KOps/s 1.8423 KOps/s $\color{#35bf28}+0.44\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.1559ms 0.5208ms 1.9201 KOps/s 1.8980 KOps/s $\color{#35bf28}+1.17\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1843ms 2.9617ms 337.6396 Ops/s 351.0647 Ops/s $\color{#d91a1a}-3.82\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.3825ms 0.5356ms 1.8671 KOps/s 1.4696 KOps/s $\textbf{\color{#35bf28}+27.05\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7461ms 0.5101ms 1.9605 KOps/s 1.9625 KOps/s $\color{#d91a1a}-0.11\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.3072ms 3.0562ms 327.1993 Ops/s 336.0389 Ops/s $\color{#d91a1a}-2.63\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8525ms 0.6656ms 1.5024 KOps/s 1.5014 KOps/s $\color{#35bf28}+0.06\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.5343ms 0.6410ms 1.5600 KOps/s 1.5403 KOps/s $\color{#35bf28}+1.28\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1183s 7.1533ms 139.7957 Ops/s 115.5358 Ops/s $\textbf{\color{#35bf28}+21.00\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.0153ms 14.6750ms 68.1430 Ops/s 69.7621 Ops/s $\color{#d91a1a}-2.32\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.0533ms 1.0857ms 921.0276 Ops/s 929.1286 Ops/s $\color{#d91a1a}-0.87\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 99.4839ms 8.6788ms 115.2231 Ops/s 149.4755 Ops/s $\textbf{\color{#d91a1a}-22.92\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.2059ms 14.6264ms 68.3694 Ops/s 70.4046 Ops/s $\color{#d91a1a}-2.89\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.0778ms 1.0568ms 946.2108 Ops/s 936.5157 Ops/s $\color{#35bf28}+1.04\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 99.6971ms 7.2008ms 138.8735 Ops/s 109.5989 Ops/s $\textbf{\color{#35bf28}+26.71\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.1883ms 14.8799ms 67.2046 Ops/s 68.4821 Ops/s $\color{#d91a1a}-1.87\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.5010ms 1.4482ms 690.5074 Ops/s 704.4839 Ops/s $\color{#d91a1a}-1.98\%$

@vmoens vmoens added the CI Has to do with CI setup (e.g. wheels & builds, tests...) label Apr 7, 2024
@vmoens vmoens merged commit 8dc35be into main Apr 7, 2024
57 of 65 checks passed
@vmoens vmoens deleted the fix-ffmpeg-robohive branch April 7, 2024 16:19
vmoens added a commit that referenced this pull request Apr 8, 2024
@vmoens
[CI] Install ffmpeg in Robohive tests (#2063)
1993d2d 
(cherry picked from commit 8dc35be)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CI Has to do with CI setup (e.g. wheels & builds, tests...) CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot