SB3 HER Example

[araffin]

Hello,
As SB2 is now deprecated, would you be interested in an updated version of the notebook that uses SB3?

I also found better hyper-parameters (the agent reaches 90% training success in 20 000 steps only) using TQC from the contrib repo.

[eleurent]

Hi @araffin, yes I would, thank you very much!

I already started switching to SB3 in a few scripts, but let's make it official :)

PS: wow I really need to check out that TQC algorithm!

[araffin]

I will do that (and probably will have to re-do it soon as the refactored version is almost ready: DLR-RM/stable-baselines3#351).

It should work well with SAC too (with the new hyperparams), but best results are with TQC.

[StarBaseOne]

@araffin super stuff. Waiting for the merge.
@eleurent yep TQC works nicely (great implementation from SB3) and outperforms SAC.

I've done some baseline experiments on HER with SB3 and Parking and Summon. I will be trying at somepoint to use SB3 benchmark for HER for all 4 algorithms (SAC,TD3,TQC & DDPG) for parking-v0 but if you have the params already done can you share them with me so I can re-run them soon?

[araffin]

all hyperparameters are in the rl zoo: https://github.com/DLR-RM/rl-baselines3-zoo
(and i would recommend using it for doing experiments)

[StarBaseOne]

all hyperparameters are in the rl zoo: https://github.com/DLR-RM/rl-baselines3-zoo
(and i would recommend using it for doing experiments)

Yes thank you. I've been using your benchmark tool. It's pretty solid but haven't checked out from repo or updated it in several months.