[Question] Tensorboard logging for SB3

[Corvin-N]

Hey, I was wondering how I can enable proper logging for SB3.

What I currently have:
[Using SB3] When training an environment using "python sb3/train.py --task Isaac-Cartpole-v0" and looking at the logs using "tensorboard --logdir logs/sb3/Isaac-Cartpole-v0/", I can see the following data in Tensorboard:

What I want to have:
The same plots as I receive when training with RSL_RL.
[Using RSL_RL] When training an environment using "python rsl_rl/train.py --task Isaac-Cartpole-v0" and looking at the logs using "tensorboard --logdir logs/rsl_rl/cartpole/", I can see the following data in Tensorboard:

Especially the Episode Reward plotted separately by each reward-term or custom Metrics that can be added in CommandTerms would be helpful to understand what happens during training.

Am I missing something and there is a simple way to add these metrics to the SB3-tensorboard log or do I have to code this on my own?
I have also tried to compare omni/isaac/orbit_tasks/utils/wrappers/rsl_rl/vecenv_wrapper.py and omni/isaac/orbit_tasks/utils/wrappers/sb3.py but I did not find any simple possibility to do so.

Thank you!

[Mayankm96]

I suppose the main question is whether one can have their logs added to the tensorboard monitor using Stable-Baselines3.

I am not 100% sure if this is directly supported by SB3 (i.e. it automatically logs all the signals in the episode dictionary). It seems to have the functionality to record some things readily to the logger:

• https://github.com/DLR-RM/stable-baselines3/blob/master/stable_baselines3/ppo/ppo.py#L291-L304
• https://github.com/DLR-RM/stable-baselines3/blob/master/stable_baselines3/common/on_policy_algorithm.py#L263-L274

To log those additional quantities, I think you'll need to implement your own callback that calls the logger to record them as well.