problem of max_grad_norm in PPOPolicy

[hyx1999]

The parameter "max_grad_norm" is optional in the PPOPolicy parameter list, but in the training phase, PPOPolicy will require that the value of max_grad_norm cannot be None (but the default value of max_grad_norm is None).

[Trinkle23897]

I don't think so.

tianshou/tianshou/policy/modelfree/ppo.py

Line 181 in c6f2648

if self._max_grad_norm:

This line checks if max_grad_norm is not None. If this parameter is None, it will directly bypass these lines.
I don't see any other lines that require max_grad_norm not None, could you please elaborate on it?

[hyx1999]

Sorry, but the ppo algorithm in the tianshou package I installed via pip is different from the code you showed. I tried to update tianshou through pip install tianshou --upgrade and pip install tianshou --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple, but it seems that there is no change. maybe I should use pip install git+https://github.com/thu-ml/tianshou.git@master --upgrade?

line 180 to line 182

loss.backward()
nn.utils.clip_grad_norm_(
    list(self.actor.parameters())
    + list(self.critic.parameters()),
    self._max_grad_norm)
self.optim.step()

The version of tianshou I currently use is as follows

>>> import tianshou as ts
>>> ts.__version__
'0.3.0.post1'
>>> 

[Trinkle23897]

Okay, it's my bad. I'll post a new release soon (after merge #263).