Inplace bias correction

[r1ckya]

Holocron/holocron/optim/radam.py

Line 90 in 7195235

exp_avg.div_(bias_correction1)

Here bias correction is made inplace so it affects exp_avg in the next steps (bias correction accumulates). I wonder if that was intentional?
Original Adam doesn't accomulate bias correction, it is applied on each step to calculate momentums with hat.

[frgfm]

Agreed, this is a good catch!
Unfortunately, I just read your issue after spending hours looking for the issue and finding it a couple of minutes ago. I'll edit my commit to close this issue!
Thanks for pointing it out!