BaseVectorEnv.__del__() crash on Windows

[songyuc]

Hi,guys,
I run the example code, but there appears an exception, like,

Epoch #1: 88%|########8 | 8810/10000 [00:29<00:03, 297.75it/s, env_step=8810, len=200, n/ep=1, n/st=10, rew=200.00]
Finished training! Use 29.80s
Exception ignored in: <function BaseVectorEnv.del at 0x000002F3A8FB0040>
Traceback (most recent call last):
File "F:\Users\songyuc\anaconda3\envs\conda-general\lib\site-packages\tianshou\env\venvs.py", line 298, in del
self.close()
File "F:\Users\songyuc\anaconda3\envs\conda-general\lib\site-packages\tianshou\env\venvs.py", line 283, in close
w.close()
File "F:\Users\songyuc\anaconda3\envs\conda-general\lib\site-packages\tianshou\env\worker\base.py", line 67, in close
self.close_env()
File "F:\Users\songyuc\anaconda3\envs\conda-general\lib\site-packages\tianshou\env\worker\dummy.py", line 39, in close_env
self.env.close()
File "F:\Users\songyuc\anaconda3\envs\conda-general\lib\site-packages\gym\core.py", line 243, in close
return self.env.close()
File "F:\Users\songyuc\anaconda3\envs\conda-general\lib\site-packages\gym\envs\classic_control\cartpole.py", line 217, in close
self.viewer.close()
File "F:\Users\songyuc\anaconda3\envs\conda-general\lib\site-packages\gym\envs\classic_control\rendering.py", line 83, in close
self.window.close()
File "F:\Users\songyuc\anaconda3\envs\conda-general\lib\site-packages\pyglet\window\win32_init_.py", line 319, in close
super(Win32Window, self).close()
File "F:\Users\songyuc\anaconda3\envs\conda-general\lib\site-packages\pyglet\window_init_.py", line 838, in close
app.windows.remove(self)
File "F:\Users\songyuc\anaconda3\envs\conda-general\lib_weakrefset.py", line 109, in remove
self.data.remove(ref(item))
KeyError: <weakref at 0x000002F3AB6FAE00; to 'Win32Window' at 0x000002F3AAF0E1C0>
Exception ignored in: <function Viewer.del at 0x000002F3AAF263A0>
Traceback (most recent call last):
File "F:\Users\songyuc\anaconda3\envs\conda-general\lib\site-packages\gym\envs\classic_control\rendering.py", line 165, in del
self.close()
File "F:\Users\songyuc\anaconda3\envs\conda-general\lib\site-packages\gym\envs\classic_control\rendering.py", line 83, in close
self.window.close()
File "F:\Users\songyuc\anaconda3\envs\conda-general\lib\site-packages\pyglet\window\win32_init_.py", line 305, in close
super(Win32Window, self).close()
File "F:\Users\songyuc\anaconda3\envs\conda-general\lib\site-packages\pyglet\window_init_.py", line 838, in close
app.windows.remove(self)
File "F:\Users\songyuc\anaconda3\envs\conda-general\lib_weakrefset.py", line 109, in remove
self.data.remove(ref(item))
KeyError: <weakref at 0x000002F3AB6FADB0; to 'Win32Window' at 0x000002F3AAF0E1C0>

Exception-raising bug
I think this is an exception-raising bug on Win10.

Environment

import tianshou, torch, numpy, sys
print(tianshou.__version__, torch.__version__, numpy.__version__, sys.version, sys.platform)

0.4.1 1.7.0 1.18.5 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 15:50:08) [MSC v.1916 64 bit (AMD64)] win32

Any answer or idea will be appreciated!

• I have marked all applicable categories:
  • exception-raising bug
  • RL algorithm bug
  • documentation request (i.e. "X is missing from the documentation.")
  • new feature request
• I have visited the source website
• I have searched through the issue tracker for duplicates
• I have mentioned version numbers, operating system and environment, where applicable:

  import tianshou, torch, numpy, sys
  print(tianshou.__version__, torch.__version__, numpy.__version__, sys.version, sys.platform)

[Trinkle23897]

I think it doesn't matter when you run the entire code, because it only raises exception at the end of env.close().
It seems like a windows-specific issue. I'll take a look these days. Thanks for reporting this issue!

[songyuc]

Hi, @Trinkle23897 , in fact, the program terminated after this exception, so the remained epoches haven't run.

[Trinkle23897]

Did you run the code in https://tianshou.readthedocs.io/en/master/tutorials/dqn.html#train-policy-with-a-trainer?

the program terminated after this exception, so the remained epochs haven't run.

Nope, in fact as shown in your log:

Epoch #1: 88%|########8 | 8810/10000 [00:29<00:03, 297.75it/s, env_step=8810, len=200, n/ep=1, n/st=10, rew=200.00]
Finished training! Use 29.80s
Exception ignored in: <function BaseVectorEnv.del at 0x000002F3A8FB0040>
Traceback (most recent call last):
File "F:\Users\songyuc\anaconda3\envs\conda-general\lib\site-packages\tianshou\env\venvs.py", line 298, in del
self.close()

This says the training is done (2nd line). And BaseVectorEnv.__del__ can only be called at the end of the program, or if you manually del train_envs, test_envs.

[songyuc]

@Trinkle23897 , I run the code here, https://github.com/thu-ml/tianshou#quick-start
I get it and am sorry for my misunderstanding.
I saw the print of "Epoch #1", so I thought it just trained "the first epoch".

[Trinkle23897]

so the remained epoches haven't run.

btw, this is the feature for early stopping. stop_fn does this thing. It is not a bug.

[songyuc]

@Trinkle23897 , thanks sincerely for your explanation!

[songyuc]

Hi, @Trinkle23897 , what should I do to refrain from this issue?

[Trinkle23897]

just re-install tianshou from the current master version: pip install git+https://github.com/thu-ml/tianshou.git@master --upgrade

[songyuc]

@Trinkle23897 , get it.
Appreciate it very much!