Question: Is there an easy way to modify tianshou to support vector Env with different state sizes?

[xmzzyo]

Hi, thank you for this great library.

I have a question about the vector Env. The shape of my customized Env's state depends on its size. For example, Env(size=n) will return nx.Graph() with n nodes as its state. However, when I create vector Env with different nodes num, it will raise an error since states with different sizes cannot be stacked (at https://github.com/thu-ml/tianshou/blob/master/tianshou/env/venvs.py#L166). And it seems that the Batch class does not support states with different sizes either.

So, is there an easy way to make the vector Env collect states with different sizes?

Thank you very much!

[Trinkle23897]

Could you please provide the example script so that I can have a test?

[xmzzyo]

Hi, I write an example as follow to illustrate my question:

import gym
import networkx as nx
import numpy as np
from tianshou.env import SubprocVectorEnv


class MyEnv(gym.Env):
    def __init__(self, size):
        self.size = size
        self.feat_dim = 32
        self.graph = nx.Graph()
        self.graph.add_nodes_from(list(range(size)))

    def reset(self):
        graph_state = np.random.rand(self.size, self.feat_dim)
        for i in range(self.size):
            self.graph.nodes[i]["data"] = graph_state[i]
        return np.stack([v["data"] for v in self.graph._node.values()])

    def step(self, action):
        next_graph_state = np.random.rand(self.size, self.feat_dim)
        for i in range(self.size):
            self.graph.nodes[i]["data"] = next_graph_state[i]
        return np.stack([v["data"] for v in self.graph._node.values()]), 1.0, 0, {}


if __name__ == "__main__":
    envs = SubprocVectorEnv(
        [lambda i=x: MyEnv(size=i) for x in [5, 10, 15, 20]])
    obs = envs.reset()

Thank you!

[Trinkle23897]

Hi @xmzzyo, please have a check with #352 and test if it works for you, thanks!

[xmzzyo]

Hi, #352 works for the venv obs issue. But there will be another exception after converting numpy arrays with different shapes to np.object (https://github.com/thu-ml/tianshou/blob/master/tianshou/data/collector.py#L118).

You can test it via this code:

import numpy as np
from tianshou.data import Batch

obs = np.array([np.random.rand(i, 10) for i in [1, 2, 3, 4, 5]], dtype=object)
data = Batch(obs={}, act={}, rew={}, done={}, obs_next={}, info={}, policy={})
data.obs = obs

Thank you!

[Trinkle23897]

How about now?

[xmzzyo]

Yes, it works now.

Thank you very much for your effort on my question!