simulate.cc as a Python module

[aftersomemath]

The release of pybind11 bindings in Mujoco 2.1.2 is very exciting! Would it be reasonable/possible to wrap simulate.cc with pybind11 so that the simulate GUI itself is a Python module?

This would make it very easy for users to work in python and still easily interact with the simulation and change parameters on the fly.

[yuvaltassa]

Hi @aftersomemath,

Yes indeed! This is a great idea and something that we intend to do, but we're not sure we'll be able to get to it very soon. Would you be interested in trying it out yourself?

If you want to see a simple implementation, check out this example by @erwincoumans. This is based on Erwin's prototype pybind11 bindings for MuJoCo, so should carry over quite easily to our new bindings.

The main issue with doing exactly what Erwin did is that it involves code duplication, which we would like to avoid. simulate.cc needs to be refactored so that the same codebase can be used both for a standalone viewer with a simulation loop, and for a Python-driven passive viewer.

If you decide to take a stab, I recommend that you start by mimicking Erwin's approach, code duplication and all, and then we can think together about refactoring.

Cheers

[aftersomemath]

Thanks for the pointer! I will take look into porting those bindings over. They do not seem too different.

To be clear, my goal is to make a Python driven interactive viewer (click and drag should still work while the simulation is running). It seems this would require two threads to guarantee the UI remains responsive as is done in simulate.cc currently.

If that works, then it will be easy to turn the interaction off (passive mode), but then there is thread that doesn't need to be there. With enough refactoring a passive renderer could run on the same thread as the Python process, I don't know if I will have the time to dig into that.

To solve the code duplication problem, the majority of simulate.cc could be made into a shared library. However this would mean simulate.cc is no longer a self-contained sample (though it is 2k lines already). Does this trade seem reasonable? The only alternative I've thought of so far is a hack requiring #include "simulate.cc" 😬.

ciao

[yuvaltassa]

Yup, Erwin's mujoco_renderer.cpp does exactly that, an interactive Python-driven viewer.

We have the same thoughts about breaking out bits of simulate into a library. If refactoring seems like too big a job right now, I reckon that if instead of a PR to this repo, you simply made your viewer available in your own branch, that might be still be useful as a reference to us and to others, as we consider what the best refactor might look like.