Coordinate conversion

[davidsonic]

Great work! The repo has provided image (256x256) as extracted observation. However, I'm encountering some difficulties performing coordinate conversions from 3D scene to 2D image and vice versa. For example, table in the scene is defined in MJCF model as pos="0.5, 0.3, 0.8" and quat, I want to extract 2D image of this particular region and after performing some detections on objects from this cropped region, transfering back 2D coordinates to 3D scene in pos and quat format. Can you provide some guidance on how to do this?

Many thanks!

[kaixindelele]

Great work! The repo has provided image (256x256) as extracted observation. However, I'm encountering some difficulties performing coordinate conversions from 3D scene to 2D image and vice versa. For example, table in the scene is defined in MJCF model as pos="0.5, 0.3, 0.8" and quat, I want to extract 2D image of this particular region and after performing some detections on objects from this cropped region, transfering back 2D coordinates to 3D scene in pos and quat format. Can you provide some guidance on how to do this?

Many thanks!

hey, have you continued to do this? In object detection tasks, this problem may not be avoided~
If you don't solve it and need it, may we can discuss.

[yukezhu]

Sorry for the much delayed reply. I suggest you look at the MuJoCo API documentations and the transform_utils.py for instructions and tools for coordinate conversion.