[Bug Report] Observation groups using the same mdp function have strange behaviour.

[vassil-atn]

Describe the bug

If you use two observation groups (low-level and high-level control) that use the same mdp function (for example mdp.base_lin_vel) have strange behaviour, where the changes in one observation group affect the other one.

Steps to reproduce

@configclass
class ObservationsCfg:

  @configclass
  class LowLevelObsCfg(ObsGroup):
      """Observations for low-level group."""
  
      base_lin_vel = ObsTerm(func=mdp.base_lin_vel, noise=Unoise(n_min=-0.1, n_max=0.1),scale=2.0)
      
  @configclass
  class HighLevelObsCfg(ObsGroup):
      """Observations for high-level group."""

      base_lin_vel = ObsTerm(func=mdp.base_lin_vel, noise=Unoise(n_min=-0.1, n_max=0.1),scale=2.0)

    # observation groups
    highLevelPolicy: HighLevelObsCfg = HighLevelObsCfg()
    lowLevelPolicy: LowLevelObsCfg = LowLevelObsCfg()

The expected behaviour (correct me if I'm wrong) would be to have two observation groups that have the same entry for base_lin_vel. However, the actual behaviour is for the first obs group (high level in this case) to have base_lin_vel scaled by 2. (as expected), but the second group has the base_lin_vel scaled by 4. (or rather it has the base_lin_vel of highLevelPolicy scaled by an additional 2.).

Similarly, if you have two observations within the same group that use the same function, the same occurs. A dirty workaround would be to create new mdp functions for the low-level and high-level policies.

Is there anything I'm missing in the definition of the observation terms and groups?

System Info

Describe the characteristic of your environment:

• Commit: bca680a
• Isaac Sim Version: 2023.1.1-rc.8+2023.1.688.573e0291.tc
• OS: Ubuntu 22.04
• GPU: RTX 3060
• CUDA: 12.2
• GPU Driver: 535.161.07

Checklist

• I have checked that there is no similar issue in the repo (required)
• I have checked that the issue is not in running Isaac Sim itself and is related to the repo

Acceptance Criteria

Add the criteria for which this task is considered done. If not known at issue creation time, you can add this once the issue is assigned.

• Clarification of how observation groups should work.
• Fixed if a bug.

[vassil-atn]

After some more testing it seems that the issue is due to the mdp function returning the asset data directly rather than a copy of it (which means the scaling is then applied to the internal asset data). A fix would be to return a copy instead like:

return asset.data.root_lin_vel_b.clone()

instead of

return asset.data.root_lin_vel_b

[Mayankm96]

Hi @Vassil17 ,

This is a really interesting bug there! Pretty surprised we did not notice it before. All the operations here are doing an operation on "self" to avoid creating new tensors:

https://github.com/NVIDIA-Omniverse/orbit/blob/main/source/extensions/omni.isaac.orbit/omni/isaac/orbit/managers/observation_manager.py#L170-L178

However, this indeed should yield the bug you reported above.

The fix is to add a "clone" operator here: https://github.com/NVIDIA-Omniverse/orbit/blob/main/source/extensions/omni.isaac.orbit/omni/isaac/orbit/managers/observation_manager.py#L170-L171

[Mayankm96]

Sent an MR with the fix. It should get merged soon :)