Add support for writing episode gfx-replay files during eval #870
Conversation
facebook-github-bot
added
the
CLA Signed
| and self._sim.sim_config.sim_cfg.enable_gfx_replay_save | ||
| ): | ||
| if not self._gfx_replay_keyframes_string: | ||
| self._gfx_replay_keyframes_string = ( |
There was a problem hiding this comment.
Note we cache the result of write_saved_keyframes_to_string() instead of allowing it to perhaps be called twice for the same episode (which wouldn't work correctly).
| # A gfx-replay list of keyframes for the episode. This is a JSON string that | ||
| # should be saved to a file; the file can be read by visualization tools | ||
| # (e.g. import into Blender for screenshots and videos). | ||
| gfx_replay_uuid = "gfx_replay_keyframes_string" |
There was a problem hiding this comment.
I'm open to suggestions for some place to declare this UUID as a global so it can be re-used in rearrange_task.py.
| @@ -792,6 +792,7 @@ def __init__(self, *args, **kwargs): | |||
| ) | |||
| # Possibly unstable optimization for extra performance with concurrent rendering | |||
| _C.SIMULATOR.HABITAT_SIM_V0.LEAVE_CONTEXT_WITH_BACKGROUND_RENDERER = False | |||
| _C.SIMULATOR.HABITAT_SIM_V0.ENABLE_GFX_REPLAY_SAVE = False | |||
There was a problem hiding this comment.
| _C.SIMULATOR.HABITAT_SIM_V0.ENABLE_GFX_REPLAY_SAVE = False | |
| # SAVE_GFX_REPLAY will save a replay of episodes so they can be rendered outside of habitat | |
| _C.SIMULATOR.HABITAT_SIM_V0.SAVE_GFX_REPLAY = False |
I think it would be nice to have a little comment on what this doe sin the default config.
I think SAVE_GFX_REPLAY is a more appropriate name (Enable seems redundant)
| @@ -389,3 +389,6 @@ def is_episode_active(self): | |||
|
|
|||
| def seed(self, seed: int) -> None: | |||
| return | |||
|
|
|||
| def get_metrics_at_episode_end(self): | |||
There was a problem hiding this comment.
I find this a little bit awkward. I do not think this method fits into the embodied task class. Would it be possible to have a measure class instead? This way there would be no need to modify the get_metrics code to get the gfx. You could even directly write to the file inside of the measure (at each episode end).
| filepath = ( | ||
| self.config.GFX_REPLAY_DIR | ||
| + "/episode{}.replay.json".format( | ||
| current_episodes[i].episode_id | ||
| ) | ||
| ) |
There was a problem hiding this comment.
| filepath = ( | |
| self.config.GFX_REPLAY_DIR | |
| + "/episode{}.replay.json".format( | |
| current_episodes[i].episode_id | |
| ) | |
| ) | |
| filepath = f"{self.config.GFX_REPLAY_DIR}/episode{current_episodes[i].episode_id}.replay.json" |
| if self._episode_over: | ||
| metrics.update(self._task.get_metrics_at_episode_end()) | ||
|
|
There was a problem hiding this comment.
I find this too awkward. Is it not possible to call the gfx code inside of a metric proper to avoid calling get_metrics_at_episode_end on the task?
| @@ -1110,6 +1110,24 @@ def _eval_checkpoint( | |||
|
|
|||
| rgb_frames[i] = [] | |||
|
|
|||
| # A gfx-replay list of keyframes for the episode. This is a JSON string that | |||
There was a problem hiding this comment.
Put this code in a separate helper method and put these comments as method comments. This method is too long already.
Also, I realize this code is only called during evaluation, I think the associated measures and config keys should have the prefix EVAL to make it clear the gfx will not be used during training.
|
All of the functionality in this PR is incorporated into: #883 (with the refactors Vince mentioned). Can we close this PR? |
|
Closing this because the functionality is included in another PR, #883. See Andrews' comment above. |
Motivation and Context
Similar to how we generate a video per episode, we also now generate a gfx-replay JSON file per episode. Coming soon, these files can be imported into Blender for high-quality visualization (see facebookresearch/habitat-sim#1759).
This feature requires some Hab-sim gfx-replay fixes, which are here: facebookresearch/habitat-sim#1758
How Has This Been Tested
Local testing with Hab 2.0 rearrange eval.
Types of changes
Checklist