ObjectStateSpec, ObjectStateMachine base classes #1842
Conversation
…chine with supporting MetdataDataInterface
aclegg3
requested review from
jturner65,
xavierpuigf,
rutadesai,
ASzot and
jacobkrantz
facebook-github-bot
added
the
CLA Signed
habitat-lab/habitat/sims/habitat_simulator/object_state_machine.py
Outdated
Show resolved
Hide resolved
…tate snapshot for later evaluation
| @@ -77,6 +78,43 @@ def save(self, output_path: str, prefix: str = "") -> str: | |||
| return file_path | |||
|
|
|||
|
|
|||
| def draw_object_highlight( | |||
There was a problem hiding this comment.
Can we control the thickness of the highlight as well? I imagine this can be important for some applications based on the camera resolution.
There was a problem hiding this comment.
I think this can be done in the debug_line_render object before it is passed into this function. https://aihabitat.org/docs/habitat-sim/habitat_sim.gfx.DebugLineRender.html#set_line_width
So it shouldn't need to be explicitly controlled here. Rather the user application can set that for all debug renders.
habitat-lab/habitat/sims/habitat_simulator/metadata_interface.py
Outdated
Show resolved
Hide resolved
| self.receptacles: List[hab_receptacle.Receptacle] = None | ||
|
|
||
| # generate a lexicon from the metadata | ||
| self.hash_to_cat: Dict[str, str] = {} |
There was a problem hiding this comment.
Will the categories have a notion of hierarchy? What if we want to say "all dinning ware can be dirty" which should include fork, plate, spoon, etc?
There was a problem hiding this comment.
I don't think we have any limitations as long as we can represent the desired hierarchy or grouping in the MetadataInterface logic. Currently we only pull the primary category and search a single map. I imagine we need to define these groupings or hierarchy somewhere. Could be an additional annotation field in the csv, or a dict-like map.
…raneous function + cleanup
aclegg3
changed the title
habitat-lab/habitat/sims/habitat_simulator/object_state_machine.py
Outdated
Show resolved
Hide resolved
habitat-lab/habitat/sims/habitat_simulator/object_state_machine.py
Outdated
Show resolved
Hide resolved
| @@ -13,6 +13,18 @@ | |||
| from habitat.sims.habitat_simulator.debug_visualizer import DebugVisualizer | |||
|
|
|||
|
|
|||
| def object_shortname_from_handle(object_handle: str) -> str: | |||
There was a problem hiding this comment.
we have a similar function in hab-llm: https://github.com/facebookresearch/habitat-llm/blob/main/dataset_generation/benchmark_generation/generate_instructions.py#L62 Not sure if it is worth making this a utils function in one place..
There was a problem hiding this comment.
Agreed. Once this merges, I'll refactor downstream code to use this util.
* add abstract implementation of ObjectStateSpec and ObjectStateMachine * add some contrived test logic for the state machine * add an obejct highlight debug line visualization util
* add abstract implementation of ObjectStateSpec and ObjectStateMachine * add some contrived test logic for the state machine * add an obejct highlight debug line visualization util
* add abstract implementation of ObjectStateSpec and ObjectStateMachine * add some contrived test logic for the state machine * add an obejct highlight debug line visualization util
Motivation and Context
Context: we want to simulate more abstract states than physics/geometry. These should be event driven (e.g. "wash dish" action) or temporally driven (e.g. state machine logic in step())
Two identified initial states examples:
Notes on Implementation:If the states are semantically driven, we need something like the MetadataInterface class in habitat-lab to manage registration and set membership. Another option would be to embed the semantic classes directly into the dataset configs so they would be parsed and available from sim.Current concrete implementations are examples and should be updated to reflect true logic better before mergingCurrent module locations within habitat_simulator module are best guess, open to discussion.Update: this PR now contains only the foundation of the feature and basic unit tests. MetadataInterface and original example logic is removed for now.
How Has This Been Tested
added basic tests with contrived scenarios
Types of changes
Checklist