Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HITL - Add selection panel and object state manipulation UI. #2058

Merged
merged 2 commits into from
Sep 4, 2024

Conversation

0mdc
Copy link
Contributor

@0mdc 0mdc commented Aug 31, 2024

Motivation and Context

This changeset adds a selection panel and object state manipulation controls.
The object state change event is recorded to HITL output.

The selection panel shows:

  • Object name.
  • Region name.
  • Contextual information.
    • Contextual controls.
    • Why an action (pick/place/open/close) cannot be done.
  • Object state toggles.

Builds on top of the following recent changes:

Held object contextual information:

selection_ui-2024-08-31_18.03.30.mp4

Object state manipulation:

selection_ui-2024-08-31_17.46.50.mp4

How Has This Been Tested

Tested on multiplayer HITL server with Unity clients.

Types of changes

  • [Development]

Checklist

  • My code follows the code style of this project.
  • I have updated the documentation if required.
  • I have read the CONTRIBUTING document.
  • I have completed my CLA (see CONTRIBUTING)
  • I have added tests to cover my changes if required.

@facebook-github-bot facebook-github-bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Aug 31, 2024
if primary_region is not None:
primary_region_name = primary_region.category.name()

# Get the contextual information.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section will be refactored in a later pass.
This will unify constraints across the application and clean up the UI code.

@aclegg3 : I'll wait until your constraint changes are applied before doing this.

Copy link
Contributor Author

@0mdc 0mdc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some more comments for reviewers.

@@ -391,6 +399,16 @@ def _on_close(self, e: UI.CloseEventData):
}
)

def _on_state_change(self, e: UI.StateChangeEventData):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zephirefaith You can use this callback to signal the agent that a state was changed.

# Only humans can change object states.
can_change_object_states = isinstance(
self.gui_agent_controller, GuiHumanoidController
)
Copy link
Contributor Author

@0mdc 0mdc Aug 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zephirefaith : I intend to move this into a rearrange_v2-level config, along with head sensor definition and feature flags.

Copy link
Contributor

@aclegg3 aclegg3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Loving all of the new UI information. 🚀

contextual_info = "Release to place."
contextual_color = color_ui_valid
else:
contextual_info = "Cannot place object here."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this will be an injection point for more specific place constraint messages

@0mdc 0mdc force-pushed the 0mdc/hitl_selection_panel branch from 252d3eb to 89f622f Compare September 4, 2024 19:06
@0mdc 0mdc merged commit 240adce into main Sep 4, 2024
3 of 4 checks passed
@0mdc 0mdc deleted the 0mdc/hitl_selection_panel branch September 4, 2024 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed Do not delete this pull request or issue due to inactivity.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants