Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ManagedObject to Receptacle matching #2074

Merged
merged 6 commits into from
Sep 10, 2024
Merged

Conversation

aclegg3
Copy link
Contributor

@aclegg3 aclegg3 commented Sep 6, 2024

Motivation and Context

This PR adds a set of utils targeting the application of matching an object or candidate object placement location to nearby Receptacles.

The util involves a few primary components:

  • point_to_tri_dist - a vectorized 3D point to triangle mesh distance computation util
    • TODO: should be moved to sim geometry utils
  • dist_to_rec - a function for Receptacles (only implemented for TriangleMeshReceptacle currently) to get distance to the receptacle from a point
  • get_obj_receptacle_and_confidence - a utility function for matching objects to receptacles. Includes logic for "floor" receptacles and region matching. Returns matches, a distance based confidence score, and a info_text string with a message explaining failure if applicable.

The util is then integrated with HitL ui to constraint object placements to active Receptacles for better alignment with oracle skills.

How Has This Been Tested

Initially tested in HitL app.
This version tested in episode validation work.
TODO: no unit tests for this functionality yet.

Types of changes

  • [Development] A pull request that add new features to the habitat-lab task and environment codebase. Development Pull Requests must be small (less that 500 lines of code change), have unit testing, very extensive documentation and examples. These are typically new tasks, environments, sensors, etc... The review process for these Pull Request is longer because these changes will be maintained by our core team of developers, so make sure your changes are easy to understand!

Checklist

  • My code follows the code style of this project.
  • I have updated the documentation if required.
  • I have read the CONTRIBUTING document.
  • I have completed my CLA (see CONTRIBUTING)
  • I have added tests to cover my changes if required.

@aclegg3 aclegg3 requested review from zephirefaith and 0mdc September 6, 2024 23:41
@facebook-github-bot facebook-github-bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Sep 6, 2024
@aclegg3 aclegg3 changed the title object to rec matching ManagedObject to Receptacle matching Sep 6, 2024
Comment on lines 1468 to 1481
# TODO: This hack is the solution to the Bullet physics raycast margin bug. It should be migrated into every raycast from habitat-sim and then removed here.
y_buffer = 0.08
raycast_results = sim.cast_ray(
habitat_sim.geo.Ray(
center + mn.Vector3(0, y_buffer, 0), grav_vector
)
)
if raycast_results.has_hits():
for hit in raycast_results.hits:
if hit.object_id != obj.object_id:
support_surface_id = hit.object_id
# print(f"first_hit true dist = {hit.ray_distance - y_buffer}")
if hit.ray_distance >= y_buffer:
break
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, this is the fix/hack which impacts the "above" check. Not integrated there yet, but be on the lookout for a habitat-sim PR.
@jacobkrantz @akshararai

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aclegg3 aclegg3 requested a review from jacobkrantz September 6, 2024 23:45
Copy link
Contributor

@0mdc 0mdc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The feature appears to work well in HITL, but has some usability caveats.
Left some comments.

Copy link
Contributor

@0mdc 0mdc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I tested the feature extensively and it looks robust.

@aclegg3
Copy link
Contributor Author

aclegg3 commented Sep 9, 2024

This one is ready, thanks for the suggestions.
I'll wait to merge until facebookresearch/habitat-sim#2466 is in conda nightly tomorrow.
Edit: I'll rebase at that point to validate that the hack can be removed.

@aclegg3
Copy link
Contributor Author

aclegg3 commented Sep 10, 2024

This one is ready, thanks for the suggestions. I'll wait to merge until facebookresearch/habitat-sim#2466 is in conda nightly tomorrow. Edit: I'll rebase at that point to validate that the hack can be removed.

done

@aclegg3 aclegg3 merged commit 0022324 into main Sep 10, 2024
4 checks passed
@aclegg3 aclegg3 deleted the alex-09_06-object_to_rec_matching branch September 10, 2024 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed Do not delete this pull request or issue due to inactivity.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants