Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New function to generate humanoid walking motion #2067

Merged
merged 1 commit into from
Sep 6, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,23 @@
from typing import List, Tuple


def bin_search(v, a, b, target_value):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docstring?

left = a
right = b

while left <= right:
mid = (left + right) // 2

if v[mid] == target_value:
return mid
elif v[mid] < target_value:
left = mid + 1
else:
right = mid - 1

return min(left, len(v) - 1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be worthwhile unit testing this one.

We could also use bisect to solve this.



class HumanoidRearrangeController(HumanoidBaseController):
"""
Humanoid Controller, converts high level actions such as walk, or reach into joints positions
Expand Down Expand Up @@ -279,6 +296,80 @@ def calculate_walk_pose(
self.obj_transform_base = obj_transform_base @ rot_offset
self.joint_pose = joint_pose

def translate_and_rotate_with_gait(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is not being called anywhere. It probably should be the default from now on.

It would be worth replacing all calls to calculate_walk_pose_directional and deprecating it.

self, forward_delta: float, rot_delta: float
Copy link
Contributor

@0mdc 0mdc Sep 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because agent velocity is driven from config, we should consider having forward_delta a value between -1.0 and 1.0, and having this function multiply it with the agent's lin_speed.

If we do this, there would be a single source of truth to determine the humanoid agent's velocity.
This would eliminate any discrepancy caused by the implementer failing to read the right config field, and would ease agent config refactoring.

While we're at it, we could use the same velocity variable as the other agents.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This multiplication happens in the BaseVelocityAction, this way we have parity with how the robot moves around.
Here is where we calculate how much to move: https://github.com/facebookresearch/habitat-llm/blob/main_improve_planner/habitat_llm/agent/env/actions.py#L116

This is similar to the robot action, which computes delta and then transforms in here:
https://github.com/facebookresearch/habitat-lab/blob/main/habitat-lab/habitat/tasks/rearrange/actions/actions.py#L489

):
"""
Translates the character in the forward direction and rotates the character

:param forward_delta: how much to move forward or backward
:param rot_delta: how much to rotate (in radians)
"""
forward_vec = self.obj_transform_base.transform_vector(
mn.Vector3(1, 0, 0)
)
self.obj_transform_base.translation += forward_delta * forward_vec
prev_angle = np.arctan2(forward_vec[0], forward_vec[2])
new_angle = prev_angle + rot_delta
new_forward_vec = mn.Vector3(np.cos(new_angle), 0, -np.sin(new_angle))
look_at_path_T = mn.Matrix4.look_at(
self.obj_transform_base.translation,
self.obj_transform_base.translation + new_forward_vec.normalized(),
mn.Vector3.y_axis(),
)
rot_offset = mn.Matrix4.rotation(
mn.Rad(-np.pi / 2), mn.Vector3(1, 0, 0)
)
self.obj_transform_base = look_at_path_T @ rot_offset

# Set a new position frame
if forward_delta != 0:
# Figure out which frame in the motion we should select
current_frame = self.walk_mocap_frame
target_displ = forward_delta
total_displ = self.walk_motion.displacement[-1]
abs_target_displ = np.abs(target_displ)

# If the motion covers multiple cycles, take remainder
if abs_target_displ > total_displ:
num_cycles = int(target_displ / total_displ)
abs_target_displ = abs_target_displ - total_displ * num_cycles
target_displ = np.sign(target_displ) * abs_target_displ

dist_to_end = (
self.walk_motion.displacement[-1]
- self.walk_motion.displacement[current_frame]
)
dist_to_beginning = self.walk_motion.displacement[current_frame]
# Moving X backward is the same as moving Total_displ - X forward
if np.sign(target_displ) < 0:
target_displ += total_displ

if dist_to_end < target_displ:
target_displ -= dist_to_end
a = 0
b = current_frame
else:
a = current_frame
b = len(self.walk_motion.displacement) - 1
target_displ += dist_to_beginning
# Bin search over which frame we should cover
index = bin_search(
self.walk_motion.displacement, a, b, target_displ
)
self.walk_mocap_frame = index

new_pose = self.walk_motion.poses[self.walk_mocap_frame]
joint_pose, obj_transform = new_pose.joints, new_pose.root_transform
# Remove the forward component, and orient according to forward_V
add_rot = mn.Matrix4.rotation(mn.Rad(np.pi), mn.Vector3(0, 1.0, 0))

obj_transform = add_rot @ obj_transform
obj_transform.translation *= mn.Vector3.x_axis() + mn.Vector3.y_axis()

self.joint_pose = joint_pose
self.obj_transform_offset = obj_transform

def calculate_walk_pose_directional(
self,
target_position: mn.Vector3,
Expand Down