Scipy transformation

[SangbumChoi]

What does this PR do?

Fixes # (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[SangbumChoi]

@NielsRogge It is good to go!

[NielsRogge]

Thanks, does this affect the logits check in the conversion script? If so, how much do they differ?

Feel free to update the conversion script

[SangbumChoi]

@NielsRogge Hi niels. I think your https://huggingface.co/datasets/nielsr/test-image/blob/main/vitpose_batch_data.pt is somehow outdated and gives

  File "/mnt/nas2/users/sbchoi/transformers/src/transformers/models/vitpose/convert_vitpose_to_hf.py", line 229, in convert_vitpose_checkpoint
    assert torch.allclose(pixel_values, original_pixel_values)

I need you to confirm or change this pt

[NielsRogge]

Hmm so that should normally still work from my branch. Let me try:

(env) nielsrogge@Nielss-MacBook-Pro-2 transformers % python src/transformers/models/vitpose/convert_vitpose_to_hf.py
Shape of pixel values: torch.Size([2, 3, 256, 192])
Looks ok!

Yes confirmed. So the pixel values won't be the same anymore with this PR, so we can probably remove that check.

Could you tell how much they differ?

[SangbumChoi]

@NielsRogge Two things to share

1. the image size problem was my mistake I didn't put the output_shape value in the warp affine. Now it is fixed
2. For the difference are like this

pass
torch.from_numpy(pose_results[0]["bbox"]).float()

• tensor([412.8000, 157.6100, 464.8500, 294.6200])
• torch.tensor([412.8, 157.61, 464.85, 294.62])

pass
torch.from_numpy(pose_results[1]["bbox"]).float()

• tensor([384.4300, 172.2100, 398.5500, 206.9500])
• torch.tensor([384.43, 172.21, 398.55, 206.95])

fail
torch.from_numpy(pose_results[1]["keypoints"][0, :3])

• tensor([398.0621, 181.8478, 0.8792])
• torch.tensor([398.07913, 181.82812, 0.88235235])

[NielsRogge]

Thanks a lot! Great work