Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Support SKPS model #2366

Merged
merged 19 commits into from
Jun 12, 2023
Merged

[Feature] Support SKPS model #2366

merged 19 commits into from
Jun 12, 2023

Conversation

610265158
Copy link
Contributor

@610265158 610265158 commented May 17, 2023

Motivation

A simple face keypoints model, achieves 0.394 nme with wflw, which is a SOTA result, and that may help with us.

Modification

Including a head and a codec. Codes placed in project.skps folder.

BC-breaking (Optional)

Use cases (Optional)

Checklist

Before PR:

  • I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
  • Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
  • Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
  • New functionalities are covered by complete unit tests. If not, please add more unit tests to ensure correctness.
  • The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

  • CLA has been signed and all committers have signed the CLA in this PR.

@Tau-J Tau-J changed the title [FEATURE] Lz/skps, New model [Feature] Support SKPS model May 17, 2023
@Tau-J
Copy link
Collaborator

Tau-J commented May 17, 2023

Hi @610265158 , can you share the ckpts and training log with me? You can upload it to an online driver. I'll help to upload it to mmpose server.

@610265158
Copy link
Contributor Author

610265158 commented May 17, 2023

Hi @610265158 , can you share the ckpts and training log with me? You can upload it to an online driver. I'll help to upload it to mmpose server.

I will rerun the code and test the reproducibility , and share them later.

@Tau-J
Copy link
Collaborator

Tau-J commented May 17, 2023

btw, is there any paper or technical report about this model? We're glad to add citation in docs.

@610265158
Copy link
Contributor Author

Hi, the pretrained weight and log is uploaded as in readme.

There is no paper or tech report about the model. However the model is a combine with some techs. The key idea is the head and loss, but not innovative enough.

Refer to my git repo if it could. And there is some postprocess methods that may help in keypoint tasks.
I can add them to mmpose, if you think it helps.

@Tau-J
Copy link
Collaborator

Tau-J commented May 23, 2023

Hi, the pretrained weight and log is uploaded as in readme.

There is no paper or tech report about the model. However the model is a combine with some techs. The key idea is the head and loss, but not innovative enough.

Refer to my git repo if it could. And there is some postprocess methods that may help in keypoint tasks. I can add them to mmpose, if you think it helps.

Of course it helps. We'll appreciate a lot if you can add them to MMPose.

@codecov
Copy link

codecov bot commented May 26, 2023

Codecov Report

Patch coverage: 49.35% and project coverage change: -2.74 ⚠️

Comparison is base (93119ae) 81.99% compared to head (210f6d0) 79.26%.

❗ Current head 210f6d0 differs from pull request most recent head 6c391ff. Consider uploading reports for the commit 6c391ff to get more accurate results

Additional details and impacted files
@@             Coverage Diff             @@
##           dev-1.x    #2366      +/-   ##
===========================================
- Coverage    81.99%   79.26%   -2.74%     
===========================================
  Files          232      248      +16     
  Lines        13638    14859    +1221     
  Branches      2319     2577     +258     
===========================================
+ Hits         11183    11778     +595     
- Misses        1919     2501     +582     
- Partials       536      580      +44     
Flag Coverage Δ
unittests 79.26% <49.35%> (-2.74%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
mmpose/codecs/regression_label.py 87.09% <ø> (ø)
mmpose/models/heads/__init__.py 100.00% <ø> (ø)
mmpose/models/heads/coord_cls_heads/rtmcc_head.py 93.61% <ø> (ø)
mmpose/models/heads/coord_cls_heads/simcc_head.py 90.26% <ø> (ø)
mmpose/models/pose_estimators/base.py 81.08% <0.00%> (-2.26%) ⬇️
mmpose/structures/bbox/transforms.py 68.70% <ø> (ø)
mmpose/apis/inference_3d.py 7.56% <7.56%> (ø)
mmpose/apis/inference.py 68.96% <7.69%> (-7.74%) ⬇️
mmpose/visualization/local_visualizer_3d.py 8.33% <8.33%> (ø)
mmpose/apis/inference_tracking.py 11.53% <11.53%> (ø)
... and 31 more

... and 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@Tau-J
Copy link
Collaborator

Tau-J commented May 26, 2023

Hi @610265158 , could you please fix the lint according to the details?

@610265158
Copy link
Contributor Author

Details

Yes, i will check it soon.


return encoded

def generate_offside_heatmap(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For 'offside', do you mean 'offset'?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'generate_offset_map' seems better. Because it is not actually heatmaps.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's better, I will check them.
And another question. For gt labels returned, which key is better for this offset_map. I place it with displacements now, is it ok?

projects/skps/tools Outdated Show resolved Hide resolved
delete this file , no use
projects/skps/README.md Outdated Show resolved Hide resolved
projects/skps/README.md Outdated Show resolved Hide resolved
@Tau-J Tau-J merged commit 75e8047 into open-mmlab:dev-1.x Jun 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants