Serverless pose estimation format

[hardikdava]

Hello, I want to know about return format for skeleton/pose estimation annotation. I looked into cvat/apps/lambda_manager/views.py and I could not find anything related to skeleton/pose label format. Can you guide me on how/where to get more information about serverless return format?

Related issues: #5816 #5324 #5177

My response from serverless function:

[{'confidence': '0.8', 'label': 'person', 'type': 'skeleton', 'elements': [{'type': 'points', 'label': '0', 'points': [139, 442]}, {'type': 'points', 'label': '1', 'points': [147, 433]}, {'type': 'points', 'label': '2', 'points': [129, 433]}, {'type': 'points', 'label': '3', 'points': [153, 443]}, {'type': 'points', 'label': '4', 'points': [105, 441]}, {'type': 'points', 'label': '5', 'points': [167, 498]}, {'type': 'points', 'label': '6', 'points': [88, 498]}, {'type': 'points', 'label': '7', 'points': [193, 574]}, {'type': 'points', 'label': '8', 'points': [116, 570]}, {'type': 'points', 'label': '9', 'points': [139, 578]}, {'type': 'points', 'label': '10', 'points': [172, 555]}, {'type': 'points', 'label': '11', 'points': [159, 653]}, {'type': 'points', 'label': '12', 'points': [98, 653]}, {'type': 'points', 'label': '13', 'points': [182, 760]}, {'type': 'points', 'label': '14', 'points': [85, 763]}, {'type': 'points', 'label': '15', 'points': [195, 861]}, {'type': 'points', 'label': '16', 'points': [69, 863]}]}, {'confidence': '0.8', 'label': 'person', 'type': 'skeleton', 'elements': [{'type': 'points', 'label': '0', 'points': [788, 432]}, {'type': 'points', 'label': '1', 'points': [794, 425]}, {'type': 'points', 'label': '2', 'points': [791, 423]}, {'type': 'points', 'label': '3', 'points': [805, 431]}, {'type': 'points', 'label': '4', 'points': [806, 423]}, {'type': 'points', 'label': '5', 'points': [809, 486]}, {'type': 'points', 'label': '6', 'points': [809, 479]}, {'type': 'points', 'label': '7', 'points': [786, 575]}, {'type': 'points', 'label': '8', 'points': [806, 559]}, {'type': 'points', 'label': '9', 'points': [743, 558]}, {'type': 'points', 'label': '10', 'points': [771, 544]}, {'type': 'points', 'label': '11', 'points': [801, 641]}, {'type': 'points', 'label': '12', 'points': [800, 636]}, {'type': 'points', 'label': '13', 'points': [768, 735]}, {'type': 'points', 'label': '14', 'points': [778, 737]}, {'type': 'points', 'label': '15', 'points': [722, 849]}, {'type': 'points', 'label': '16', 'points': [729, 848]}]}]

More information:

• If I change "type": "skeleton" to "type": "points" and "points": List of points then it works as keypoint detection.

[bsekachev]

Hi @hardikdava

For now type: "skeleton" is not supported for automatic annotation, we do not have serverless models with type.

[KTXKIKI]

Will there be posture estimation in the future, and is there any type of target classification currently available

[hardikdava]

Hey @bsekachev, I was trying to work with yolov8 human pose estimation model to work with serverless feature. But I do not see anything in user interface. Everything looks fine from serverless function. Let me know if such functionality is supported by cvat server.

[bsekachev]

Please, refer to original issue #5324
Right now it is not supported and we are not working on it actively.
Would be great if somebody could contribute.

[hardikdava]

@bsekachev Thanks for reply. Does the CVAT backend server missing those functionality or external serverless function? For your information, I am working on the external serverless function.

[bsekachev]

CVAT backend, CVAT frontend both don't support serverless functions returning skeletons.

And we do not have such serverless functions themselves.