-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enhancement: XMem mask video propagation as CVAT AI Tools polygon tracker #5465
Comments
My tests have shown that XMem is not lost among many objects with the same texture as the selected mask. |
I also think "Xmem for automatic annotation" is an excellent enhancement project. |
Given the segment results requested per frame, should we have long-term memory for each user? |
I think you are right. Most likely we need a separate memory for each user. |
My actions before raising this issue
There is a great instance segmentation mask propagation tool
[ECCV 2022] XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model
by Deep learning researchers Yue Zhao and Rex Cheng.We have a video, and one instance segmentation mask for the first frame of our video.
We give them to the neural network and it predicts masks for all other frames.
There is a link to git repo.
Features:
SiamMask
, which has a specific set of classes that it can work with)They have a Colab demo with their neural network link. There are easy launch and simple code that can be integrated.
The example from the link is great for creating a potential CVAT interactor of mask polygon tracker type )
There are some demo on video:
It works great and looks as fantastic.
This DL model also have a GUI. We can annotate mask for some objects and then propagate masks for them through all video.
It's interesting that the creators use f-BRS for creating instance segmentation mask for future mask propagation.
CVAT have f-BRS as segmentation interactor too.
But in general, it is not so important what the preliminary mask is created with.
I think other CVAT's AI tool's interactors will work great too (for example HRnet).
It seems like a very intresting tool for CVAT for video instance segmentation tool.
Context
Get a powerful instance segmentation annotation tool for video )
The text was updated successfully, but these errors were encountered: