-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Validation API core #8348
base: develop
Are you sure you want to change the base?
Validation API core #8348
Conversation
…sk creation (put chunk creation into the end), need to update chunk generation approach to to per job
@klakhov ,
Hm, I think yes, it should produce a validation error during task creation.
I think it will be ether in a separate PR or in the PR with allocation reports, it seems to be the most relevant place. |
@coderabbitai review |
✅ Actions performedReview triggered.
|
cvat/apps/dataset_manager/task.py
Outdated
if data.tracks and db_data.validation.mode == models.ValidationMode.GT_POOL: | ||
# Only tags and shapes can be used in tasks with GT pool | ||
raise ValidationError("Tracks are not supported when task validation mode is {}".format( | ||
models.ValidationMode.GT_POOL | ||
)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, fixed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zhiltsov-max What is the nature of such restriction?
I understand that tracks with interpolation do not make sense in honeypot jobs, but is it the only reason?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Frames are in the random order, tracks make no sense in such case. If the frames were ordered, honeypots could be (a) clearly identifiable or (b) could not be reused multiple times. Simple GT job doesn't have such a problem, so it can be used if frame ordering or tracks are required in the task.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It makes sense if each frame of the track is a keyframe.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just do not understand the reason why we forced this restriction.
If a user does it, why not. It is the user's decision.
Quality Gate passedIssues Measures |
Motivation and context
Depends on #8272
Depends on #8321
random_per_job
, which guarantees each annotation job gets the specified GT overlap, making each annotation job validatableServer API changes:
validation_mode
field, reflecting the current validation configuration (immutable)validation_params
field, which allow to enable GT / GT_POOL validation for a task on its creationTasks with Honeypots
This validation mode affects task creation, so can only be used in task creation. It cannot be disabled or changed after the task is created. When honeypots are configured, each job in the task gets several extra validation frames.
The pool of available frames and the number of validation frames per job are specified by the user at task creation.
Limitations:
random
frame ordering.job_frame_mapping
andoverlap
cannot be used in such tasks.Honeypot frames and GT annotations are accessible via the GT job, as in the case with regular GT jobs. However, unlike regular tasks with GT jobs, task annotation import affects the GT job as well in tasks with honeypots. Task annotation export contains only GT annotations on validation frames (so, only the GT copy of validation frames is included).
How has this been tested?
Checklist
develop
branch(cvat-canvas,
cvat-core,
cvat-data and
cvat-ui)
License
Feel free to contact the maintainers if that's a concern.
Summary by CodeRabbit
New Features
Bug Fixes
Documentation
Chores