Quality control: honeypot

[nmanovic]

One way to control quality is to implement so called honeypots. An owner of a task can provide annotations for a couple of frames. These frames can be used to calculate quality of the whole task. For example, we need to annotate 1000 frames with bounding boxes for vehicles. The owner of the task can annotate 20 arbiturarly frames (2%). If all frames are annotated correctly we can say that quality is very high (about 100%).

[vivekkalyanarangan30]

@nmanovic is this available for self-hosted version? If yes, can you point me to some documentation on the setup?

[zhiltsov-max]

Implemented for tasks in Release 2.5.0, extended for projects in #6367.

Documentation is available here and here.

[vivekkalyanarangan30]

@zhiltsov-max actually I am unable to see the same interface in spite os using version 2.5 -

[zhiltsov-max]

@vivekkalyanarangan30, I suppose you can be on the develop branch somewhere between the previous release and 2.5.0. Please try to run CVAT from the release tag (git checkout v2.5.0 && CVAT_VERSION=v2.5.0 docker compose up -d).