Merge branch 'develop' into ay/extra-anno-in-cvat

.github/workflows/full.yml

@@ -41,6 +41,10 @@ jobs:
 
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v2
+        with:
+          # we specify version of buildkit due to issue that occurs in latest version
+          # https://github.com/moby/buildkit/issues/2631
+          driver-opts: image=moby/buildkit:v0.10.0
 
       - name: Create artifact directories
         run: |
@@ -343,13 +347,13 @@ jobs:
               --headed \
               --browser chrome \
               --env coverage=false \
-              --config-file cypress_canvas3d.json \
-              --spec 'cypress/integration/${{ matrix.specs }}/**/*.js,cypress/integration/remove_users_tasks_projects_organizations.js'
+              --config-file cypress_canvas3d.config.js \
+              --spec 'cypress/e2e/${{ matrix.specs }}/**/*.js,cypress/e2e/remove_users_tasks_projects_organizations.js'
           else
             npx cypress run \
               --browser chrome \
               --env coverage=false \
-              --spec 'cypress/integration/${{ matrix.specs }}/**/*.js,cypress/integration/remove_users_tasks_projects_organizations.js'
+              --spec 'cypress/e2e/${{ matrix.specs }}/**/*.js,cypress/e2e/remove_users_tasks_projects_organizations.js'
           fi
 
       - name: Creating a log file from "cvat" container logs

.github/workflows/main.yml

@@ -302,13 +302,13 @@ jobs:
               --headed \
               --browser chrome \
               --env coverage=false \
-              --config-file cypress_canvas3d.json \
-              --spec 'cypress/integration/${{ matrix.specs }}/**/*.js,cypress/integration/remove_users_tasks_projects_organizations.js'
+              --config-file cypress_canvas3d.config.js \
+              --spec 'cypress/e2e/${{ matrix.specs }}/**/*.js,cypress/e2e/remove_users_tasks_projects_organizations.js'
           else
             npx cypress run \
               --browser chrome \
               --env coverage=false \
-              --spec 'cypress/integration/${{ matrix.specs }}/**/*.js,cypress/integration/remove_users_tasks_projects_organizations.js'
+              --spec 'cypress/e2e/${{ matrix.specs }}/**/*.js,cypress/e2e/remove_users_tasks_projects_organizations.js'
           fi
 
       - name: Creating a log file from "cvat" container logs

.github/workflows/schedule.yml

@@ -347,13 +347,13 @@ jobs:
             npx cypress run \
               --headed \
               --browser chrome \
-              --config-file cypress_canvas3d.json \
-              --spec 'cypress/integration/${{ matrix.specs }}/**/*.js,cypress/integration/remove_users_tasks_projects_organizations.js'
+              --config-file cypress_canvas3d.config.js \
+              --spec 'cypress/e2e/${{ matrix.specs }}/**/*.js,cypress/e2e/remove_users_tasks_projects_organizations.js'
             mv ./.nyc_output/out.json ./.nyc_output/out_${{ matrix.specs }}.json
           else
             npx cypress run \
               --browser chrome \
-              --spec 'cypress/integration/${{ matrix.specs }}/**/*.js,cypress/integration/remove_users_tasks_projects_organizations.js'
+              --spec 'cypress/e2e/${{ matrix.specs }}/**/*.js,cypress/e2e/remove_users_tasks_projects_organizations.js'
             mv ./.nyc_output/out.json ./.nyc_output/out_${{ matrix.specs }}.json
           fi
 

README.md

@@ -2,8 +2,6 @@
 
 # Computer Vision Annotation Tool (CVAT)
 
-<a href="https://www.producthunt.com/posts/cvat-computer-vision-annotation-tool?utm_source=badge-featured&utm_medium=badge&utm_souce=badge-cvat&#0045;computer&#0045;vision&#0045;annotation&#0045;tool" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=353415&theme=light" alt="CVAT&#0032;–&#0032;Computer&#0032;Vision&#0032;Annotation&#0032;Tool - The&#0032;open&#0032;data&#0032;annotation&#0032;platform&#0032;for&#0032;AI | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
-
 [![CI][ci-img]][ci-url]
 [![Gitter chat][gitter-img]][gitter-url]
 [![Discord][discord-img]][discord-url]
@@ -18,14 +16,17 @@ companies around the world. Our mission is to help developers, companies, and
 organizations around the world to solve real problems using the Data-centric
 AI approach.
 
-CVAT is free and open-source.
-
-**A new repo**: CVAT core team moved the active development of the tool
-to this new repository.
+Start using CVAT online: [cvat.ai](https://cvat.ai). You can use it for free,
+or [subscribe](https://www.cvat.ai/pricing/cloud) to get unlimited data,
+organizations, autoannotations, and [Roboflow and HuggingFace integration](
+  https://www.cvat.ai/post/integrating-hugging-face-and-roboflow-models).
 
-Start using CVAT online for free: [cvat.ai](https://cvat.ai).
-Or set it up as a self-hosted solution:
+Or set CVAT up as a self-hosted solution:
 [Self-hosted Installation Guide](https://opencv.github.io/cvat/docs/administration/basics/installation/).
+We provide [Enterprise support](https://www.cvat.ai/pricing/on-prem) for
+self-hosted installations with premium features: SSO, LDAP, Roboflow and
+HuggingFace integrations, and advanced analytics (coming soon). We also
+do trainings and a dedicated support with 24 hour SLA.
 
 ![CVAT screencast](site/content/en/images/cvat-ai-screencast.gif)
 
@@ -95,10 +96,19 @@ Here are some screencasts showing how to use CVAT.
 
 <!--lint disable maximum-line-length-->
 
-[Computer Vision Annotation Course](https://www.youtube.com/playlist?list=PL0to7Ng4PuuYQT4eXlHb_oIlq_RPeuasN): we introduce our course series designed to help you annotate data faster and better using CVAT. This course is about CVAT deployment and integrations, it includes presentations and covers the following topics:
-
-- **Speeding up your data annotation process: introduction to CVAT and Datumaro**. What problems do CVAT and Datumaro solve, and how they can speed up your model training process. Some resources you can use to learn more about how to use them.
-- **Deployment and use CVAT**. Use the app online at [app.cvat.ai](app.cvat.ai). A local deployment. A containerized local deployment with Docker Compose (for regular use), and a local cluster deployment with Kubernetes (for enterprise users). A 2-minute tour of the interface, a breakdown of CVAT’s internals, and a demonstration of how to deploy CVAT using Docker Compose.
+[Computer Vision Annotation Course](https://www.youtube.com/playlist?list=PL0to7Ng4PuuYQT4eXlHb_oIlq_RPeuasN):
+we introduce our course series designed to help you annotate data faster and better
+using CVAT. This course is about CVAT deployment and integrations, it includes
+presentations and covers the following topics:
+
+- **Speeding up your data annotation process: introduction to CVAT and Datumaro**.
+What problems do CVAT and Datumaro solve, and how they can speed up your model
+training process. Some resources you can use to learn more about how to use them.
+- **Deployment and use CVAT**. Use the app online at [app.cvat.ai](app.cvat.ai).
+A local deployment. A containerized local deployment with Docker Compose (for regular use),
+and a local cluster deployment with Kubernetes (for enterprise users). A 2-minute
+tour of the interface, a breakdown of CVAT’s internals, and a demonstration of how
+to deploy CVAT using Docker Compose.
 
 [Product tour](https://www.youtube.com/playlist?list=PL0to7Ng4Puua37NJVMIShl_pzqJTigFzg): in this course, we show how to use CVAT, and help to get familiar with CVAT functionality and interfaces. This course does not cover integrations and is dedicated solely to CVAT. It covers the following topics:
 

cvat/apps/engine/filters.py

@@ -45,7 +45,7 @@ def get_schema_fields(self, view):
 
         search_fields = getattr(view, 'search_fields', [])
         full_description = self.search_description + \
-            f' Avaliable search_fields: {search_fields}'
+            f' Available search_fields: {search_fields}'
 
         return [
             coreapi.Field(
@@ -62,7 +62,7 @@ def get_schema_fields(self, view):
     def get_schema_operation_parameters(self, view):
         search_fields = getattr(view, 'search_fields', [])
         full_description = self.search_description + \
-            f' Avaliable search_fields: {search_fields}'
+            f' Available search_fields: {search_fields}'
 
         return [{
             'name': self.search_param,
@@ -100,7 +100,7 @@ def get_schema_fields(self, view):
 
         ordering_fields = getattr(view, 'ordering_fields', [])
         full_description = self.ordering_description + \
-            f' Avaliable ordering_fields: {ordering_fields}'
+            f' Available ordering_fields: {ordering_fields}'
 
         return [
             coreapi.Field(
@@ -117,7 +117,7 @@ def get_schema_fields(self, view):
     def get_schema_operation_parameters(self, view):
         ordering_fields = getattr(view, 'ordering_fields', [])
         full_description = self.ordering_description + \
-            f' Avaliable ordering_fields: {ordering_fields}'
+            f' Available ordering_fields: {ordering_fields}'
 
         return [{
             'name': self.ordering_param,
@@ -206,7 +206,7 @@ def get_schema_fields(self, view):
 
         filter_fields = getattr(view, 'filter_fields', [])
         full_description = self.filter_description + \
-            f' Avaliable filter_fields: {filter_fields}'
+            f' Available filter_fields: {filter_fields}'
 
         return [
             coreapi.Field(
@@ -223,7 +223,7 @@ def get_schema_fields(self, view):
     def get_schema_operation_parameters(self, view):
         filter_fields = getattr(view, 'filter_fields', [])
         full_description = self.filter_description + \
-            f' Avaliable filter_fields: {filter_fields}'
+            f' Available filter_fields: {filter_fields}'
         return [
             {
                 'name': self.filter_param,

cvat/apps/engine/serializers.py

@@ -642,24 +642,27 @@ def create_labels(labels, parent_label=None):
                 sublabels = label.pop('sublabels', [])
                 svg = label.pop('svg', '')
                 db_label = models.Label.objects.create(task=db_task, parent=parent_label, **label)
+                logger.info(f'label:create: Label id:{db_label.id} for spec:{label} with sublabels:{sublabels}, parent_label:{parent_label}')
                 create_labels(sublabels, parent_label=db_label)
                 if db_label.type == str(models.LabelType.SKELETON):
                     for db_sublabel in list(db_label.sublabels.all()):
                         svg = svg.replace(f'data-label-name="{db_sublabel.name}"', f'data-label-id="{db_sublabel.id}"')
-                    models.Skeleton.objects.create(root=db_label, svg=svg)
+                    db_skeleton = models.Skeleton.objects.create(root=db_label, svg=svg)
+                    logger.info(f'label:create Skeleton id:{db_skeleton.id} for label_id:{db_label.id}')
 
                 for attr in attributes:
                     if attr.get('id', None):
                         del attr['id']
                     models.AttributeSpec.objects.create(label=db_label, **attr)
 
-        create_labels(labels)
         task_path = db_task.get_dirname()
         if os.path.isdir(task_path):
             shutil.rmtree(task_path)
 
         os.makedirs(db_task.get_task_logs_dirname())
         os.makedirs(db_task.get_task_artifacts_dirname())
+        logger = slogger.task[db_task.id]
+        create_labels(labels)
 
         db_task.save()
         return db_task
@@ -674,16 +677,22 @@ def update(self, instance, validated_data):
         instance.subset = validated_data.get('subset', instance.subset)
         labels = validated_data.get('label_set', [])
 
+        logger = slogger.task[instance.id]
         def update_labels(labels, parent_label=None):
             for label in labels:
                 sublabels = label.pop('sublabels', [])
                 svg = label.pop('svg', '')
                 db_label = LabelSerializer.update_instance(label, instance, parent_label)
+                if db_label:
+                    logger.info(f'label:update Label id:{db_label.id} for spec:{label} with sublabels:{sublabels}, parent_label:{parent_label}')
+                else:
+                    logger.info(f'label:delete label:{label} with sublabels:{sublabels}, parent_label:{parent_label}')
                 update_labels(sublabels, parent_label=db_label)
                 if label.get('id') is None and db_label.type == str(models.LabelType.SKELETON):
                     for db_sublabel in list(db_label.sublabels.all()):
                         svg = svg.replace(f'data-label-name="{db_sublabel.name}"', f'data-label-id="{db_sublabel.id}"')
-                    models.Skeleton.objects.create(root=db_label, svg=svg)
+                    db_skeleton = models.Skeleton.objects.create(root=db_label, svg=svg)
+                    logger.info(f'label:update Skeleton id:{db_skeleton.id} for label_id:{db_label.id}')
 
         if instance.project_id is None:
             update_labels(labels)
@@ -870,24 +879,27 @@ def create_labels(labels, parent_label=None):
                 sublabels = label.pop('sublabels', [])
                 svg = label.pop('svg', [])
                 db_label = models.Label.objects.create(project=db_project, parent=parent_label, **label)
+                logger.info(f'label:create Label id:{db_label.id} for spec:{label} with sublabels:{sublabels}, parent_label:{parent_label}')
                 create_labels(sublabels, parent_label=db_label)
                 if db_label.type == str(models.LabelType.SKELETON):
                     for db_sublabel in list(db_label.sublabels.all()):
                         svg = svg.replace(f'data-label-name="{db_sublabel.name}"', f'data-label-id="{db_sublabel.id}"')
-                    models.Skeleton.objects.create(root=db_label, svg=svg)
+                    db_skeleton = models.Skeleton.objects.create(root=db_label, svg=svg)
+                    logger.info(f'label:create Skeleton id:{db_skeleton.id} for label_id:{db_label.id}')
 
                 for attr in attributes:
                     if attr.get('id', None):
                         del attr['id']
                     models.AttributeSpec.objects.create(label=db_label, **attr)
 
-        create_labels(labels)
-
         project_path = db_project.get_dirname()
         if os.path.isdir(project_path):
             shutil.rmtree(project_path)
         os.makedirs(db_project.get_project_logs_dirname())
 
+        logger = slogger.project[db_project.id]
+        create_labels(labels)
+
         return db_project
 
     # pylint: disable=no-self-use
@@ -898,18 +910,24 @@ def update(self, instance, validated_data):
         instance.bug_tracker = validated_data.get('bug_tracker', instance.bug_tracker)
         labels = validated_data.get('label_set', [])
 
+        logger = slogger.project[instance.id]
         def update_labels(labels, parent_label=None):
             for label in labels:
                 sublabels = label.pop('sublabels', [])
                 svg = label.pop('svg', '')
                 db_label = LabelSerializer.update_instance(label, instance, parent_label)
+                if db_label:
+                    logger.info(f'label:update Label id:{db_label.id} for spec:{label} with sublabels:{sublabels}, parent_label:{parent_label}')
+                else:
+                    logger.info(f'label:delete label:{label} with sublabels:{sublabels}, parent_label:{parent_label}')
                 if not label.get('deleted'):
                     update_labels(sublabels, parent_label=db_label)
 
                     if label.get('id') is None and db_label.type == str(models.LabelType.SKELETON):
                         for db_sublabel in list(db_label.sublabels.all()):
                             svg = svg.replace(f'data-label-name="{db_sublabel.name}"', f'data-label-id="{db_sublabel.id}"')
-                        models.Skeleton.objects.create(root=db_label, svg=svg)
+                        db_skeleton = models.Skeleton.objects.create(root=db_label, svg=svg)
+                        logger.info(f'label:update: Skeleton id:{db_skeleton.id} for label_id:{db_label.id}')
 
         update_labels(labels)
 

site/content/en/docs/administration/advanced/k8s_deployment_with_helm.md

@@ -115,8 +115,8 @@ Before starting, ensure that the following prerequisites are met:
      minikube addons enable registry
      minikube addons enable registry-aliases
      ```
-     Before Docker container images can be pushed to your newly created unsecure registry,
-     you need to add its address (`$(minikube ip):5000`) to the list of unsecure registries to
+     Before Docker container images can be pushed to your newly created insecure registry,
+     you need to add its address (`$(minikube ip):5000`) to the list of insecure registries to
      instruct Docker to accept working against it:
      follow the instructions in the [Docker documentation](https://docs.docker.com/registry/insecure/#deploy-a-plain-http-registry)
 
@@ -127,7 +127,7 @@ Before starting, ensure that the following prerequisites are met:
    ```shell
    nuctl --namespace <your cvat namespace> create project cvat
    ```
-1. Finaly deploy the fuction, i.e.:
+1. Finally deploy the function, i.e.:
    - using minikube registry:
      ```shell
      nuctl deploy --project-name cvat --path serverless/tensorflow/faster_rcnn_inception_v2_coco/nuclio --registry $(minikube ip):5000 --run-registry registry.minikube
@@ -311,7 +311,7 @@ Then reference it in helm update/install command using `-f` flag
 ### Why you used external charts to provide redis and postgres?
 Because they definitely know what they do better then we are, so we are getting more quality and less support
 ### How to use custom domain name with k8s deployment:
-The default value `cvat.local` may be overriden with `--set ingress.hosts[0].host` option like this:
+The default value `cvat.local` may be overridden with `--set ingress.hosts[0].host` option like this:
 ```shell
 helm upgrade -n default cvat -i --create-namespace helm-chart -f helm-chart/values.yaml -f helm-chart/values.override.yaml --set ingress.hosts[0].host=YOUR_FQDN
 ```

site/content/en/docs/administration/advanced/upgrade_guide.md

@@ -102,7 +102,7 @@ docker compose up -d
 1. It is highly recommended backup all CVAT data before updating, follow the
    [backup guide](/docs/administration/advanced/backup_guide/) and backup CVAT database volume.
 
-1. Run previosly used CVAT version as usual
+1. Run previously used CVAT version as usual
 
 1. Backup current database with `pg_dumpall` tool:
    ```shell

site/content/en/docs/administration/basics/AWS-Deployment-Guide.md

@@ -22,7 +22,7 @@ There are two ways of deploying the CVAT.
    [installation instructions](/docs/administration/basics/installation/).
    The additional step is to add a [security group and rule to allow incoming connections](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html).
 
-For any of above, don't forget to set the `CVAT_HOST` environemnt variable to the exposed
+For any of above, don't forget to set the `CVAT_HOST` environment variable to the exposed
 AWS public IP address or hostname:
 
 ```bash

site/content/en/docs/administration/basics/admin-account.md

@@ -5,7 +5,7 @@ weight: 2
 description: 'A CVAT installation guide to create a superuser.'
 ---
 
-This section is for users who whant to be a bit more flexible with CVAT use.
+This section is for users who want to be a bit more flexible with CVAT use.
 
 The user you register by default does not have full permissions on the instance,
 so you must create a superuser.