Release v2.4.0

cvat-ai · Mar 16, 2023 · 61470b6 · 61470b6
2 parents 2c4676a + 86c9da3
commit 61470b6
Show file tree

Hide file tree

Showing 878 changed files with 34,471 additions and 23,144 deletions.
diff --git a/.eslintrc.js b/.eslintrc.js
@@ -39,7 +39,7 @@ module.exports = {
         'func-names': 0,
         'valid-typeof': 0,
         'no-useless-constructor': 0, // sometimes constructor is necessary to generate right documentation in cvat-core
-        'quotes': ['error', 'single'],
+        'quotes': ['error', 'single', { "avoidEscape": true }],
         'lines-between-class-members': 0,
         'class-methods-use-this': 0,
         'no-underscore-dangle': ['error', { allowAfterThis: true }],

diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md
@@ -4,6 +4,14 @@
 
 <!--- Provide a general summary of the issue in the Title above -->
 
+### Steps to Reproduce (for bugs)
+<!--- Provide a link to a live example or an unambiguous set of steps to
+reproduce this bug. Include code to reproduce, if relevant -->
+1.
+1.
+1.
+1.
+
 ### Expected Behaviour
 <!--- If you're describing a bug, tell us what should happen. If you're
 suggesting a change/improvement, tell us how it should work -->
@@ -14,24 +22,16 @@ behavior. If suggesting a change/improvement, explain the difference from
 current behavior -->
 
 ### Possible Solution
-<!--- Not obligatory, but suggest a fix/reason for the bug, or ideas how
+<!--- Not obligatory, but suggest a fix/reason for the bug, or ideas on how
 to implement the addition or change -->
 
-### Steps to Reproduce (for bugs)
-<!--- Provide a link to a live example or an unambiguous set of steps to
-reproduce this bug. Include code to reproduce, if relevant -->
-1.
-1.
-1.
-1.
-
 ### Context
 <!--- How has this issue affected you? What are you trying to accomplish?
 Providing context helps us come up with a solution that is most useful in
 the real world -->
 
 ### Your Environment
-<!--- Include as many relevant details about the environment you experienced
+<!--- Include relevant details about the environment you experienced
 the bug in -->
 - Git hash commit (`git log -1`):
 - Docker version `docker version` (e.g. Docker 17.0.05):
@@ -42,4 +42,3 @@ the bug in -->
     <details>
     <summary>Logs from `cvat` container</summary>
     </details>
-
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
@@ -1,8 +1,7 @@
-<!-- Raised an issue to propose your change (https://github.com/cvat-ai/cvat/issues).
+<!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues).
 It helps to avoid duplication of efforts from multiple independent contributors.
 Discuss your ideas with maintainers to be sure that changes will be approved and merged.
-Read the [CONTRIBUTION](https://github.com/cvat-ai/cvat/blob/develop/CONTRIBUTING.md)
-guide. -->
+Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). -->
 
 <!-- Provide a general summary of your changes in the Title above -->
 
@@ -18,22 +17,23 @@ see how your change affects other areas of the code, etc. -->
 
 ### Checklist
 <!-- Go over all the following points, and put an `x` in all the boxes that apply.
-If an item isn't applicable by a reason then ~~explicitly strikethrough~~ the whole
-line. If you don't do that github will show an incorrect process for the pull request.
+If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole
+line. If you don't do that, GitHub will show incorrect progress for the pull request.
 If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
 - [ ] I submit my changes into the `develop` branch
-- [ ] I have added a description of my changes into [CHANGELOG](https://github.com/cvat-ai/cvat/blob/develop/CHANGELOG.md) file
-- [ ] I have updated the [documentation](
-  https://github.com/cvat-ai/cvat/blob/develop/README.md#documentation) accordingly
+- [ ] I have added a description of my changes into the [CHANGELOG](https://github.com/opencv/cvat/blob/develop/CHANGELOG.md) file
+- [ ] I have updated the documentation accordingly
 - [ ] I have added tests to cover my changes
-- [ ] I have linked related issues ([read github docs](
+- [ ] I have linked related issues (see [GitHub docs](
   https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))
-- [ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/cvat-ai/cvat/tree/develop/cvat-canvas#versioning),
-[cvat-core](https://github.com/cvat-ai/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/cvat-ai/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/cvat-ai/cvat/tree/develop/cvat-ui#versioning))
+- [ ] I have increased versions of npm packages if it is necessary
+  ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning),
+  [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning),
+  [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and
+  [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))
 
 ### License
 
 - [ ] I submit _my code changes_ under the same [MIT License](
-  https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the project.
+  https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project.
   Feel free to contact the maintainers if that's a concern.
-
diff --git a/.github/workflows/cache.yml b/.github/workflows/cache.yml
@@ -5,42 +5,25 @@ on:
       - 'develop'
 
 jobs:
+  get-sha:
+    uses: ./.github/workflows/search-cache.yml
+
   Caching_CVAT:
+    needs: get-sha
     env:
       GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
       REPO: ${{ github.repository }}
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v3
 
-      - name: Getting SHA with cache from the default branch
-        id: get-sha
-        run: |
-          DEFAULT_BRANCH=$(gh api /repos/$REPO | jq -r '.default_branch')
-          for sha in $(gh api "/repos/$REPO/commits?per_page=100&sha=${DEFAULT_BRANCH}" | jq -r '.[].sha');
-          do
-            RUN_status=$(gh api /repos/${REPO}/actions/workflows/cache.yml/runs | \
-              jq -r ".workflow_runs[]? | select((.head_sha == \"${sha}\") and (.conclusion == \"success\")) | .status")
-
-            if [[ ${RUN_status} == "completed" ]]; then
-              SHA=$sha
-              break
-            fi
-          done
-
-          echo Default branch is ${DEFAULT_BRANCH}
-          echo Workflow will try to get cache from commit: ${SHA}
-
-          echo "default_branch=${DEFAULT_BRANCH}" >> $GITHUB_OUTPUT
-          echo "sha=${SHA}" >> $GITHUB_OUTPUT
-
       - uses: actions/cache@v3
         id: server-cache-action
         with:
           path: /tmp/cvat_cache_server
           key: ${{ runner.os }}-build-server-${{ github.sha }}
           restore-keys: |
-            ${{ runner.os }}-build-server-${{ steps.get-sha.outputs.sha }}
+            ${{ runner.os }}-build-server-${{ needs.get-sha.outputs.sha }}
             ${{ runner.os }}-build-server-
 
       - uses: actions/cache@v3
@@ -49,27 +32,9 @@ jobs:
           path: /tmp/cvat_cache_ui
           key: ${{ runner.os }}-build-ui-${{ github.sha }}
           restore-keys: |
-            ${{ runner.os }}-build-ui-${{ steps.get-sha.outputs.sha }}
+            ${{ runner.os }}-build-ui-${{ needs.get-sha.outputs.sha }}
             ${{ runner.os }}-build-ui-
 
-      - uses: actions/cache@v3
-        id: elasticsearch-cache-action
-        with:
-          path: /tmp/cvat_cache_elasticsearch
-          key: ${{ runner.os }}-build-elasticsearch-${{ github.sha }}
-          restore-keys: |
-            ${{ runner.os }}-build-elasticsearch-${{ steps.get-sha.outputs.sha }}
-            ${{ runner.os }}-build-elasticsearch-
-
-      - uses: actions/cache@v3
-        id: logstash-cache-action
-        with:
-          path: /tmp/cvat_cache_logstash
-          key: ${{ runner.os }}-build-logstash-${{ github.sha }}
-          restore-keys: |
-            ${{ runner.os }}-build-logstash-${{ steps.get-sha.outputs.sha }}
-            ${{ runner.os }}-build-logstash-
-
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v2
 
@@ -89,34 +54,10 @@ jobs:
           cache-from: type=local,src=/tmp/cvat_cache_ui
           cache-to: type=local,dest=/tmp/cvat_cache_ui-new
 
-      - name: Caching CVAT Elasticsearch
-        uses: docker/build-push-action@v2
-        with:
-          context: ./components/analytics/elasticsearch/
-          file: ./components/analytics/elasticsearch/Dockerfile
-          cache-from: type=local,src=/tmp/cvat_cache_elasticsearch
-          cache-to: type=local,dest=/tmp/cvat_cache_elasticsearch-new
-          build-args: ELK_VERSION=6.8.23
-
-      - name: Caching CVAT Logstash
-        uses: docker/build-push-action@v2
-        with:
-          context: ./components/analytics/logstash/
-          file: ./components/analytics/logstash/Dockerfile
-          cache-from: type=local,src=/tmp/cvat_cache_logstash
-          cache-to: type=local,dest=/tmp/cvat_cache_logstash-new
-          build-args: ELK_VERSION=6.8.23
-
       - name: Moving cache
         run: |
           rm -rf /tmp/cvat_cache_server
           mv /tmp/cvat_cache_server-new /tmp/cvat_cache_server
 
           rm -rf /tmp/cvat_cache_ui
           mv /tmp/cvat_cache_ui-new /tmp/cvat_cache_ui
-
-          rm -rf /tmp/cvat_cache_elasticsearch
-          mv /tmp/cvat_cache_elasticsearch-new /tmp/cvat_cache_elasticsearch
-
-          rm -rf /tmp/cvat_cache_logstash
-          mv /tmp/cvat_cache_logstash-new /tmp/cvat_cache_logstash
diff --git a/.github/workflows/full.yml b/.github/workflows/full.yml
@@ -17,31 +17,7 @@ env:
 
 jobs:
   search_cache:
-    runs-on: ubuntu-latest
-    outputs:
-      sha: ${{ steps.get-sha.outputs.sha}}
-    env:
-      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      REPO: ${{ github.repository }}
-    steps:
-      - name: Getting SHA with cache from the default branch
-        id: get-sha
-        run: |
-          DEFAULT_BRANCH=$(gh api /repos/$REPO | jq -r '.default_branch')
-          for sha in $(gh api "/repos/$REPO/commits?per_page=100&sha=$DEFAULT_BRANCH" | jq -r '.[].sha');
-          do
-            RUN_status=$(gh api /repos/${REPO}/actions/workflows/cache.yml/runs | \
-              jq -r ".workflow_runs[]? | select((.head_sha == \"${sha}\") and (.conclusion == \"success\")) | .status")
-            if [[ ${RUN_status} == "completed" ]]; then
-              SHA=$sha
-              break
-            fi
-          done
-          echo Default branch is ${DEFAULT_BRANCH}
-          echo Workflow will try to get cache from commit: ${SHA}
-
-          echo "default_branch=${DEFAULT_BRANCH}" >> $GITHUB_OUTPUT
-          echo "sha=${SHA}" >> $GITHUB_OUTPUT
+    uses: ./.github/workflows/search-cache.yml
 
   build:
     needs: search_cache
@@ -65,6 +41,10 @@ jobs:
 
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v2
+        with:
+          # we specify version of buildkit due to issue that occurs in latest version
+          # https://github.com/moby/buildkit/issues/2631
+          driver-opts: image=moby/buildkit:v0.10.0
 
       - name: Create artifact directories
         run: |
@@ -97,9 +77,7 @@ jobs:
             --entrypoint /bin/bash -u root cvat/server \
             -c 'python manage.py spectacular --file /transfer/schema.yml'
           pip3 install --user -r cvat-sdk/gen/requirements.txt
-          cd cvat-sdk/
-          gen/generate.sh
-          cd ..
+          ./cvat-sdk/gen/generate.sh
 
           cp -r cvat-sdk/* /tmp/cvat_sdk/
 
@@ -136,38 +114,6 @@ jobs:
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@master
 
-      - name: Getting CVAT Elasticsearch cache from the default branch
-        uses: actions/cache@v3
-        with:
-          path: /tmp/cvat_cache_elasticsearch
-          key: ${{ runner.os }}-build-elasticsearch-${{ needs.search_cache.outputs.sha }}
-
-      - name: Getting CVAT Logstash cache from the default branch
-        uses: actions/cache@v3
-        with:
-          path: /tmp/cvat_cache_logstash
-          key: ${{ runner.os }}-build-logstash-${{ needs.search_cache.outputs.sha }}
-
-      - name: Building CVAT Elasticsearch
-        uses: docker/build-push-action@v2
-        with:
-          context: ./components/analytics/elasticsearch/
-          file: ./components/analytics/elasticsearch/Dockerfile
-          cache-from: type=local,src=/tmp/cvat_cache_elasticsearch
-          tags: cvat_elasticsearch:latest
-          load: true
-          build-args: ELK_VERSION=6.8.23
-
-      - name: Building CVAT Logstash
-        uses: docker/build-push-action@v2
-        with:
-          context: ./components/analytics/logstash/
-          file: ./components/analytics/logstash/Dockerfile
-          cache-from: type=local,src=/tmp/cvat_cache_logstash
-          tags: cvat_logstash:latest
-          load: true
-          build-args: ELK_VERSION=6.8.23
-
       - name: Download CVAT server image
         uses: actions/download-artifact@v3
         with:
@@ -208,7 +154,8 @@ jobs:
         run: |
             mkdir $LOGS_DIR
             docker logs test_cvat_server_1 > $LOGS_DIR/cvat_server.log
-            docker logs test_cvat_worker_default_1 > $LOGS_DIR/cvat_worker_default.log
+            docker logs test_cvat_worker_export_1 > $LOGS_DIR/cvat_worker_export.log
+            docker logs test_cvat_worker_import_1 > $LOGS_DIR/cvat_worker_import.log
             docker logs test_cvat_opa_1 2> $LOGS_DIR/cvat_opa.log
 
       - name: Uploading "cvat" container logs as an artifact
@@ -246,7 +193,7 @@ jobs:
           python cvat/apps/iam/rules/tests/generate_tests.py \
             --output-dir cvat/apps/iam/rules/
 
-          curl -L -o opa https://openpolicyagent.org/downloads/v0.34.2/opa_linux_amd64_static
+          curl -L -o opa https://openpolicyagent.org/downloads/v0.45.0/opa_linux_amd64_static
           chmod +x ./opa
           ./opa test cvat/apps/iam/rules
 
@@ -255,14 +202,14 @@ jobs:
           HOST_COVERAGE_DATA_DIR: ${{ github.workspace }}
           CONTAINER_COVERAGE_DATA_DIR: "/coverage_data"
         run: |
-          docker-compose -f docker-compose.yml -f docker-compose.dev.yml up -d cvat_opa cvat_server
+          docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d cvat_opa cvat_server
           max_tries=12
           while [[  $(curl -s -o /dev/null -w "%{http_code}" localhost:8181/health?bundles) != "200" && max_tries -gt 0 ]]; do (( max_tries-- )); sleep 5; done
 
-          docker-compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.ci.yml run cvat_ci /bin/bash \
+          docker compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.ci.yml run cvat_ci /bin/bash \
             -c 'python manage.py test cvat/apps -v 2'
 
-          docker-compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.ci.yml run cvat_ci /bin/bash \
+          docker compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.ci.yml run cvat_ci /bin/bash \
             -c 'yarn --frozen-lockfile --ignore-scripts && yarn workspace cvat-core run test'
 
       - name: Creating a log file from cvat containers
@@ -325,7 +272,7 @@ jobs:
 
       - name: Run CVAT instance
         run: |
-          docker-compose \
+          docker compose \
             -f docker-compose.yml \
             -f docker-compose.dev.yml \
             -f components/serverless/docker-compose.serverless.yml \
@@ -363,13 +310,13 @@ jobs:
               --headed \
               --browser chrome \
               --env coverage=false \
-              --config-file cypress_canvas3d.json \
-              --spec 'cypress/integration/${{ matrix.specs }}/**/*.js,cypress/integration/remove_users_tasks_projects_organizations.js'
+              --config-file cypress_canvas3d.config.js \
+              --spec 'cypress/e2e/${{ matrix.specs }}/**/*.js,cypress/e2e/remove_users_tasks_projects_organizations.js'
           else
             npx cypress run \
               --browser chrome \
               --env coverage=false \
-              --spec 'cypress/integration/${{ matrix.specs }}/**/*.js,cypress/integration/remove_users_tasks_projects_organizations.js'
+              --spec 'cypress/e2e/${{ matrix.specs }}/**/*.js,cypress/e2e/remove_users_tasks_projects_organizations.js'
           fi
 
       - name: Creating a log file from "cvat" container logs