Support running CVAT with an external database via Docker Compose

With Docker Compose, you can initialize a secret from an environment
variable, but you can't load a secret _into_ an environment variable. So in
order to be able to read the DB password from a secret, I had to introduce a
new `CVAT_DB_PASSWORD_FILE` variable.

backend_entrypoint.sh

@@ -8,7 +8,7 @@ fail() {
 }
 
 wait_for_db() {
-    ~/wait-for-it.sh "${CVAT_POSTGRES_HOST}:5432" -t 0
+    ~/wait-for-it.sh "${CVAT_POSTGRES_HOST}:${CVAT_POSTGRES_PORT:-5432}" -t 0
 }
 
 cmd_bash() {

changelog.d/20231024_190737_roman_docker_compose_external_db.md

@@ -0,0 +1,4 @@
+### Added
+
+- Support for using an external database in a Docker Compose-based deployment
+  (<https://github.com/opencv/cvat/pull/7055>)

cvat/settings/base.py

@@ -33,6 +33,8 @@
 
 from pathlib import Path
 
+from django.core.exceptions import ImproperlyConfigured
+
 # Build paths inside the project like this: os.path.join(BASE_DIR, ...)
 BASE_DIR = str(Path(__file__).parents[2])
 
@@ -673,6 +675,17 @@ class CVAT_QUEUES(Enum):
     }
 }
 
+if (postgres_password_file := os.getenv('CVAT_POSTGRES_PASSWORD_FILE')) is not None:
+    if 'CVAT_POSTGRES_PASSWORD' in os.environ:
+        raise ImproperlyConfigured(
+            'The CVAT_POSTGRES_PASSWORD and CVAT_POSTGRES_PASSWORD_FILE'
+            ' environment variables must not be set at the same time'
+        )
+
+    postgres_password = Path(postgres_password_file).read_text(encoding='UTF-8').rstrip('\n')
+else:
+    postgres_password = os.getenv('CVAT_POSTGRES_PASSWORD', '')
+
 # Database
 # https://docs.djangoproject.com/en/3.2/ref/settings/#databases
 DATABASES = {
@@ -681,7 +694,7 @@ class CVAT_QUEUES(Enum):
         'HOST': os.getenv('CVAT_POSTGRES_HOST', 'cvat_db'),
         'NAME': os.getenv('CVAT_POSTGRES_DBNAME', 'cvat'),
         'USER': os.getenv('CVAT_POSTGRES_USER', 'root'),
-        'PASSWORD': os.getenv('CVAT_POSTGRES_PASSWORD', ''),
+        'PASSWORD': postgres_password,
         'PORT': os.getenv('CVAT_POSTGRES_PORT', 5432),
     }
 }

docker-compose.external_db.yml

@@ -0,0 +1,33 @@
+# Copyright (C) 2023 CVAT.ai Corporation
+#
+# SPDX-License-Identifier: MIT
+
+# This optional Docker Compose file may be used to deploy CVAT with an external database.
+
+x-backend-settings: &backend-settings
+  environment:
+    CVAT_POSTGRES_HOST:
+    CVAT_POSTGRES_PORT:
+    CVAT_POSTGRES_DBNAME:
+    CVAT_POSTGRES_USER:
+    CVAT_POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
+  secrets:
+    - postgres_password
+
+services:
+  cvat_db:
+    deploy:
+      replicas: 0
+
+  cvat_server: *backend-settings
+  cvat_utils: *backend-settings
+  cvat_worker_analytics_reports: *backend-settings
+  cvat_worker_annotation: *backend-settings
+  cvat_worker_export: *backend-settings
+  cvat_worker_import: *backend-settings
+  cvat_worker_quality_reports: *backend-settings
+  cvat_worker_webhooks: *backend-settings
+
+secrets:
+  postgres_password:
+    environment: CVAT_POSTGRES_PASSWORD

site/content/en/docs/administration/basics/installation.md

@@ -521,6 +521,30 @@ docker compose -f docker-compose.yml -f docker-compose.https.yml up -d
 
 Then, the CVAT instance will be available at your domain on ports 443 (HTTPS) and 80 (HTTP, redirects to 443).
 
+### Deploy CVAT with an external database
+
+By default, `docker compose up` will start a PostgreSQL database server,
+which will be used to store CVAT's data.
+If you'd like to use your own PostgreSQL instance instead, you can do so as follows.
+Note that CVAT only supports the same major version of PostgreSQL
+as is used in `docker-compose.yml`.
+
+First, define environment variables with database connection settings:
+
+```shell
+export CVAT_POSTGRES_HOST=<PostgreSQL hostname> # mandatory
+export CVAT_POSTGRES_PORT=<PostgreSQL port> # defaults to 5432
+export CVAT_POSTGRES_DBNAME=<PostgreSQL database name> # defaults to "cvat"
+export CVAT_POSTGRES_USER=<PostgreSQL role name> # defaults to "root"
+export CVAT_POSTGRES_PASSWORD=<PostgreSQL role password> # mandatory
+```
+
+Then, add the `docker-compose.external_db.yml` file to your `docker compose up` command:
+
+```shell
+docker compose -f docker-compose.yml -f docker-compose.external_db.yml up -d
+```
+
 ### How to pull/build/update CVAT images
 
 - **For a CVAT version lower or equal to 2.1.0**, you need to pull images using docker because