Environment variables ignored in docker-compose scenarios

[ctorgalson]

The documentation explicitly states strongly implies that environment variables can be used to configure containers when using docker-compose. For example:

Make sure to pass in values for MYSQL_ROOT_PASSWORD and MYSQL_PASSWORD variables before you run this setup.

Also some of the docker-compose.yml examples apparently use env vars to set MYSQL* vars for the Nextcloud container:

  app:
    image: nextcloud:apache
    restart: always
    ports:
      - 8080:80
    volumes:
      - nextcloud:/var/www/html
    environment:
      - MYSQL_HOST=db
    env_file:
      - db.env
    depends_on:
      - db

But this doesn't work. AFAICT, the documentation is assuming that since passing env vars works with a Dockerfile, it will also work with docker-compose, but docker-compose doesn't run the docker-entrypoint.sh file that gets run by a Dockerfile.

The question for me is whether this is purely a documentation problem, or if this is a missing feature.

To reproduce

Reproduction script

Script

#!/usr/bin/env bash                                                                                                                                                                                                                                                                
#
# NOTE: you'll need to manually remove the containers, images, and volumes created by this script to re-run it.

mkdir nc-docker-compose-test

cd nc-docker-compose-test

echo ""
echo "Download sample docker compose file"
echo ""

curl -OL https://gist.githubusercontent.com/ctorgalson/0a05ef42acb931529c58c50b37b62dc6/raw/99e310b7ca1ab1ed1650ecfb1ed14e9365b1d3fd/docker-compose.yml

echo ""
echo "Build and start Nextcloud and Mariadb containers."
echo ""

docker-compose up --force-recreate --renew-anon-volumes --build --detach

echo ""
echo "NEXTCLOUD_ADMIN_* env vars."
echo ""

docker exec nc-docker-compose-test_db_1 env | grep NEXTCLOUD_ADMIN

echo ""
echo "MYSQL_* env vars."
echo ""

docker exec nc-docker-compose-test_db_1 env | grep MYSQL

echo ""
echo "Install Nextcloud--we should NOT need to pass login or mysql vars."
echo ""

echo "------------------------------------------------------------"

docker exec --user www-data nc-docker-compose-test_app_1 \
  php occ maintenance:install

echo "------------------------------------------------------------"

echo ""
echo "Install Nextcloud--this time, set an admin password."
echo ""

echo "------------------------------------------------------------"

docker exec --user www-data nc-docker-compose-test_app_1 \
  php occ maintenance:install --admin-pass="nc_admin_password_non_env"

echo "------------------------------------------------------------"

echo ""
echo "Check for database variables in Nextcloud config/config.php."
echo ""

echo "------------------------------------------------------------"

docker exec --user www-data nc-docker-compose-test_app_1 \
  cat config/config.php

echo "------------------------------------------------------------"

Output

The above script uses a very slightly modified version of the 'Base version - apache' sample from the README to:

• Build and start nextcloud and db containers with the relevant env vars,
• Greps the relevant env vars from the two containers to show they're present,
• Tries running php occ maintenance:install with no admin password (this fails),
• Tries running php occ maintenance:install with an admin password (this succeeds),
• Dumps the contents of config/config.php.

The output demonstrates that:

1. Env vars are correctly set in nextcloud container,
2. Env vars are correctly set in db container,
3. NEXTCLOUD_ADMIN_* vars are not used by the installer,
4. MYSQL_* vars are not used by the installer.

Download sample docker compose file

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   634  100   634    0     0  21862      0 --:--:-- --:--:-- --:--:-- 21862

Build and start Nextcloud and Mariadb containers.

Recreating nc-docker-compose-test_db_1 ... done
Recreating nc-docker-compose-test_app_1 ... done

NEXTCLOUD_ADMIN_* env vars.


MYSQL_* env vars.

MYSQL_ROOT_PASSWORD=nextcloud_mysql_root_pass
MYSQL_PASSWORD=nextcloud_mysql_pass
MYSQL_DATABASE=nextcloud
MYSQL_USER=nextcloud

Install Nextcloud--we should NOT need to pass login or mysql vars.

------------------------------------------------------------
Set an admin password.
------------------------------------------------------------

Install Nextcloud--this time, set an admin password.

------------------------------------------------------------
Nextcloud was successfully installed
------------------------------------------------------------

Check for database variables in Nextcloud config/config.php.

------------------------------------------------------------
<?php
$CONFIG = array (
  'htaccess.RewriteBase' => '/',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'apps_paths' => 
  array (
    0 => 
    array (
      'path' => '/var/www/html/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 => 
    array (
      'path' => '/var/www/html/custom_apps',
      'url' => '/custom_apps',
      'writable' => true,
    ),
  ),
  'passwordsalt' => 'oUH/PIrsL7xTajHk/9Xm8Pyr0Bt/Ej',
  'secret' => '1pF7ClUXmaIGOmODkJYOGTlEqNZOZAlZUJ6Jmsa1P22rShb3',
  'trusted_domains' => 
  array (
    0 => 'localhost',
  ),
  'datadirectory' => '/var/www/html/data',
  'dbtype' => 'sqlite3',
  'version' => '19.0.1.1',
  'overwrite.cli.url' => 'http://localhost',
  'installed' => true,
);
------------------------------------------------------------

[cacack]

I believe I've found my way here from the reverse proxy problems, and the TRUSTED_PROXIES environment variables not being honored. Dumping my config I can see those values are missing...

[ptoulouse]

Running it with docker-compose still runs the entrypoint.sh file.

I just had the same issue and the problem is that the way the entrypoint is written is an "all or nothing at all" approach.

To set the DB, you need to specify the 4 variables (MYSQL_HOST, MYSQL_DATABASE, MYSQL_USER, MYSQL_PASSWORD). But even if you set these, you cannot set the DB or Trusted Domains if you don't define NEXTCLOUD_ADMIN_USER and NEXTCLOUD_ADMIN_PASSWORD. There is a big if clause that will skip treating the SQL and trusted domains env vars if the admin vars are not set.

The documentation does not mention this. In fact, the documentation is misleading as it says that the env vars are used to prepopulate values. I was expecting to get to the start page and have the DB variables populated,even if I had not specified an admin user.

[iFreilicht]

@ptoulouse I cannot reproduce this. I struggled initially because I forgot to add the MYSQL_HOST variable, but after that, the prepopulation worked perfectly, as implied by the documentation.

[ptoulouse]

What I am saying is that the MYSQL_* and NEXTCLOUD_TRUSTED_DOMAINS are ignored if the NEXTCLOUD_ADMIN_* variables are not set. See entrypoint.sh at line 121 and down. Also the 4 MYSQL_* vars need to be set to initialize the DB.

The Readme implies that the MYSQL_* vars will be populated even without NEXTCLOUD_ADMIN_* being set.

[iFreilicht]

@ptoulouse I understood. What I'm saying is that despite what the entrypoint.sh looks like, I was able to install nextcloud with mariadb following the README using docker-compose:

version: '2'
services:
  nextcloud:
    image: nextcloud
    container_name: nextcloud
    restart: always
    environment:
      - MYSQL_HOST=nextcloud-database
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=nextcloud_database_user
      - MYSQL_PASSWORD=nextcloud_database_password
      - VIRTUAL_HOST=nextcloud.example.com
      - LETSENCRYPT_HOST=nextcloud.example.com
      - LETSENCRYPT_EMAIL=admin@example.com
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro
      - /mnt/vol1/nextcloud:/var/www/html
    ports:
      - "8080:80"
    links:
      - nextcloud-database

  nextcloud-database:
    image: mariadb:10.3.22 # See https://github.com/nextcloud/docker/issues/1038
    container_name: nextcloud-database
    restart: always
    command: --transaction-isolation=READ-COMMITTED --binlog-format=ROW
    environment:
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=nextcloud_database_user
      - MYSQL_PASSWORD=nextcloud_database_password
      - MYSQL_ROOT_PASSWORD=nextcloud_database_root_password
    volumes:
      - /mnt/vol1/database:/var/lib/mysql

I did try this two more times after completely wiping vol1 and it works perfectly. The database connection is initialized, I could enter a username and password in the web-interface, config.php was properly populated and the installation ran through fully:

<?php
$CONFIG = array (
  'htaccess.RewriteBase' => '/',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'apps_paths' => 
  array (
    0 => 
    array (
      'path' => '/var/www/html/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 => 
    array (
      'path' => '/var/www/html/custom_apps',
      'url' => '/custom_apps',
      'writable' => true,
    ),
  ),
  'instanceid' => 'ocaykbzzgr4w',
  'passwordsalt' => 'rt2LI5mPfLZ69i9B0WwFSefzjlOgJI',
  'secret' => 'M0v1wTgTKTOcVcb1HEVJrZAAEDO/5sKWML7NmOobIgckPQJU',
  'trusted_domains' => 
  array (
    0 => 'nextcloud.example.com',
  ),
  'datadirectory' => '/var/www/html/data',
  'dbtype' => 'mysql',
  'version' => '21.0.1.1',
  'overwrite.cli.url' => 'http://nextcloud.example.com',
  'dbname' => 'nextcloud',
  'dbhost' => 'nextcloud-database',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'nextcloud_database_user',
  'dbpassword' => 'nextcloud_database_password',
  'installed' => true,
);

However, this doesn't work when using the _FILE suffixes for some reason. What I did now instead was to use a .env-file and set the variables via substitution.

[joshtrichards]

But this doesn't work. AFAICT, the documentation is assuming that since passing env vars works with a Dockerfile, it will also work with docker-compose, but docker-compose doesn't run the docker-entrypoint.sh file that gets run by a Dockerfile.

This is incorrect:

• https://docs.docker.com/reference/dockerfile/#entrypoint
• https://docs.docker.com/engine/reference/run/#default-entrypoint

The ENTRYPOINT definition in the Dockerfile specifies what a container should run as the entrypoint at start time. The entrypoint is absolutely invoked when you start a container.

Tries running php occ maintenance:install with an admin password (this succeeds),
[...]
NEXTCLOUD_ADMIN_* vars are not used by the installer,

Well, they're not used when you manually run the Nextcloud installer as you did in your test cases. But that's not surprising. You're overriding how this image works. :-) Those variables are injected through the entrypoint when a container starts for the first time. Those particular ones are passed to the installer within the entrypoint.

What I am saying is that the MYSQL_* and NEXTCLOUD_TRUSTED_DOMAINS are ignored if the NEXTCLOUD_ADMIN_* variables are not set. See entrypoint.sh at line 121 and down. Also the 4 MYSQL_* vars need to be set to initialize the DB.

The Readme implies that the MYSQL_* vars will be populated even without NEXTCLOUD_ADMIN_* being set.

Because they do. There are multiple ways of injecting them internally. What you don't see in the entrypoint, is that the MYSQL_* variables still get picked up by the installer via Nextcloud's autoconfig.php support located in the image here. It doesn't matter if that part of the entrypoint is skipped. Those are different code paths.

I'm going to close this for the following reasons:

• The README has been expanded in several places since this original report.
• Some hints have been added within the entrypoint itself to help point out certain scenarios
• Since the Docker image uses Nextcloud's multi-config.php support, it's important to use occ config:list system to see the real runtime configuration rather than just checking config.php (Emphasize using occ config:list system to view running config  #2244)
• It'll be even clearer (to those not familiar with the code) where/how each variable is injected and what dependencies each one has when Inventory and clarify how each auto-config value is handled [WIP] #2224 is completed
• There isn't anything actionable that remains in this issue