Docker compose not working (new install) #144

breakingflower · 2024-02-14T11:15:37Z

Followed the guide

for building the image -> does not complete with multiple errors about a) symphony-cmd not found and after a fix b) could not chown var (because its not a directory). Updated the dockerfile to chown /var instead but still cant get to run
for running directly using docker compose in the docker folder with the github build. Runs, but web interface is returning 404. Cant execute the initial migration because sh: bin/console: not found. There is no bin console available in the image.

Can you please have a look?

The text was updated successfully, but these errors were encountered:

tchapi · 2024-02-14T23:37:11Z

Hi @breakingflower

I see that Composer has updated its super-user policy, I've just fixed it in d8e9f6e
The initial migration works fine when running docker exec -it davis sh -c "APP_ENV=prod bin/console doctrine:migrations:migrate --no-interaction" on my side - try to rebuild the container with the fix above and see how it fares (don't chown /var, we're looking at ./var here)
About the 404, it's probably a uid/gid user difference between the nginx container (101) and the PHP-fpm container (82). You can try to use this branch: [Chore] Allow setting the uid/gid of the FPM image #145 by making sure that you build the image with the docker compose flavour (docker compose build davis) so it takes the fpm_user argument into account

breakingflower · 2024-02-15T13:35:21Z

Hi @tchapi

I tried the following:

rm -rf davis
git clone https://github.com/tchapi/davis
cd davis
git checkout cc/fpm_user
cd docker
docker compose build
# edit some env vars
docker compose up -d

If im using a bind mount, e.g.

    volumes:
      - ./data/www:/var/www/davis

the data/www directory is empty (overwritten)

If I run the command with a volume configuration and make nginx wait for the davis copy to complete

nginx: 
  depends_on:
      davis:
         condition: service_started
...
davis:
    volumes:
      - www:/var/www/davis
volumes:
  www:
    driver_opts:
      type: none
      device: ${PWD}/data/www
      o: bind

It works.

I can now run the migrations docker exec -it davis sh -c "APP_ENV=prod bin/console doctrine:migrations:migrate --no-interaction".

Curious to hear if you know why the bind mount initially is not working, where the named volume mount is? I always thought a bind mount copies the content (in this case at /var/www/davis) if the directory does not exist or is empty.

tchapi · 2024-02-15T21:46:48Z

the data/www directory is empty (overwritten)

Yes it's expected, bind mounting obscures the container directory

If I run the command with a volume configuration and make nginx wait for the davis copy to complete

Davis is not copying anything, it's Docker that manages the volume internally, the content being pre-populated by the container. A caveat to that is that if you update the Davis container, it won't update the files (because they are now managed in the volume, not in the container), so you have to remove the container fully before updating

breakingflower · 2024-02-16T07:53:17Z

What is the use of a mount / volume if the files are overwritten by the container? Is there a way to only expose the relevant files (contacts, calendars, config files that a user can adjust prior to starting the container) to the mount?
e.g. mount points:

- ./my_calendars:/var/www/davis/calendars
- ./my_contacts:/var/www/davis/contacts
- ./config:/var/www/davis/config

romner-set · 2024-02-22T15:53:06Z

Same issue here, replacing davis_www with a bind mount results in an empty /var/www/davis inside of the container on first start. I'm using the prebuilt ghcr.io image and trying to use an NFS mount for persistent storage.

Pre-populating directories that are later used for persistent data with binaries and stuff is not a very good idea IMO.

tchapi · 2024-02-25T16:14:37Z

Hi

The Davis container has only one service in it: php-fpm. If you want a reverse-proxy (say, Nginx) to serve the app, it needs access to the files too (which are in/var/www/davis), hence why it is shared as in the docker-compose.yml file example.

Is there a way to only expose the relevant files (contacts, calendars, config files that a user can adjust prior to starting the container)

I'm not sure to see which files you are talking about?

Pre-populating directories that are later used for persistent data with binaries and stuff is not a very good idea IMO.

There is no "persistent data" in there (it's only the PHP source code, logs, tmp files), so I don't get your point 🤔

romner-set · 2024-02-25T17:25:41Z

The Davis container has only one service in it: php-fpm. If you want a reverse-proxy (say, Nginx) to serve the app, it needs access to the files too (which are in/var/www/davis), hence why it is shared as in the docker-compose.yml file example.

Pre-populating directories that are later used for persistent data with binaries and stuff is not a very good idea IMO.

There is no "persistent data" in there (it's only the PHP source code, logs, tmp files), so I don't get your point 🤔

This is the first time I've seen it done this way, and it's pretty confusing – especially since I've never even seen volumes/mounts being used for any non-persistent data, so I assumed the contacts, calendars and config dirs in the volume are used for data alongside the DB.

Either way, dockerfiles for PHP apps usually include a basic web server so this kind of stuff isn't necessary. This both avoids the issue of updates not actually doing anything since all of the code is in a volume, and makes the whole setup significantly simpler (see koel for an example). Is there any specific reason Davis does it differently?

tchapi · 2024-02-25T18:20:09Z

The general rule is that Docker containers should only run a single process - to decouple concerns. It's not a hard rule, I know, but I don't consider Davis and a web proxy to be coupled enough to warrant being a single concern.

I assumed the contacts, calendars and config dirs in the volume are used for data alongside the DB.

No, the data is only stored in the DB (except for webdav, but you should mount that separately if needed of course)

Either way, dockerfiles for PHP apps usually include a basic web server so this kind of stuff isn't necessary

YMMV, I've seen both, and when I started Davis back in '19, It was mainly the multiple container option that was the norm.

Having everything directly in the image might simplify the setup, but it hides the complexity and increases the maintenance burden (I need a proxy + php-fpm + a process manager to run them + a different entrypoint etc etc). Moreover, I don't want to impose a specific proxy to users that might be opinionated (Nginx or Apache? Caddy? Traefik?) — it's up to you to decide which one you prefer and want to have for your use case.

In a nutshell: My goal is to maintain and improve Davis as much as I can, but I don't want to (can't) maintain a convoluted Dockerfile on top of that, for lack of time and expertise

All that being said:

the Dockerfile is pretty straightforward, it'd be rather easy to roll your own with Nginx in it if you need
I'm not strongly against providing this option in the future, if time allows, or if someone from the community wants to maintain it

tchapi · 2024-02-26T21:01:19Z

FYI I've merged #145 and the edge image was correctly built so I consider this issue solved for now.

I've started to create a roadmap project where I will add feature requests (like having a combined Docker image) to keep track of it

tchapi · 2024-03-04T20:36:59Z

→ first attempt at a combined Docker image is here: #150 if you want to have a look

tchapi · 2024-03-17T10:41:14Z

@breakingflower @romner-set would you be so kind as to test https://github.com/tchapi/davis/pkgs/container/davis-standalone (standalone image with a reserve-proxy) and tell me if it fits your needs / if you have feedback?

Thanks a lot!

breakingflower · 2024-03-24T16:33:09Z

Hi @tchapi

Thanks for the follow-up!
I've tried with this new setup as follows:

version: "3.7"

services:

  mysql:
    image: mariadb:10.6.10
    env_file: .env
    environment:
      - MYSQL_ROOT_PASSWORD=${DB_ROOT_PASSWORD}
      - MYSQL_DATABASE=davis
      - MYSQL_USER=${DB_USER}
      - MYSQL_PASSWORD=${DB_PASSWORD}
    volumes:
      - ./db:/var/lib/mysql

  davis:
    # build:
    #   context: ../
    #   dockerfile: ./docker/Dockerfile-standalone
    # image: davis:latest
    # If you want to use a prebuilt image from Github
    image: ghcr.io/tchapi/davis-standalone:edge
    env_file: .env
    # container_name: davis-standalone
    environment:
      - APP_ENV=prod
      - DATABASE_DRIVER=mysql
      - DATABASE_URL=mysql://${DB_USER}:${DB_PASSWORD}@mysql:3306/davis?serverVersion=mariadb-10.6.10&charset=utf8mb4
      - MAILER_DSN=smtp://${MAIL_USERNAME}:${MAIL_PASSWORD}@${MAIL_HOST}:${MAIL_PORT}
      - ADMIN_LOGIN=${ADMIN_LOGIN}
      - ADMIN_PASSWORD=${ADMIN_PASSWORD}
      - AUTH_REALM=${AUTH_REALM}
      - AUTH_METHOD=${AUTH_METHOD}
      - CALDAV_ENABLED=${CALDAV_ENABLED}
      - CARDDAV_ENABLED=${CARDDAV_ENABLED}
      - WEBDAV_ENABLED=${WEBDAV_ENABLED}
      - WEBDAV_TMP_DIR=${WEBDAV_TMP_DIR}
      - WEBDAV_PUBLIC_DIR=${WEBDAV_PUBLIC_DIR}
      - INVITE_FROM_ADDRESS=${INVITE_FROM_ADDRESS}
      - APP_TIMEZONE=Europe/Brussels
    depends_on:
      - mysql
    ports:
      - 9000:9000

After, I applied the migrations: docker exec -it davis-davis-1 bin/console doctrine:migrations:migrate. Migrations return no error.
I am not getting any content to load on the 9000 endpoint, it's just a white screen.

Any idea?

tchapi · 2024-03-24T20:45:46Z

Hmm, just tested locally with the exact same compose file and it works (page loads correctly):

Are you sure you ran the migrations on the correct container?

breakingflower · 2024-03-25T08:01:45Z

Yes, this is the output:

docker exec -it davis-davis-1 bin/console doctrine:migrations:migrate

 WARNING! You are about to execute a migration in database "davis" that could result in schema changes and data loss. Are you sure you wish to continue? (yes/no) [yes]:
 > yes

[notice] Migrating up to DoctrineMigrations\Version20231229203515
[notice] Migration DoctrineMigrations\Version20221106220412 skipped during Execution. Reason: "This migration is specific to 'postgresql'. Skipping it is fine."
[notice] Migration DoctrineMigrations\Version20221211154443 skipped during Execution. Reason: "This migration is specific to 'sqlite'. Skipping it is fine."
[notice] Migration DoctrineMigrations\Version20230209142217 skipped during Execution. Reason: "This migration is specific to 'postgresql'. Skipping it is fine."
[notice] Migration DoctrineMigrations\Version20231001214112 skipped during Execution. Reason: "This migration is specific to 'postgresql'. Skipping it is fine."
[notice] Migration DoctrineMigrations\Version20231001214113 skipped during Execution. Reason: "This migration is specific to 'sqlite'. Skipping it is fine."
[notice] finished in 3713ms, used 14M memory, 14 migrations executed, 46 sql queries

I've tried with a dedicated volume for the db but it yields the same result.
The davis container remains in the "Health: Starting" state. There are no logs.

tchapi · 2024-03-25T21:34:25Z

That's weird. You have a blank page, but what is the status? Is it a 500, a 400?

Can you check the supervisor logs? By connecting to the container, then supervisorctl and tail caddy or tail php-fpm to see if there is anything suspicious there

You should be also able to run ./console in /var/www/davis/bin to see if it works correctly

breakingflower · 2024-03-26T07:06:08Z

Ok,

$ docker compose exec davis sh
WARN[0000] The "MAIL_USERNAME" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MAIL_PASSWORD" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MAIL_HOST" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MAIL_PORT" variable is not set. Defaulting to a blank string. 
/var/www/davis # ls 
bin            composer.json  composer.lock  config         migrations     public         src            symfony.lock   templates      tests          translations   var            vendor
/var/www/davis # supervisorctl
caddy                            RUNNING   pid 7, uptime 0:00:27
php-fpm                          RUNNING   pid 8, uptime 0:00:27
supervisor> tail caddy
:"shutdown complete","signal":"SIGTERM","exit_code":0}
{"level":"info","ts":1711436429.130123,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":""}
{"level":"warn","ts":1711436429.1319635,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
{"level":"info","ts":1711436429.133241,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"warn","ts":1711436429.1334443,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
{"level":"info","ts":1711436429.1336386,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc00062aa80"}
{"level":"info","ts":1711436429.1342876,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1711436429.1345258,"msg":"autosaved config (load with --resume flag)","file":"/root/.config/caddy/autosave.json"}
{"level":"info","ts":1711436429.1345387,"msg":"serving initial configuration"}
{"level":"warn","ts":1711436429.147367,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/root/.local/share/caddy","instance":"36eb1b7d-651c-4197-ab04-5ab784f13f4f","try_again":1711522829.1473556,"try_again_in":86399.999998981}
{"level":"info","ts":1711436429.1475737,"logger":"tls","msg":"finished cleaning storage units"}

supervisor> tail php-fpm 
ndex.php" 500
- -  26/Mar/2024:06:35:33 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:36:33 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:37:33 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:38:33 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:39:33 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:40:33 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:41:33 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:42:34 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:43:34 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:44:34 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:45:34 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:46:34 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:47:34 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:48:34 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:49:34 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:50:34 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:51:35 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:52:35 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:53:35 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:54:35 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:55:35 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:56:35 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:57:35 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:58:35 +0000 "GET /index.php" 500
- -  26/Mar/2024:06:59:35 +0000 "GET /index.php" 500
[26-Mar-2024 07:00:24] NOTICE: Terminating ...
[26-Mar-2024 07:00:24] NOTICE: exiting, bye-bye!
[26-Mar-2024 07:00:29] NOTICE: fpm is running, pid 8
[26-Mar-2024 07:00:29] NOTICE: ready to handle connections
- -  26/Mar/2024:07:01:27 +0000 "GET /index.php" 500

When port-forwarding from ip:9000 to localhost:9000 I get the following:

Oops! An Error Occurred
The server returned a "500 Internal Server Error".

Something is broken. Please let us know what you were doing when this error occurred. We will fix it as soon as possible. Sorry for any inconvenience caused.

Is there a way to set logging to debug & stdout?

NyaomiDEV · 2024-03-27T20:11:25Z

Trying the standalone docker container as well. White page here too; using the image on GHCR.

davis/docker/configurations/Caddyfile

Line 5 in eaac75f

http://localhost:9000 {

should be

:9000 {

tchapi · 2024-03-27T20:17:57Z

I really can't reproduce on my side. Can you have a look in /var/www/davis/var/log/? There should be a prod.log there with the reason behind the 500

NyaomiDEV · 2024-03-27T20:20:21Z

I really can't reproduce on my side. Can you have a look in /var/www/davis/var/log/? There should be a prod.log there with the reason behind the 500

There's nothing because you're binding the container to localhost, that is itself. Running curl localhost:9000 inside the container yields a result, but Caddy responds 200 and ignores everything from the outside. Putting :9000 makes Caddy listen to all addresses, which means that we can either port forward or put the container behind a reverse proxy at our discretion.

With the modification in place, the container is working.

tchapi · 2024-03-27T20:47:09Z

There's nothing because you're binding the container to localhost

The 500 page above (message from @breakingflower) is served by PHP through Caddy, so there will be something in the production log. The two issues are separate.

With the modification in place, the container is working.

Thanks, it's indeed better, done in 7ee1cae — unfortunately this will still not solve the 500

NyaomiDEV · 2024-03-27T20:54:46Z

The 500 page above (message from @breakingflower) is served by PHP through Caddy, so there will be something in the production log. The two issues are separate.

Oh, that message being under mine made me thought you were replying to me. Sorry!

tchapi · 2024-03-27T22:27:41Z

The package is up to date, so this should work for you now @NyaomiDEV : https://github.com/tchapi/davis/pkgs/container/davis-standalone

breakingflower · 2024-03-28T08:04:14Z

Hi @tchapi ,

With this image I get the forward, but there is still a 500. There are no prod / error log files. The only ones in the container are:

/var/www/davis # find / -type f -name "*.log"
/var/log/caddy/access.log
/var/log/php-fpm/access.log

breakingflower · 2024-03-28T08:05:47Z

Correction, the dashboard is loading fine at <ip>:9000/dashboard.

tchapi · 2024-03-28T08:34:12Z

Correction, the dashboard is loading fine at :9000/dashboard.

Ah, so it's just the root url <ip>:9000/ that is having a problem?

EDIT: Thinking about it if you don't have any log, could this be some kind of write permissions in the log folder?

breakingflower · 2024-03-28T08:43:34Z

There is no redirect from 9000/ to 9000/dashboard so i get an error on / ?

There is no log, the /var/www/davis/var/ directory is owned by root on container spawn. Perhaps a chown -R www-data:www-data /var/www/davis is needed? I'm not sure which user needs to have permissions to write here.

tchapi · 2024-03-28T16:17:20Z

There is no redirect from 9000/ to 9000/dashboard so i get an error on / ?

No redirect, but a status web page on /

Perhaps a chown -R www-data:www-data /var/www/davis is needed?

Hmm, interesting. Could you try to chown it and see if it's better on your side?

romner-set · 2024-04-09T12:19:23Z

@tchapi Apologies for not responding earlier, but I just tested the combined image using the provided compose file and it seems to work well, no issues so far.

tchapi · 2024-04-09T15:19:37Z

Great @romner-set ! Thank you for your feedback.

tchapi added the question A question about Davis and how it works label Feb 14, 2024

tchapi closed this as completed Feb 26, 2024

kang-makes mentioned this issue Mar 3, 2024

Unable to login using a Docker image #149

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docker compose not working (new install) #144

Docker compose not working (new install) #144

breakingflower commented Feb 14, 2024

tchapi commented Feb 14, 2024

breakingflower commented Feb 15, 2024 •

edited

Loading

tchapi commented Feb 15, 2024

breakingflower commented Feb 16, 2024

romner-set commented Feb 22, 2024 •

edited

Loading

tchapi commented Feb 25, 2024

romner-set commented Feb 25, 2024 •

edited

Loading

tchapi commented Feb 25, 2024

tchapi commented Feb 26, 2024

tchapi commented Mar 4, 2024

tchapi commented Mar 17, 2024

breakingflower commented Mar 24, 2024

tchapi commented Mar 24, 2024

breakingflower commented Mar 25, 2024

tchapi commented Mar 25, 2024 •

edited

Loading

breakingflower commented Mar 26, 2024

NyaomiDEV commented Mar 27, 2024 •

edited

Loading

tchapi commented Mar 27, 2024

NyaomiDEV commented Mar 27, 2024

tchapi commented Mar 27, 2024 •

edited

Loading

NyaomiDEV commented Mar 27, 2024

tchapi commented Mar 27, 2024

breakingflower commented Mar 28, 2024

breakingflower commented Mar 28, 2024

tchapi commented Mar 28, 2024 •

edited

Loading

breakingflower commented Mar 28, 2024

tchapi commented Mar 28, 2024

romner-set commented Apr 9, 2024

tchapi commented Apr 9, 2024

Docker compose not working (new install) #144

Docker compose not working (new install) #144

Comments

breakingflower commented Feb 14, 2024

tchapi commented Feb 14, 2024

breakingflower commented Feb 15, 2024 • edited Loading

tchapi commented Feb 15, 2024

breakingflower commented Feb 16, 2024

romner-set commented Feb 22, 2024 • edited Loading

tchapi commented Feb 25, 2024

romner-set commented Feb 25, 2024 • edited Loading

tchapi commented Feb 25, 2024

tchapi commented Feb 26, 2024

tchapi commented Mar 4, 2024

tchapi commented Mar 17, 2024

breakingflower commented Mar 24, 2024

tchapi commented Mar 24, 2024

breakingflower commented Mar 25, 2024

tchapi commented Mar 25, 2024 • edited Loading

breakingflower commented Mar 26, 2024

NyaomiDEV commented Mar 27, 2024 • edited Loading

tchapi commented Mar 27, 2024

NyaomiDEV commented Mar 27, 2024

tchapi commented Mar 27, 2024 • edited Loading

NyaomiDEV commented Mar 27, 2024

tchapi commented Mar 27, 2024

breakingflower commented Mar 28, 2024

breakingflower commented Mar 28, 2024

tchapi commented Mar 28, 2024 • edited Loading

breakingflower commented Mar 28, 2024

tchapi commented Mar 28, 2024

romner-set commented Apr 9, 2024

tchapi commented Apr 9, 2024

breakingflower commented Feb 15, 2024 •

edited

Loading

romner-set commented Feb 22, 2024 •

edited

Loading

romner-set commented Feb 25, 2024 •

edited

Loading

tchapi commented Mar 25, 2024 •

edited

Loading

NyaomiDEV commented Mar 27, 2024 •

edited

Loading

tchapi commented Mar 27, 2024 •

edited

Loading

tchapi commented Mar 28, 2024 •

edited

Loading