Cannot make Email catch-all to work in the intranet

[masiorama]

ISSUE TYPE

• Bug Report

Checklist

• .env file is attached
• docker-compose.override.yml is attached (if it exists)
• Custom configs from cfg/ dir are attached (if customized)
• docker-compose logs and log/ output is added
• I've looked through the docs: https://devilbox.readthedocs.io/en/latest/
• I've looked through existing issues: https://github.com/cytopia/devilbox/issues

OS / ENVIRONMENT

1. Host operating system and version: Win 10
2. (Windows) Native Docker or Docker Toolbox: Native Docker
3. Docker version: 18.06.1-ce-win73 (19507)
4. Docker Compose version: 1.22.0
5. What git commit hash are you on?:

SUMMARY

I cannot catch any email, not even a test one from the very same Email page.
Here the debug logs from the intranet.

STEPS TO REPRODUCE

No idea

EXPECTED BEHAVIOUR

Email catch-all working again.

ACTUAL BEHAVIOUR

Complete empty table list, and debug errors list in the footer section.

OTHER INFORMATION

Start command

$ docker-compose up mysql

.env

###
###  ---------------------------------------------------
###  D E V I L B O X   R U N - T I M E   S E T T I N G S
###  ---------------------------------------------------
###
###  All the following settings are applied during
###  $ docker-compose up
###
###  No need to rebuild any dockers!
###
###  For custom variables, scroll to the bottom
###


###
### Show all executed commands in each
### docker image during docker-compose up?
###
### 1: Yes
### 0: No
DEBUG_COMPOSE_ENTRYPOINT=1


###
### Log to file or Docker logs.
###
### Logging to file means log files are available under log/
### on your host operating system.
### Logging to Docker logs means log files are streamed to
### stdout and stderr.
###
### 1: Log to Docker logs
### 0: Log to file
###
DOCKER_LOGS=0


###
### Relative or absolute path to the devilbox repository.
### (Used as a prefix for all mount paths)
### There is no need to change this.
###
### The only exception is for OSX users wanting to use NFS
### mounts instead of Filesystem mounts due to degraded performance
### on OSX.
###
### Note: When changing this variable you must re-create the container.
###       Simply remove it, it will be auto-created during the next start:
###
### $ docker-compose rm -f
###
DEVILBOX_PATH=.


###
### At what IP address should the docker services listen
### on the Host computer?
###
### The specified default should be fine for Linux and OSX (127.0.0.1:).
### If you are on windows, you will probably have to change
### it to the IP address of the docker machine.
###
### a.) Leave blank, to listen on all interfaces (no trailing color ':')
###    LOCAL_LISTEN_ADDR=
### b.) If an IP is specified, note the trailing colon ':'
###    LOCAL_LISTEN_ADDR=127.0.0.1:
###
LOCAL_LISTEN_ADDR=


###
### This is the domain suffix your projects will be made available
### with mass-virtual-hosting.
### It is also required for the internal DNS server to be setup.
###
### Note: Only ALPHA ([a-zA-Z]+) characters are supported.
###
### Example:
###   TLD_SUFFIX=loc
### Makes your project available under xxxx.loc
###
### Example:
###   TLD_SUFFIX=local
### Makes your project available under xxxx.local
###
TLD_SUFFIX=loc


###
### Optional DNS configuration
### Allows you to add extra DNS records (above the wildcard entry)
### Useful if your host computer run other Docker services that you want to connect to or reach
### from within the Devilbox network by a custom hostname.
###
### Format:
### -------
### Resolve any custom defined hostname to an IP address (useable inside container and host os)
###     EXTRA_HOSTS=<hostname>=<ip>[,<hostname>=<ip>]
###
### Resolve any custom defined hostname to whatever IP address a CNAME resolves to
### (Useable inside the container and host OS).
### Note: CNAME must be resolvable by Google DNS
###     EXTRA_HOSTS=<hostname>=<CNAME>[,<hostname>=<CNAME>]
###
### Examples:
### ---------
### EXTRA_HOSTS=hostname.loc=1.2.3.4
### EXTRA_HOSTS=host.loc=1.2.3.4,host.example.org=3.4.5.6
EXTRA_HOSTS=


###
### Set your user id and group id
###
### This should be changed to the value of your local
### users uid and gid
###
### Type `id` on the terminal to find out your values
###
NEW_UID=1000
NEW_GID=1000


###
### Timezone for all dockers and service config files
###
TIMEZONE=Europe/Rome




################################################################################
###
### INTRANET SETTINGS
###
################################################################################

###
### TLD_SUFFIX domains are checked if they are set in the
### host computer /etc/hosts or available via attached DNS server.
### Timeout is done on vhosts.php (intranet) via ajax calls.
### In order to keep performance, set this to a low value.
### DNS checks might not succeed in time on slow machines.
### If DNS is valid, but timeout is expired, set this to a higher value.
###
### DNS_CHECK_TIMEOUT value is how many seconds to time out
### Default is to timeout after 1 second (DNS_CHECK_TIMEOUT=1)
###
DNS_CHECK_TIMEOUT=1


###
### Devilbox UI SSL Certificate generation
###
### When using SSL each certificate requires names for which it is responsible:
### Common Name as well as alternative names.
###
### Specify comma separated hostnames below by which you want to access the Devilbox.
### Those hostnames will be included in the SSL certificate for the Devilbox intranet.
### This has nothing to do for SSL certificates for projects, it is just for the intranet
### itself.
###
DEVILBOX_UI_SSL_CN=localhost,*.localhost,devilbox,*.devilbox


###
### Devilbox UI Password protection enable/disable (1/0)
###
### Set DEVILBOX_UI_PROTECT to 1 in order to password protect the
### intranet.
###
### Example:
###   DEVILBOX_UI_PROTECT=1
###   DEVILBOX_UI_PROTECT=0
###
DEVILBOX_UI_PROTECT=0


###
### Devilbox UI Password
###
### When DEVILBOX_UI_PROTECT=1, use the following password
### to log in. The password can always be changed.
### When changing the password, make sure to restart your
### PHP container.
###
### Example:
###   DEVILBOX_UI_PASSWORD=my-very-secure-password
###   DEVILBOX_UI_PASSWORD=Some pass with spaces
###
### The default username is 'devilbox'
###
DEVILBOX_UI_PASSWORD=password


###
### Enable the Devilbox Intranet?
###
### Example:
###   DEVILBOX_UI_ENABLE=1
###   DEVILBOX_UI_ENABLE=0
###
DEVILBOX_UI_ENABLE=1



################################################################################
###
### 1. Choose Images (Version)
###
################################################################################

###
### You can choose any combination of httpd, mysql, postgresql or php.
### Each of them are fully compatible between one another.
###


###
### 1.1 Choose PHP Server Image
###
### Note: PHP 5.2 is not officially supported. Intranet won't work (due to lack of namespace support).
###       PHP 5.2 only works with Apache 2.4, Nginx stable and Nginx mainline.
###       Use at your own risk.
###
#PHP_SERVER=5.2
#PHP_SERVER=5.3
#PHP_SERVER=5.4
#PHP_SERVER=5.5
#PHP_SERVER=5.6
#PHP_SERVER=7.0
PHP_SERVER=7.1
#PHP_SERVER=7.2
#PHP_SERVER=7.3


###
### 1.2 Choose HTTPD Server Image
###
#HTTPD_SERVER=apache-2.2
HTTPD_SERVER=apache-2.4
#HTTPD_SERVER=nginx-stable
#HTTPD_SERVER=nginx-mainline


###
### 1.3 Choose MySQL Server Image
###
#MYSQL_SERVER=mysql-5.5
MYSQL_SERVER=mysql-5.6
#MYSQL_SERVER=mysql-5.7
#MYSQL_SERVER=mysql-8.0
#MYSQL_SERVER=mariadb-5.5
#MYSQL_SERVER=mariadb-10.0
#MYSQL_SERVER=mariadb-10.1
#MYSQL_SERVER=mariadb-10.2
#MYSQL_SERVER=mariadb-10.3
#MYSQL_SERVER=percona-5.5
#MYSQL_SERVER=percona-5.6
#MYSQL_SERVER=percona-5.7


###
### 1.4 Choose PostgreSQL Server Image
###
#PGSQL_SERVER=9.1
#PGSQL_SERVER=9.2
#PGSQL_SERVER=9.3
#PGSQL_SERVER=9.4
#PGSQL_SERVER=9.5
PGSQL_SERVER=9.6
#PGSQL_SERVER=10.0
#PGSQL_SERVER=10.1
#PGSQL_SERVER=10.2
#PGSQL_SERVER=10.3


###
### 1.5 Choose Redis Server Image
###
#REDIS_SERVER=2.8
#REDIS_SERVER=3.0
#REDIS_SERVER=3.2
REDIS_SERVER=4.0
#REDIS_SERVER=5.0-rc


###
### 1.6 Choose Memcached Server Image
###
#MEMCD_SERVER=1.4.21
#MEMCD_SERVER=1.4.22
#MEMCD_SERVER=1.4.23
#MEMCD_SERVER=1.4.24
#MEMCD_SERVER=1.4.25
#MEMCD_SERVER=1.4.26
#MEMCD_SERVER=1.4.27
#MEMCD_SERVER=1.4.28
#MEMCD_SERVER=1.4.29
#MEMCD_SERVER=1.4.30
#MEMCD_SERVER=1.4.31
#MEMCD_SERVER=1.4.32
#MEMCD_SERVER=1.4.33
#MEMCD_SERVER=1.4.34
#MEMCD_SERVER=1.4.35
#MEMCD_SERVER=1.4.36
#MEMCD_SERVER=1.4.37
#MEMCD_SERVER=1.4.38
#MEMCD_SERVER=1.4.39
#MEMCD_SERVER=1.5.0
#MEMCD_SERVER=1.5.1
#MEMCD_SERVER=1.5.2
#MEMCD_SERVER=1.5.3
#MEMCD_SERVER=1.5.4
MEMCD_SERVER=1.5.5
#MEMCD_SERVER=latest


###
### 1.7 Choose Mongo Server Image
###
#MONGO_SERVER=2.8
#MONGO_SERVER=3.0
#MONGO_SERVER=3.2
#MONGO_SERVER=3.4
#MONGO_SERVER=3.5
#MONGO_SERVER=3.6
MONGO_SERVER=3.7



################################################################################
###
### 2. Host Mounts (Your computer)
###
################################################################################

###
### Global mount options
###
### Note: When adding custom mount options, ensure to start with a
###       leading ',' (comma), as those options are prepended to already
###       existing mount options.
###
### Note: If no mount options are specified, leave this variable empty
###       and do not add a leading ',' (comman).
###
### MOUNT_OPTIONS=,cached
### MOUNT_OPTIONS=
###
### Example: Allow to share mounts accross container with SELINUX enabled
###
### MOUNT_OPTIONS=,z
###
MOUNT_OPTIONS=


###
### Local filesystem path to www projects.
###
### Note: When changing this variable you must re-create the container.
###       Simply remove it, it will be auto-created during the next start:
###
### $ docker-compose rm -f
###
HOST_PATH_HTTPD_DATADIR=./../development/web


###
### Local filesystem path to mysql/mariadb datadir.
###
### This can be an existing mysql data directory or empty.
### If it already is a mysql data directory with content,
### it will be mounted into the docker and used.
###
### If this directory is empty, a new mysql database will be
### created.
###
### Note: Inside this path, a subdirectory with the mysql|mariadb
###       version will be created where the actual data resides.
###       This is to protect databases from being altered by
###       newer or older mysql|mariadb server versions.
###
### Note: When changing this variable you must re-create the container.
###       Simply remove it, it will be auto-created during the next start:
###
### $ docker-compose rm -f
###
HOST_PATH_MYSQL_DATADIR=./../development/db/mysql


###
### Local filesystem path to postgresql datadir.
###
### This can be an existing postgresql data directory or empty.
### If it already is a postgresql data directory with content,
### it will be mounted into the docker and used.
###
### If this directory is empty, a new postgresql database will be
### created.
###
### Note: Inside this path, a subdirectory with the postgresql
###       version will be created where the actual data resides.
###       This is to protect databases from being altered by
###       newer or older postgres server versions.
###
### Note: When changing this variable you must re-create the container.
###       Simply remove it, it will be auto-created during the next start:
###
### $ docker-compose rm -f
###
HOST_PATH_PGSQL_DATADIR=./../development/db/pgsql


###
### Local filesystem path to mongodb datadir.
###
### This can be an existing mongodb data directory or empty.
### If it already is a mongodb data directory with content,
### it will be mounted into the docker and used.
###
### If this directory is empty, a new mongodb database will be
### created.
###
### Note: When changing this variable you must re-create the container.
###       Simply remove it, it will be auto-created during the next start:
###
### $ docker-compose rm -f
###
HOST_PATH_MONGO_DATADIR=./../development/db/mongo



################################################################################
###
### 3. PHP Docker Settings
###
################################################################################

###
### Enable certain PHP modules which are not enabled by default
###
### Currently the only module that can be enabled is 'ioncube'
###
### PHP_MODULES_ENABLE=ioncube
###
PHP_MODULES_ENABLE=


###
### Disable any PHP modules that you don't require
###
### Specify a comma separated list without spaces of modules to disable
###
### PHP_MODULES_DISABLE=xdebug,imagick,swoole
###
PHP_MODULES_DISABLE=


###
### Configure everything else about PHP in
### * cfg/php-ini-X.X/*.ini
### * cfg/php-fpm-X.X/*.conf



################################################################################
###
### 4. HTTPD Docker Settings
###
################################################################################

###
### Expose HTTPD Port to Host
###
HOST_PORT_HTTPD=80
HOST_PORT_HTTPD_SSL=443


###
### Document Root Subdirectory
###
### In your project directory, which subfolder should
### serve your files?
###
### When changing this value, restart the devilbox.
###
HTTPD_DOCROOT_DIR=htdocs


###
### Per vHost Config Subdirectory
###
### In your project directory, which subfolder should
### hold apache, nginx templates for a customized vhost?
###
### When changing this value, restart the devilbox.
###
HTTPD_TEMPLATE_DIR=.devilbox


###
### Webserver timeout (in seconds) to upstream PHP-FPM server
###
### This value should be greater than PHP's max_execution_time,
### otherwise the php script could still run and the webserver will
### simply drop the connection before getting an answer by PHP.
###
HTTPD_TIMEOUT_TO_PHP_FPM=180



################################################################################
###
### 5. MySQL Docker Settings
###
################################################################################

###
### MySQL root user password
###
### If $HOST_PATH_MYSQL_DATADIR already contains an existing
### mysql datadir, enter the password for the existing mysql database
###
### If $HOST_PATH_MYSQL_DATADIR is empty, choose a new password that
### will be applied
###
MYSQL_ROOT_PASSWORD=


###
### Custom MySQL Runtime Settings
###
MYSQL_GENERAL_LOG=1


###
### Expose MySQL Port to Host
###
HOST_PORT_MYSQL=3306



################################################################################
###
### 6. PostgreSQL Docker Settings
###
################################################################################

###
### PostgreSQL 'root' user name (usually postgres)
###
PGSQL_ROOT_USER=postgres


###
### PostgreSQL 'root' user password
###
PGSQL_ROOT_PASSWORD=


###
### Expose PostgreSQL Port to Host
###
HOST_PORT_PGSQL=5432



################################################################################
###
### 7. Redis Docker Settings
###
################################################################################

###
### Expose Redis Port to Host
###
HOST_PORT_REDIS=6379


###
### Custom startup arguments
###
### Apply custom startup arguments to redis
###
### Example: Password protection
###   Add password protection to the Redis server by specifying it should
###   require a password.
###   Note: Do not add quotes or spaces to the password
###
###   REDIS_ARGS=--requirepass my-redis-root-password
###
### Example: Verbosity
###
###   REDIS_ARGS=--loglevel verbose
###
REDIS_ARGS=
#REDIS_ARGS=--loglevel verbose --requirepass my-redis-root-password




################################################################################
###
### 8. Memcached Docker Settings
###
################################################################################

###
### Expose Memcached Port to Host
###
HOST_PORT_MEMCD=11211



################################################################################
###
### 9. MongoDB Docker Settings
###
################################################################################

###
### Expose MongoDB Port to Host
###
HOST_PORT_MONGO=27017



################################################################################
###
### 10. Bind Docker Settings
###
################################################################################

###
### Expose Bind Port to Host
###
HOST_PORT_BIND=1053

###
### Add comma separated DNS server from which you want to receive DNS
### You can also add DNS servers from your LAN (if any are available)
###
BIND_DNS_RESOLVER=8.8.8.8,8.8.4.4

###
### Validate DNSSEC
###
### Values:
###  no:    DNSSEC validation is disabled
###  yes:   DNSSEC validation is enabled, but a trust anchor must be manually configured.
###  auto:  DNSSEC validation is enabled, and a default trust anchor for root zone is used.
###
BIND_DNSSEC_VALIDATE=no


###
### Bind timing options (time in seconds)
###
### Leave empty for defaults.
### Only change when you know what you are doing.
###
BIND_TTL_TIME=
BIND_REFRESH_TIME=
BIND_RETRY_TIME=
BIND_EXPIRY_TIME=
BIND_MAX_CACHE_TIME=


###
### Show DNS Queries in Docker logs output?
###
### 1: Yes
### 0: No
BIND_LOG_DNS_QUERIES=0



################################################################################
###
### 11. Custom variables
###
################################################################################

###
### Any variable defined in this file will be available
### as environment variables to your PHP/HHV Docker container.
###
### This might be useful to set application environment and retrieve
### them via: <?php getenv('MY_APPLICATION_ENV'); ?>
###


###
### Example:
### <?php echo getenv('Foo'); ?> would produce: 'some value'
###
#Foo=some value

cfg\php-ini-7.1\custom.ini

[PHP]
; Memory
; Note: "memory_limit" should be larger than "post_max_size"
memory_limit            = 512M

; Timeouts
max_execution_time = 120
max_input_time = 120

; Uploads
; Note: "post_max_size" should be greater than "upload_max_filesize"
post_max_size = 72M
upload_max_filesize = 64M
max_file_uploads = 20

; Vars
variables_order = EGPCS
max_input_vars = 8000
max_input_nesting_level = 64

; Error reporting
; Note: error_log is dynamic and handled during start to set appropriate setting
error_reporting = E_ALL | E_NOTICE | E_STRICT | E_DEPRECATED
xmlrpc_errors = Off
report_memleaks = On
display_errors = On
display_startup_errors = On
track_errors = On
log_errors = On
html_errors = On

[cytopia]

The only thing I could imagine for is that the mail/ directory does not exist or has the wrong permission. Can you check in the Devilbox git directory if the mail dir is there and has permissions as shown in the screenshot (writeable by the user)

If it still doesnt work, try to give it write permissions for everyone (also all files inside that directory)

[masiorama]

Hello there, the image i linked show the errors from the debug but I just understood that those exceptions are raised when I try to send a test email without filling the fields. Otherwise the debug list is 0.
So maybe as you said the problem is related to the permissions of the mail folder, but I did not understand how to check.
I'm on Windows 10, and the folder exists, and I check with right click on the folder and properties, and the checkbox value of Read only is black square (which means undefined). If I uncheck it nothing happens, and if I check again is restored to black square.
The only files inside the mail folder are .keepme and an empty devilbox file with no content/extension.

I'm kinda lost here.
Thanks in advance for any help.

[cytopia]

As you are on Windows I would suggest to check out a different branch which should fix a lot of issues regarding mounts/Volumes. The PR for that is here: #383

When doing this, first try to spin it up with minimum required containers and without any custom configuration and go step by step:

docker-compose stop
docker-compose rm -f

git fetch --all --prune
git checkout docker-volumes
git pull origin docker-volumes

docker-compose up php httpd bind

[cytopia]

If the above Windows branch still does not work, then it might be related to the Docker version.

There seems to be an issue with Docker 18.06.1-ce, also reported by others here: #374

I would suggest to downgrade Docker back to 18.06.0-ce

However, as you are on Windows I would recommend to stay on the docker-volumes branch until it has been merged to master.

[masiorama]

Ok thank you very much, I'll test it and report back.

[cytopia]

I am also currently building CI tests around this feature so it can be catched as early as possible.

[masiorama]

Hello there, I just tried a few things:

• docker-compose down my current devilbox
• cloned and docker-composed docker-volumes and unfortunately the email catch-all behave the same (no email catched)
• after that I tried downgrading to docker CE 18.06.0, and the catch-all seems to work with docker-volumes.
• then I tried also with my current docker (main branch) with CE 18.06.0 and the result is that it didn't work.

I'm a bit confused, it seems to be partially due to docker version and partially to my current master devilbox config.

[cytopia]

@masiorama thanks for digging into this.

There's currently a general issue with 18.06.1-ce. This however does not seem to affect everybody. I still couldn't figure out what exactly is misbehaving with that version.

Additionally when it comes to Windows, mounts a treated very differently, that's why I am in the middle of rewriting the mounts in question to Docker volumes.

So stick with docker-volumes branch, until it hits master and stay away from the new Docker version.

[cytopia]

@masiorama

Intranet Email catch-all now has its own CI check against all PHP vs HTTPD versions and Docker vs Compose versions (latest five). See here: https://travis-ci.org/cytopia/devilbox/builds/465971267

This ensures compatibility against latest five Docker versions against latest five Compose versions for each and every PHP-FPM version. PR for that was here: https://github.com/cytopia/devilbox/pull/430/files#diff-3fac1a644d6e83bef0eca8aba4283dbaR1

However CI only tests against Linux (should be OS independent though)

[masiorama]

Thanks, I'm gonna test it in the next days.

[cytopia]

@masiorama does the release branch resolve your issue?

[masiorama]

@cytopia sorry for the huge delay, I'm gonna check it right now and keep you posted.

[masiorama]

If I'm not wrong, the branch to test is release/v1.0.0, right?
I cloned it and tested, but the behavior is the same as on the first comment (aka no email catched).

[cytopia]

If I'm not wrong, the branch to test is release/v1.0.0, right?

Yes, that's the one.

I cloned it and tested, but the behavior is the same as on the first comment (aka no email catched).

Hmm, that's strange. Can you paste the PHP code that sends the email please.

[masiorama]

I'm using the test mail accessible from localhost, no custom code at all.

[cytopia]

That should work without any hassle. I even have CI tests on the email catch-all functionality of the intranet. The only thing I could currently think of is:

1. Diversion divergence of the .env file
2. Custom *.ini or *.fpm.conf

Can you diff your .env against the provided env-example and also check that no custom configs apply to php itself or php-fpm

[masiorama]

To be sure it is not something I did with the customization I simply cloned the branch on an empty folder, did a cp env-example .env, and started devilbox docker-compose up mysql. That's all. I'm totally lost here ^_^'

[cytopia]

Could you run the following and see if it resolves it:

docker-compose stop
docker-compose stop rm -f
docker-compose up php httpd bind mysql

[masiorama]

I did

docker-compose stop
docker-compose rm -f
docker-compose up php httpd bind mysql

ERROR: Pool overlaps with other one on this address space

update:
I had to docker network prune and then the up worked.
Just tried and it works, tha email is catched!
It worked with docker-compose up mysql too.

Thanks!

[cytopia]

OK, maybe the docker-compose rm -f part was never done (?) and you still had old container states? Is this fixed for you now?

[masiorama]

Indeed it was never done recently. I tried with my other devilbox installation (v0.15) and unfortunately the mailcatch thing is not working, but on the release/v.1.0.0 it works perfectly, so I will switch to this installation and migrate all my projects there.
I'm gonna close the issue since the problem is solved.
Thank you very much for your assistance.

[cytopia]

Ok, glad it works for you now. I will also update the documentation to emphasize the need todocker-compose rm -f.

[cytopia]

@masiorama thanks again for this catch. I've updated the documentation accordingly and emphasized the importance of docker-compose rm in relevant sections:

https://github.com/cytopia/devilbox/pull/513/files

[cytopia]

Relevant documentation:

• https://devilbox.readthedocs.io/en/latest/getting-started/start-the-devilbox.html#stop-and-restart
• https://devilbox.readthedocs.io/en/latest/maintenance/update-the-devilbox.html#stop-container
• https://devilbox.readthedocs.io/en/latest/support/troubleshooting.html#what-to-always-do-first