Add box and support for Debian 9 'Stretch'

[geerlingguy]

Issue Type

• Feature Idea

Summary

Debian Stretch was released a short while ago, and it should hopefully be easy enough to adapt the current Debian 8 box to Debian 9.

I'm already working on testing a Debian 9 Docker image, and will hopefully get that released within a few hours. The box will take a tiny bit longer to build, but should be pretty quick if there aren't many substantive changes (e.g. like initv to systemd...).

[geerlingguy]

Added a Debian 9 (Stretch) Ansible Docker image; the only changes required are documented in this commit: geerlingguy/docker-debian9-ansible@1a73a34 (needed to install python-setuptools and python-wheel to get Ansible's installation via Pip to work).

[geerlingguy]

I also had to install systemd (since by default it's not all there—the binaries used to launch things / respond to events are not present in at least the Docker image).

[geerlingguy]

First and only issue found so far is with my Varnish role:

The repository 'https://repo.varnish-cache.org/debian stretch Release' does not have a Release file.

Full error below:

TASK [geerlingguy.varnish : Add Varnish apt repository.] ***********************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: apt.cache.FetchFailedException: E:The repository 'https://repo.varnish-cache.org/debian stretch Release' does not have a Release file.
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_5C07Yh/ansible_module_apt_repository.py\", line 565, in <module>\n    main()\n  File \"/tmp/ansible_5C07Yh/ansible_module_apt_repository.py\", line 553, in main\n    cache.update()\n  File \"/usr/lib/python2.7/dist-packages/apt/cache.py\", line 464, in update\n    raise FetchFailedException(e)\napt.cache.FetchFailedException: E:The repository 'https://repo.varnish-cache.org/debian stretch Release' does not have a Release file.\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 0}

Looks like it might be related to varnishcache/pkg-varnish-cache#54, and also because I might not be using the new/official apt repo that has all the latest updates...

[geerlingguy]

See related upstream issue: geerlingguy/ansible-role-varnish#58

[geerlingguy]

Postponing this for now. I opened a PR (#1453)—but I'll leave it at that until the Varnish role is happy and working on Stretch.

[geerlingguy]

Vagrant box Packer config: https://github.com/geerlingguy/packer-debian-9
Vagrant box on Vagrant Cloud: https://app.vagrantup.com/geerlingguy/boxes/debian9

[geerlingguy]

Box is up, and I'm testing it with the defaults except for Varnish.

[geerlingguy]

I'm trying to use NFS shared folders, but it seems the mount command gets stuck; something with the Debian 9 box's networking is messed up, because I can't even ping the host at 192.168.88.1.

It gets stuck here:

==> drupalvm: Preparing to edit /etc/exports. Administrator privileges will be required...
==> drupalvm: Mounting NFS shared folders...

I went through the debug process using the command VAGRANT_LOG=debug vagrant up, and found another interesting error earlier in the build:

 INFO subprocess: Starting process: ["/usr/local/bin/VBoxManage", "guestproperty", "get", "3329e230-fb50-42fb-8527-917877d39fa7", "/VirtualBox/GuestInfo/Net/1/V4/IP"]
DEBUG subprocess: Command not in installer, not touching env vars.
 INFO subprocess: Command not in installer, restoring original environment...
DEBUG subprocess: Selecting on IO
DEBUG subprocess: stdout: No value set!
DEBUG subprocess: Waiting for process to exit. Remaining to timeout: 32000
DEBUG subprocess: Exit status: 0
 INFO retryable: Retryable exception raised: #<Vagrant::Errors::VirtualBoxGuestPropertyNotFound: Could not find a required VirtualBox guest property:
  /VirtualBox/GuestInfo/Net/1/V4/IP
This is an internal error that should be reported as a bug.>

Googling that now.

Edit: Also got:

 INFO retryable: Retryable exception raised: #<Vagrant::Errors::NFSMountFailed: The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

mount -o vers=3,udp 192.168.88.1:/Users/jgeerling/Dropbox/Development/GitHub/drupal-vm /var/www/drupalvm

Stdout from the command:



Stderr from the command:

mount.nfs: Connection timed out

[geerlingguy]

See more debugging here: hashicorp/vagrant#7138 — as it turns out it was because I was adding a loopback alias on the same IP address I was using for Drupal VM. I think it might be beneficial to update the Docker docs at some point and choose a different default IP, just to prevent this same thing from happening to others who just want to play with Docker and Drupal VM and all the defaults...

[geerlingguy]

So, just ran a complete build with all the defaults except for removing varnish from installed_extras... and it worked flawlessly, yay!

So the minimal changes I had to make will go into the next release, and then I'll just hold off from merging the automated testing until the Varnish repos work on Debian 9.

[geerlingguy]

Well, now Varnish install works (see build https://travis-ci.org/geerlingguy/drupal-vm/jobs/247773638 and upstream issue varnishcache/pkg-varnish-cache#78), but Drupal install fails. Might be unrelated.

[oxyc]

Seems unrelated. It works in a VM so I wonder if it's some init system thing again? The drush site-install task doesn't seem too see the database and therefore tries to create it.

[geerlingguy]

Testing a rebase and will see what happens. I found that there are some weird database issues now on Mac, over in #1497

If it turns out that is the problem, then I'll have to figure out how to tell Docker to use a volume for /var/lib/docker in Travis CI. Shouldn't be too terribly difficult. Hopefully.

[geerlingguy]

So, after almost the entire build completes, I get:

TASK [geerlingguy.drupal : Install Drupal with drush.] *************************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["/var/www/drupalvm/drupal/vendor/drush/drush/drush", "site-install", "standard", "-y", "--root=/var/www/drupalvm/drupal/web", "--site-name=Drupal", "--account-name=admin", "--account-pass=admin", "--db-url=mysql://drupal:drupal@127.0.0.1/drupal"], "delta": "0:00:00.229405", "end": "2017-08-03 04:05:14.846887", "failed": true, "rc": 1, "start": "2017-08-03 04:05:14.617482", "stderr": "Failed to create database: ERROR 1698 (28000): Access denied for user\u001b[31;40m\u001b[1m[error]\u001b[0m\n'root'@'localhost'", "stderr_lines": ["Failed to create database: ERROR 1698 (28000): Access denied for user\u001b[31;40m\u001b[1m[error]\u001b[0m", "'root'@'localhost'"], "stdout": "You are about to CREATE the 'drupal' database. Do you want to continue? (y/n): y", "stdout_lines": ["You are about to CREATE the 'drupal' database. Do you want to continue? (y/n): y"]}

The last things that were done with MySQL were:

TASK [geerlingguy.mysql : Ensure MySQL databases are present.] *****************
changed: [localhost] => (item={u'collation': u'utf8mb4_general_ci', u'name': u'drupal', u'encoding': u'utf8mb4'})
TASK [geerlingguy.mysql : Ensure MySQL users are present.] *********************
changed: [localhost] => (item=(censored due to no_log))

I might have to build it locally to see what exactly happens in here. Maybe Debian 9's MySQL defaults are a little different than expected?

[geerlingguy]

Annoyingly, if I log in via docker exec and run:

drush site-install standard -y \
    --root=/var/www/drupalvm/drupal/web \
    --site-name="Testing" \
    --account-name=admin \
    --account-pass=admin \
    --db-url=mysql://drupal:drupal@localhost/drupal

Then I get:

You are about to DROP all tables in your 'drupal' database. Do you want to continue? (y/n): y
Starting Drupal installation. This takes a while. Consider using the --notify global option.                 [ok]
Installation complete.                                                                                       [ok]
Congratulations, you installed Drupal!                                                                       [status]

[geerlingguy]

Ah, but Access denied for user 'root'@'localhost' — how is it using root instead of drupal?

Especially when the error message even says: --db-url=mysql://drupal:drupal@127.0.0.1/drupal. Puzzling.

[geerlingguy]

Running the exact command manually results in the same failure. Interesting:

root@drupalvm:/# /var/www/drupalvm/drupal/vendor/drush/drush/drush site-install standard -y --root=/var/www/drupalvm/drupal/web --site-name=Drupal --account-name=admin --account-pass=admin --db-url=mysql://drupal:drupal@127.0.0.1/drupal
You are about to CREATE the 'drupal' database. Do you want to continue? (y/n): y
Failed to create database: ERROR 1698 (28000): Access denied for user 'root'@'localhost'                     [error]

Maybe it's a Drush bug with Debian 9? If I run the other command (using global drush at /usr/local/bin/drush—version 8.1.10), then it works fine.

[geerlingguy]

Hmm. The one change I made was using localhost instead of 127.0.0.1—and Drush 8.1.10 doesn't seem to work either, with the IP. So testing just using localhost as the default now.

[geerlingguy]

Posted notes upstream in Drush issue that seems to be most closely related: drush-ops/drush#2183 — tl;dr:

Maybe MySQL itself has some weird undocumented limitation when using IP vs. socket with what order of variable precedence it uses?

[geerlingguy]

Sooooo close: https://travis-ci.org/geerlingguy/drupal-vm/jobs/262075833

Varnish install fail
Script ./tests/run-tests.sh handling the run-tests event returned with error code 1

[geerlingguy]

Strange, apparently Varnish is listening on port 6081—but only on Debian 9. This Docker image is messing with my mind.

# cat /etc/default/varnish | grep VARNISH_LISTEN_PORT
VARNISH_LISTEN_PORT=81
DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \

[geerlingguy]

Ah... but the systemd config is different:

# cat /lib/systemd/system/varnish.service 
[Unit]
Description=Varnish Cache, a high-performance HTTP accelerator

[Service]
Type=forking

# Maximum number of open files (for ulimit -n)
LimitNOFILE=131072

# Locked shared memory - should suffice to lock the shared memory log
# (varnishd -l argument)
# Default log size is 80MB vsl + 1M vsm + header -> 82MB
# unit is bytes
LimitMEMLOCK=85983232

# On systemd >= 228 enable this to avoid "fork failed" on reload.
#TasksMax=infinity

# Maximum size of the corefile.
LimitCORE=infinity

# Set WARMUP_TIME to force a delay in reload-vcl between vcl.load and vcl.use
# This is useful when backend probe definitions need some time before declaring
# configured backends healthy, to avoid routing traffic to a non-healthy backend.
#WARMUP_TIME=0

ExecStart=/usr/sbin/varnishd -a :6081 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m
ExecReload=/usr/share/varnish/reload-vcl

[Install]
WantedBy=multi-user.target

[geerlingguy]

Fixed the Varnish issue via geerlingguy/ansible-role-varnish#66 — now waiting on a test build to come back green! 🤞🏼