Use Docker-based Travis CI tests for multi-platform support

[oxyc]

I realised that we probably can't integrate these for a while due to stuff like geerlingguy/ansible-role-php-xhprof#8, but I thought I'd send it over for some brainstorming.

1. I came up with at least a temporary solution for moving the drush_makefile to the VM before running it, instead of relying on it being a vagrant environment. If Add support for storing drupal vm in a subdirectory #378 gets merged, we should pass the host_project_root var to ansible and use that instead. This issue was identified earlier in Make Drupal VM prod-ready (on a DigitalOcean 1GB Droplet) #84.
2. Also re-arranged the order of how drush files are generated so if run locally at least the local alias will be kept instead of overridden by an invalid remote alias.
3. Besides that I added some basic sanity checks to see that some of the stuff in the example.config.yml works out of the box. If we can get travis provisioned correctly, maybe we should create a tests/ directory and move the tests there instead (so you could also run them locally). One rather nice test framework that could fit is bats. I've used it on a couple of projects before.

If you don't think we can run the entire playbook on travis (maybe just the test duration is too annoying already), I can cherry pick the other commits to separate PRs if you want. I guess even without the added tests, this would be a good step forward to solving #84.

[geerlingguy]

@oxyc - I'm currently running every individual role in Travis (it's an Ubuntu 12.04 environment by default), and that works well enough.

I definitely would like to add a 'tests' directory in the root, and put in whatever extra testing stuff we need there.

My super-amazing-moonshot-endgoal would be to set up tests like: https://github.com/geerlingguy/ansible-role-apache/blob/master/.travis.yml

It would bootstrap one Docker environment for each of the supported OSes, and inside those Docker environments, run playbook.yml. We can't test Vagrant inside Travis' infrastructure, but we can test all the Ansible bits, and treat Docker as an additional destination besides Vagrant VMs.

[oxyc]

The docker setup you have for the apache role is pretty impressive for sure! You said you were ironing out some kinks still?

Even if the individual roles have tests, I feel the project could still benefit from some integration tests to catch regressions. The tests added in this commit would have caught #411 for example. Would also verify that drush aliases work. Probably a few other slightly fragile pieces that we should be testing.

[oxyc]

Adminer test passes on travis already!

Any idea what causes the failing test, something travis specific?

TASK: [Ensure MySQL AppArmor profile is disabled (for slow query log).] ******* 
failed: [localhost] => {"failed": true, "path": "/etc/apparmor.d/disable/usr.sbin.mysqld", "state": "absent"}
msg: Error while linking: [Errno 2] No such file or directory

[geerlingguy]

@oxyc - This is the main reason I'd like to isolate inside Docker—these kinds of errors pop up all the time because Travis does some insane stuff to their environments. Not to mention they have about a jillion different things installed that break every preconceived notion you can make about a default Ubuntu 12.04 install :)

And yes, I have some kinks to work out for my Ansible roles, but if we can lift what's there for Drupal VM, I have a feeling it'll be very helpful here, and since this use case is different than my individual roles, I wouldn't wait until I work out the rest of the kinks to use the same setup.

Even if the individual roles have tests, I feel the project could still benefit from some integration tests to catch regressions.

Yes, 100%—I think I was a little unclear in my post. I would really like some Drupal VM integration tests! Can't get perfect, since I don't have a spare Mac and Windows laptop laying around to do end-to-end testing with Vagrant... but we can get pretty close by testing all the Ansible bits inside Docker containers.

[oxyc]

Ah makes sense. Okay will mostly hold off on this then. Now I'm just curious to see how far it can go before bailing out completely. So I might push a few changes.

[geerlingguy]

@oxyc - Sounds good—it's Travis' infrastructure, so feel free to hit it hard with more tests :)

[oxyc]

Well that was surprising, just ignoring one AppArmor restart error and it passes.

[oxyc]

Aha #346 (comment)! That explains why my CentOS 6 build kept failing!

So I modified the test so nginx is tested on CentOS 6 as apache doesn't work out of the box anyways.

So with this we're testing:

• CentOS 7 with apache
• CentOS 6 with nginx

Let's hope the tests pass :) @geerlingguy what do you think about my build_makefile change? Will probably cause a lot of invalid bug reports but it would also help solve #84. Actually it would more or less solve it as I've successfully built it on a docker container now.

Edit: I removed the AppArmor missing directory task as that was travis specific.

[geerlingguy]

Can we just include the ../../ in the path (in example.config.yml) and that way people can specify absolute or relative paths to use? Either way we'll get bug reports... but this way we improve the feature and make use on non-Vagrant environments easier :)

[geerlingguy]

If this passes, I think the only change I'd request for now is to update the path for the makefile to include the ../../

[oxyc]

Fixed the path, it's definitely better that way.

An earlier build already passed yey :) https://travis-ci.org/geerlingguy/drupal-vm/builds/107331206

[geerlingguy]

@oxyc - So is the only holdup to getting ubuntu12/14 tests going the mysql tests? I'd like to focus on getting those fixed asap so we can unblock a whole chain of different tests and make this the most automatedly-tested awesome project ever :)

[oxyc]

Currently that's what's failing yes. And any other role that relies on upstart.

[oxyc]

Builds are pretty slow though hehe. Wonder why it's specifically CentOS 6 every time. And more specifically:

TASK: [geerlingguy.php | Ensure PHP packages are installed.] ******************
changed: [localhost] => (item=php,php-cli,php-common,php-devel,php-fpm,php-gd,php-imap,php-ldap,php-mbstring,php-opcache,php-pdo,php-pear,php-pecl-apc,php-xml,php-xmlrpc)

this one takes several minutes.

[geerlingguy]

I wonder if the 6.x PHP repos are just really slow or something :/

[geerlingguy]

@oxyc - I'm going to merge this now, just because it gives a heck of a lot better coverage than we're currently providing. We will continue to work on the Ubuntu support for MySQL and here, thanks so much for the help on the geerlingguy.security role and for this. This is awesome. Mind-blowingly awesome!

[oxyc]

Hmmm https://travis-ci.org/geerlingguy/drupal-vm/builds/108702338

Since the passing tests for cb3d7ac (https://travis-ci.org/geerlingguy/drupal-vm/builds/107337793) this has been added: cb3d7ac...master.

Seems it's pretty unstable as it should still be passing. I just triggered 3 more tests so lets see how they come back

[oxyc]

So seems it's just unstable. The tests for #404 passed.

[geerlingguy]

@oxyc - That earlier build was due to a mailhog test gone bad; I updated a path in the upstream daemonize role, but forgot to update the default in mailhog as well.

[okapies]

@oxyc @geerlingguy

Hi. I also get trapped in this problem on Travis this week and found that docker run -d --privileged /sbin/init makes CPU 100% and lead to death. You should use such like docker run -d -it /bin/bash instead.

https://bugzilla.redhat.com/show_bug.cgi?id=1046469

[geerlingguy]

lead to death

Welcome back from the dead then! 😄

Thanks for posting this tidbit.