Harmonize ACE and ACSF deploy artifacts

[danepowell]

When we added ACSF deployment support, we assumed that simultaneous deployments of projects to ACE and ACSF would be an edge case. However, after discussion with several project leads, we're realizing that maintaining parallel ACE and ACSF environments may actually be a best practice.

The problem is that right now, each deploy has to specifically target ACE or ACSF, and it generates a distinct build artifact for each. This is problematic for several reasons, the biggest being the overhead for the release manager who has to manage branches and tags in repos that are pseudo-remotes of each other. This has caused me no end of confusion and grief. It also adds complexity to the entire build process.

It would be much simpler if BLT simply generated a single build artifact that could be deployed to either ACE or ACSF, instead of having to specifically target one or the other.

The biggest obstacle to this will be finding a way to work around ACSF's rigidly defined settings.php.

[greylabel]

I would propose the idea of making the type of deploy artifact configurable/plug-able so we can easily add support for non-Acquia hosting services that need a differently structured deploy payload.

[grasmash]

@danepowell Can you provide more detail around the constraints that ACSF puts on settings.php, and why the pose issues? We can then brainstorm solutions.

I agree I'd like to prevent needing multiple deployment artifacts of different types.

@greylabel We did just merge a PR that will let you more easily customize the files that are copied into the deployment artifact, and those that are ignored. Though, we don't have a well developed concept of deployment artifact "types" which I think is a bit of a rabbit hole. In what way do you need to modify the artifact?

[greylabel]

@grasmash That might address most of my initial concerns. I was thinking about not needing to deploy Acquia hooks, factory-hooks, etc to non-AH. The other case I am considering is a deployment including rather involved front-end workflows/artifacts. Let me explore this a bit more and see where I run into hard edges with the new functionality you mentioned.

Speaking of a rabbit hole...

This does touch more broadly on how environment configuration is considered throughout the project and looking at alternative approaches to how local, hosting, and CI are differentiated. To @danepowell's point, right now we address environmental differences in settings.php, but I think this is a higher-level issue for the project and we will need to solve similar problems in many of the tasks, deployment artifacts in this case.

By not treating everything non-AH as local, but treating all environments as high-level peers, many of BLT's tasks could be made more contextual to the environment in which they are running. Thinking there might be reasons to have commands like setup:travis / setup:tugboat or deploy:acquia / deploy:internal-hosting. Even if only the Acquia options are bundled with BTL, I see value in extensibility, especially with the ever-growing number or local, CI, and hosting solution out there.

[grasmash]

@greylabel We did introduce some commands that target local and ci environments specifically--we could expand that to more tasks. See https://github.com/acquia/blt/blob/8.x/template/build/core/phing/tasks/ci.xml.

[danepowell]

So ACSF actually does a checksum of settings.php to make sure that it's bit-for-bit identical to the copy it provides here:
http://cgit.drupalcode.org/acsf/tree/acsf_init/lib/sites/default/settings.php?h=8.x-1.x

It runs this verification whenever acsf_init is run. BLT runs acsf_init during deploys to put this settings.php in place, and then ACSF runs the copy of acsf_init included in your project whenever it deploys code to verify that settings.php hasn't changed.

As you can see, that settings.php will not work as-is on ACE, and there's no way at all that I can see to extend or override it out of the box. I think it will be necessary to patch the ACSF module to make settings.php more flexible. Ideally we could get this patch accepted upstream, but more likely we'd have to maintain it locally.

[grasmash]

@danepowell I agree that the correct path forward is to make a change in the ACSF module upstream. I'm strongly opposed to maintaining a patch. We should have a discussion with the ACSF team and agree on a mutually beneficial solution.

[danepowell]

In the meantime, for anyone needing to deploy to ACE and ACSF in parallel, you should define one environment (let's assume ACE) as the "default" by configuring it in project.yml. Then create a file build/custom/phing/acsf.yml and in it define hosting: acsf and the git config for ACSF. Finally, add a second deploy step to .travis.yml and modify the BLT command like so:
./blt.sh -propertyfile build/custom/phing/acsf.yml -propertyfileoverride deploy

[msherron]

@danepowell quick add to the steps you listed above: you also need to add the ACSF repo url to the ssh_known_hosts lists in .travis.yml. Otherwise the build will time out while waiting to verify the host. Otherwise this worked out well for me.

[grasmash]

This was resolved by #471