More granular install_site

[nerdstein]

In a lot of ways, I want the functionality of the following line:

install_site = TRUE

Here is a problem: when I have an existing codebase, this line blows away my settings.php file.

In some sites I work on, this is the place in which global configuration is placed. It also may contain database settings, and such.

Blowing this away with Drupal's default.settings.php, while desirable for spinning up a brand new site, is not ideal for an existing code base.

However, I do want the ability for the following:

1. DrupalVM to provision a local database and database account
2. DrupalVM to (optionally) provision an environment-specific 'sandbox.settings.php' in lieu of overwriting settings.php, integration with settings.php and/or the host name settings to effectively bootstrap upon install

I am more than confident you have thought of this but I have yet to determine how to set this up.

[nerdstein]

Oh, and, what would be really slick is if I can somehow pull down an existing database to perform the site install (whether that's a local SQL file or a Drush alias)

[oxyc]

Not sure if there's a good way to solve this as the drush installation step just calls Drupal cores drupal_rewrite_settings function.

Where I work we do the following:

• install_site set to false.
• a version tracked sites/default/example.settings.local.php configured for the VM.
• a note in the projects README about copying this to settings.local.php and running the drush sql-sync @production @dev command.

The two last steps could be added to a post provisioning script. Out of experience host key verification is a nightmare to figure out properly though. I opted for less magic and just going with good docs.

[nerdstein]

@oxyc I don't foresee a need to always invoke Drupal's built-in installation. The term "install_site" is a bit ambiguous then.

I think there is merit to evaluating some more granular configuration where we separate the installation of DrupalVM's database and other scaffolding needed (like the set up of a VirtualHost) from DrupalVM's site installation routine.

Some configuration like "provision_database", "provision_vhost", "provision_drupal_site" (equating to a DrupalVM specific multi site), etc would be really beneficial.

[oxyc]

Might need some better docs. The only tasks that are done if install_site is set to true are:

• drush si to install the site (Drupals install.php)
• Enable the modules set in drupal_enable_modules

Neither of these are something you want to do when importing an existing database and it seems like these are the only two steps you wish to exclude as well.

In retrospect I guess drush_site_install would have been a better variable name.

[oxyc]

I believe @geerlingguy mentioned he would be interested in adding ansible tags if vagrant adds support for it. Meaning that if vagrant supported vagrant provision --tags=database, Drupal VM would add support as well. Currently that's not the case though.

[oxyc]

Sorry for spamming... One last thing. The site installation will only occur during the first provision when drush status bootstrap informs us that it's not installed yet. Any provision done after will not trigger the site installation.

Besides support for tags, do you consider the current functionality solves your use? Could definitely improve the docs on this at least, but do you feel there's anything else that should be changed?

[nerdstein]

@oxyc - you bring up a great point. We should really emphasize the use cases and try to articulate them more clearly. I'm not fully convinced this doesn't warrant some development attention. As such, here goes:

1. As a developer, I want to provision a DrupalVM from an existing code base and an existing database (from alias or from SQL)
2. As a developer, I want the ability to bring over aliases from my local system to use within DrupalVM for remote sites
3. As a developer, I want documentation (and configuration templates) for the various ways to configure DrupalVM for new sites or existing sites
4. As a developer, I want to have the ability to configure DrupalVM to not overwrite a standard settings.php file that ships with an existing codebase.

I would be curious to discuss this with @oxyc or @geerlingguy

[oxyc]

As a developer, I want to provision a DrupalVM from an existing code base and an existing database (from alias or from SQL)

Currently not supported. My personal opinion would be that an example post provisioning script would be enough. Otherwise might be something worth adding to geerlingguy.drupal if it gets cleaned up. @geerlingguy probably has better input.

As a developer, I want the ability to bring over aliases from my local system to use within DrupalVM for remote sites

The way I see it is a project should be entirely contained within a version controlled repository. Currently you can store drush aliases in sites/all/drush which is what I do for work. I feel that solves this issue. Here's some drush docs about where aliases can be stored. This might be something worth adding to the docs. Also, at some point I thought #406 was related to this but maybe it wasnt.

As a developer, I want documentation (and configuration templates) for the various ways to configure DrupalVM for new sites or existing sites

This is a constant work in progress and needs improvement for sure. http://docs.drupalvm.com/en/latest/deployment/local-codebase/ covers some of these already.

As a developer, I want to have the ability to configure DrupalVM to not overwrite a standard settings.php file that ships with an existing codebase.

Setting install_site: false solves this right?

[kevinquillen]

This was kind of the idea behind the Yeoman config generator, to abstract the config overrides to question prompts that explain it a bit more.

https://github.com/kevinquillen/generator-drupalvm

In any case, this is something that is likely solvable from updating or extending on the current documentation.

[nerdstein]

My personal opinion would be that an example post provisioning script would be enough.

I actually value DrupalVM because I don't have to script around it :) I'd love to find some way to incorporate this because I see it as a principle use case for installing a local VM and working with an existing site locally.

The way I see it is a project should be entirely contained within a version controlled repository

While I agree, I think most hosting providers and agencies maintain their aliases outside of repositories to centrally control them. It would be helpful if DrupalVM provided a construct to specify a list of local alias files (e.g. ~/.drush/mysite.aliases.php, ~/.drush/mysite2.aliases.php) to bring over and install upon provision.

Setting install_site: false solves this right?

Yes, but there may be merit to having another utility upon provision to create a DrupalVM specific settings file, as noted before.

Maybe an additional user story is:

As a developer, I want my DrupalVM specific settings to be added into my code-base as it's own settings file and potentially as it's own site.

Adding hooks to do so would be a huge time saver, in my opinion.

[nerdstein]

@kevinquillen I'm not familiar with the approach, but if it solves one or more of the user stories I mentioned, I'd be happy to try it

[oxyc]

My personal opinion would be that an example post provisioning script would be enough.

I actually value DrupalVM because I don't have to script around it :) I'd love to find some way to incorporate this because I see it as a principle use case for installing a local VM and working with an existing site locally.

True, and me too :) My concern is that there's quite a few steps to integrate this properly. Eg. in my case I need to use a ssh proxy to access our production environments as I keep signing in from weird IP locations around the globe. The VM also need to have the private key required to access production (or have a ssh agent running). And then we need to have ansible fetch and authorize the RSA fingerprints (and in my case I need a proxy for this). There are probably more scenarios which I missed and everyone uses a different combination of these. Doing all these things from simple config settings sounds difficult and might not even make it easier than just editing an example post provisioning script.

Edit: I had something like this in a custom make install task at work once, but when something failed it became impossible for anyone to figure out what went wrong (host key verification for production/vm, missing private key, missing public key on remote, or something else entirely). Which is why I opted for docs. This is what it had become before I scratched the idea of hiding it behind configs https://github.com/generoi/genero-conf/blob/0ae3edec42239854125635c7d6d9162e3a7958e1/make/production.mk#L12:L34

The way I see it is a project should be entirely contained within a version controlled repository

While I agree, I think most hosting providers and agencies maintain their aliases outside of repositories to centrally control them. It would be helpful if DrupalVM provided a construct to specify a list of local alias files (e.g. ~/.drush/mysite.aliases.php, ~/.drush/mysite2.aliases.php) to bring over and install upon provision.

Oh. You mean you have a separate repository for aliases only or how do you distribute these to everyone in the organisation? Also, why is this better than bundling the aliases with the repo in question?

Setting install_site: false solves this right?

Yes, but there may be merit to having another utility upon provision to create a DrupalVM specific settings file, as noted before.

I agree, this would make a nice addition now that Drupal actually has core support for it!

[geerlingguy]

I'm going to close this in favor of keeping the discussion around #305.

For personal use, I have about 12 clones of Drupal VM with different config.yml files and different post-provisioning scripts, along with self-managed Drush alias files that allow me to do things like:

drush @mysite.live sql-dump | drush @mysite.local sql-cli

That syncs the database back. Then I have another drush command, or a shell script, ansible playbook, etc., that resets everything locally.

In a team environment, there might be a set of Phing tasks, or a shell script, etc. that is called to finish the provisioning or resetting of a local developer's environment.

But in general, overarching changes to the individual "site synchronization" or "reset" (I shy away from calling that installation, because in my mind install means you have no database, no files... just a codebase, and you install a fresh database with either a standin local.settings.php or just a fresh new settings.php) will go into something like a Drupal VM 3.0, or even a side project like Lunchbox.

Acquia has an internal system called 'Bolt' that manages all those aspects (building, syncing, testing, resetting and debugging a site that's installed inside Drupal VM). I don't want to maintain both a development environment and a full deployment system all in one—there are too many variables there to make a nice, simple, maintainable solution.

I'd rather the deployment/synchronization solutions be managed separately. For myself, that almost always means a drush alias and a couple shell scripts.

[geerlingguy]

Also, for team-based aliases, I do something like a drush-aliases.sh script inside the scripts directory:

#!/bin/bash
#
# Configure Drush aliases for VMs.

DRUSH_HOME=/home/vagrant/.drush

mkdir -p $DRUSH_HOME
chown vagrant:vagrant $DRUSH_HOME
chmod 775 $DRUSH_HOME
cp /vagrant/scripts/projectname.aliases.drushrc.php $DRUSH_HOME/projectname.aliases.drushrc.php

And then I just stick the aliases (e.g. live, test, local) into projectname.aliases.drushrc.php next to the shell script.

[nerdstein]

I read over #305 and yes I think it makes sense.

I was thinking of creating a DrupalVM utilities Ansible role for these types of use cases. I may explore this on my own accord with the intent as a bolt-on set of candidate functionality.

[oxyc]

Any plans for Acquia to release Bolt? ;) For deployment I'm using capistrano and love it.

[geerlingguy]

I hope, but not my battle to fight right now :(

Have you seen the ansistrano role for Ansible?

[oxyc]

Oh, I haven't. I'll take a look. Thanks

[oxyc]

Nice, saw that Acquia recently released Bolt 👍

[geerlingguy]

@oxyc - Was it formally announced anywhere yet? I know they stuck it on GitHub... but I've been waiting for a more formal announcement before spreading the good news elsewhere :/

[oxyc]

Not that I've seen. I just found it by snooping around randomly. I solved one of the issues with my own base repo thanks to it \o/