Can we work on a better way of keeping a full trellis build under git? #883

partounian · 2017-09-09T23:20:57Z

Submit a feature request or bug report

I've read the guidelines for Contributing to Roots Projects
This is a feature request
This is a bug report
This request isn't a duplicate of an existing issue
I've read the docs and followed them (if applicable)
This is not a personal support request that should be posted on the Roots Discourse forums

Replace any X with your information.

Of course there is the roots discourse thread from December, but it has issues in different ways, either overwrites local changes or creates merge conflicts for every single change. I imagine other people have issues also, I think we should come up with a solution to his issue with the subtrees.

The text was updated successfully, but these errors were encountered:

partounian · 2017-09-10T05:54:21Z

@tangrufus You have been doing some great work recently, any ideas? Or how did you setup your project(s)?

tangrufus · 2017-09-13T03:57:53Z

> How did you setup your projects?

More less the same as the subtrees method, but:

trellis and bedrock are 2 different git repos
keep .git despite the doc tells you to delete it
i also have a branch (tangrufus/trellis@project-template) to keep all my common customization to start a project
3 git remotes for every project:
- roots/trellis
- tangrufus/trellis
- actual origin of the project

$ git clone -o roots https://github.com/roots/trellis.git www-example-com-trellis
$ cd www-example-com-trellis
$ git remote add tangrufus https://github.com/tangrufus/trellis.git
$ hub create -p
$ git remote -v
origin	https://github.com/TangRufus/www-example-com-trellis.git (fetch)
origin	https://github.com/TangRufus/www-example-com-trellis.git (push)
roots	https://github.com/roots/trellis.git (fetch)
roots	https://github.com/roots/trellis.git (push)
tangrufus	https://github.com/tangrufus/trellis.git (fetch)
tangrufus	https://github.com/tangrufus/trellis.git (push)

## Merge my project temaplate
$ git fetch --all
$ git merge tangrufus/project-template

## Do your job...
## Commit and push to `origin` as usual

## When roots/trellis has been updated...
## Important: Reading change logs, commit messages and `git diff` are necessary
## Tips: Make use of git branches
$ git fetch --all
$ git merge roots/master

Do I recommend everyone to use this method?

No. Use it only if you are comfortable with git. Otherwise, git gives you more headache than benefits.

Should you always keep your forks update with roots/trellis?

Yes! Always!
This is actually missing from the doc: the doc tells you to delete .git but gives no instructions about updating.

However, updating Trellis is challenging, I don't think of a good and easy way that suitable for everyone.

Edit: By forks, I mean both origin and tangrufus remotes in the above example.

Avoiding merge conflicts

Since I use git a bit differently than the subtrees method, I got less merge conflicts.

Tips:

Whenever possible, don't change existing files in roles/*
When changing j2 templates, make use of variables
When changing variable values, define them in group_vars/<env>/xxx.yml
When a j2 template doesn't allow customizing with variables, submit pull requests, example: Allow overriding php-fpm params by ansible variables #856
When you need to add more tasks:
- Bad: change existing files in roles/*
- Good: make a new role and add it into dev.yml / server.yml
- Best: extract it to galaxy.ansible.com
- see also: add provisioning-hooks directory to operate like deploy-hooks #830, Add Provisioning Hooks that Allow Custom Tasks #882
Whenever possible, don't use Nginx child templates because you lose all the benefits Trellis update gives you

Let say you want to change something in sites-available/example.com.conf:

Best: define variables in group_vars/<env>/xxx.yml
Good: template out to {{ nginx_path }}/includes.d/{{ item.key }}/xxx.conf but beware Trellis might delete your files during this task, see how I do it
Bad: use Nginx child templates is the last resort

⚠️

Reading change logs, commit messages and git diff are necessary
No merge conflict doesn't mean it won't break
Same commit doesn't mean same server, see: Normalize apt tasks #881 (comment)

The ultimate way to avoid merge conflict

Get it merged into roots/trellis

partounian · 2017-09-13T04:13:12Z

And just to confirm you create a second repo just for bedrock and the actual WP related stuff?

tangrufus · 2017-09-13T04:21:45Z

Correct.

For every project, I have 2 git repo:

example-project-com-trellis
example-project-com-bedrock

You need to change group_vars/<env>/wordpress_sites.yml accordingly

trellis/group_vars/production/wordpress_sites.yml

Lines 12 to 13 in db89d49

    
           repo: git@github.com:example/example.com.git # replace with your Git repo URL 
        
           repo_subtree_path: site # relative path to your Bedrock/WP directory in your repo

swalkinshaw · 2017-09-18T18:39:04Z

Of course there is the roots discourse thread from December, but it has issues in different ways, either overwrites local changes or creates merge conflicts for every single change

If you make changes to Trellis as @tangrufus outlined, merge conflicts and overwritten local changes shouldn't happen too much.

One thing which I don't know has been mentioned on on Discourse is specifying merge strategies for specific sub paths.

So if you know that roles/ is free of local changes, you can run git merge -X theirs subtree=trellis/roles --squash trellis/master (or something like that).

-X theirs means git will automatically pick every change from trellis/master without prompting for conflicts. ours does the reverse which can be useful for certain files/dirs.

swalkinshaw · 2017-09-18T18:39:35Z

We definitely need some guides/docs about:

how to properly customize/change Trellis
how to update Trellis

partounian · 2017-09-18T20:27:37Z

I believe it is `-X ours` not `yours`, I'll definitely follow Tang's plan in the future, it'd probably be a headache for my pre-existing ones.

swalkinshaw · 2017-09-18T20:30:23Z

Correct, updated it

partounian · 2017-10-22T15:37:46Z

@tangrufus Hey Tang, I've almost wrapped up a setup according to your specs, except I'm stuck on a nginx-includes situation. Basically I want to add location blocks to remote servers, in my case one file to do browser caching and another to disable logging for the favicon.

I feel like browser caching should be added into Trellis and enabled when we enable cache. Thoughts? @swalkinshaw

swalkinshaw · 2017-10-22T15:42:49Z

We can't really do that automatically. Just because you enable fast-cgi caching doesn't mean you have a proper setup with asset digests.

partounian · 2017-10-22T15:44:18Z

Super sorry @swalkinshaw I was just reading other the tickets about browser caching, forgot that you have some concerns with browser caching.

strarsis · 2017-11-26T14:51:57Z

Couldn't we make a "child config" from the base roots config?
It works so well with themes 😄

partounian · 2017-12-01T20:19:24Z

@tangrufus I've been using your method for a current project and it works great. Two notes though.

It can be slightly simplified by your project's trellis having two remotes not three, one being origin and the second being your user's trellis. You would just keep project-template up-to-date and rebase off of that instead of roots remote.

And the second one is a question. Any idea how to keep bedrock up-to-date without having to deal with merge conflicts for composer.lock?

tangrufus · 2017-12-04T03:01:28Z

deal with merge conflicts for composer.lock?

You don't. First resolve conflicts in composer.json, then $ composer update. Let composer generates a new composer.lock for you.

partounian · 2017-12-04T04:01:19Z

Yup that's what I do. I know it's best practice to keep composer.lock in vc, but I wonder if it'll work fine without it being in vc. Especially since it's wp packages and not anything with other Composer dependencies.

On Sun, Dec 3, 2017, 7:01 PM Tang Rufus ***@***.***> wrote: deal with merge conflicts for composer.lock? You don't. First resolve conflicts in composer.json, then $ composer update. Let composer generate a new composer.lock for you. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#883 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACbIG0vR9isVNOYp62U5IAn_8HuayeBmks5s82CMgaJpZM4PSJjF> .

-- Thank you, Patrick Artounian

tangrufus · 2017-12-09T13:38:03Z

It works without composer.lock. However, the deployment will be a little bit different. Trellis may silently upgrade your wp core / plugins / themes during deploy.

Reason

Trellis runs $ composer install during deploy.
$ composer install with .lock always install the exact versions listed.
$ composer install without .lock works as $ composer update.

Workaround

If you really want to exclude .lock from git, always specify the exact package versions in composer.json. Otherwise, trellis will silently upgrade your wp core / plugins / themes during deploy.

Recommendation

Commiting your composer.lock is the best practice.
Use the workaround only if you have a very strong reason.

See:

partounian · 2017-12-09T16:27:35Z

Yup, that is what I thought. I actually always specify the exact version, as not all minor versions can be without breaking changes.

robrecord · 2019-04-10T21:46:46Z

deal with merge conflicts for composer.lock?

You don't. First resolve conflicts in composer.json, then $ composer update. Let composer generates a new composer.lock for you.

To avoid unintentionally upgrading packages, use composer update --lock instead. This will only update the lock file.

robrecord · 2021-06-19T23:15:25Z

@tangrufus ,

Thank you for laying out your personal method of updating trellis.

Can I ask what is the benefit you see in having bedrock and trellis in two different repositories? For updating trellis it’s a simpler git operation, but beyond that?

Plus, unless I have it set up wrong, the virtualenv isn’t operational inside the bedrock directory when I set it up in two repos.

I love the idea of keeping a set of customisations in a branch of a fork, and keeping it up to date. I will be doing this with bedrock also.

Are you still doing it all this way? Any changes/improvements you could mention?

Thank you.

strarsis · 2021-06-19T23:46:15Z

@robrecord: My two cents here:
Each Bedrock copy is a site. Trellis is a server that primarily hosts bedrock sites.
A Bedrock site can also be hosted on non-Trellis servers and on multiple, different Trellis servers.

Bedrock-sites are made to be like 12-factor apps, with configuration, data, etc cleanly separated.
So a Bedrock-site shouldn't need to care about the server it is hosted on (besides the credentials and some PHP requirements of course).

robrecord · 2021-06-20T01:00:34Z

@strarsis - thank you for adding these considerations. Absolutely, I agree with this. It does seem like a there could be a natural possibility for separation here.

The thing that is bringing me back is that my trellis instances only ever reference one site, and as such are configured only for that site. They would not be useful for any other. So while the bedrock instance can exist without the trellis instance, it is not the case the other way round. Plus trellis is used to generate the .env file (though not strictly necessary).

Thus, while the separation is possible, the two also seem to belong together in one repo and it feels unnatural to maintain a separation.

If my resistance ends up being purely about semantics then maybe I need to give it up and try the two separate repos again, but I'd like to be fully aware of all the practical benefits.

tangrufus · 2021-06-22T01:58:18Z

Can I ask what is the benefit you see in having bedrock and trellis in two different repositories? For updating trellis it’s a simpler git operation, but beyond that?

Simpler git operations
No need to give away server side stuffs to 3rd party developers
Bedrock site can also be hosted on non-Trellis servers

Simpler git operations

With the roots/trellis and my trellis fork as the git remotes, updating with roots or my common customizations is just 1 $ git merge away. And, these git operations is extactly what you do when doing open source PRs. Thus, the whole team is familiar with the process.

However, if your team knows git well enough, using the git subtree way could also achive the same goal.

No need to give away server side stuffs to 3rd party developers

There are times we hire freelancers to do one-off tasks. In those cases, we don't want them to see the trellis repo because they have no need to change server side stuffs nor to know what's in vault.yml.

Besides, when clients leaving us, we could just handover the bedrock repo to the new devlopers. 1 less document to write because:

Bedrock-sites are made to be like 12-factor apps, with configuration, data, etc cleanly separated.
So a Bedrock-site shouldn't need to care about the server it is hosted on

-- #883 (comment)

Bedrock sites can also be hosted on non-Trellis servers

A Bedrock site can also be hosted on non-Trellis servers

-- #883 (comment)

I stand with the idea of "unless your team has experts in all of DevOps, database, linux, security, nginx, redis/memcached, and every other tools WordPress servers might use, don't host clients' websites yourself."

As I am using Kinsta, only the deploy part of Trellis is in use. If a better deployment workflow come along, I might just ditch Trellis.

robrecord · 2021-06-25T10:53:03Z

Thanks for the detailed reply, @tangrufus !

The second point, I hadn't considered.

partounian changed the title ~~Can we work on a better way of keeping a full trellis build under git~~ Can we work on a better way of keeping a full trellis build under git? Sep 10, 2017

swalkinshaw closed this as completed Sep 18, 2017

tangrufus mentioned this issue Nov 18, 2018

Extract Sage 9 compiler #1030

Closed

tangrufus mentioned this issue Feb 21, 2020

Add deploy start & success notifications & multiple slack channel support ItinerisLtd/trellis-slack-webhook-notify-during-deploy#7

Closed

techieshark mentioned this issue Dec 30, 2021

extract nginx package as variable? #1345

Closed

fkohrt mentioned this issue May 24, 2022

trellis update trellis roots/trellis-cli#135

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can we work on a better way of keeping a full trellis build under git? #883

Can we work on a better way of keeping a full trellis build under git? #883

partounian commented Sep 9, 2017

partounian commented Sep 10, 2017

tangrufus commented Sep 13, 2017 •

edited

Loading

partounian commented Sep 13, 2017 via email •

edited by swalkinshaw

Loading

tangrufus commented Sep 13, 2017

swalkinshaw commented Sep 18, 2017 •

edited

Loading

swalkinshaw commented Sep 18, 2017

partounian commented Sep 18, 2017 via email •

edited

Loading

swalkinshaw commented Sep 18, 2017

partounian commented Oct 22, 2017

swalkinshaw commented Oct 22, 2017

partounian commented Oct 22, 2017

strarsis commented Nov 26, 2017

partounian commented Dec 1, 2017

tangrufus commented Dec 4, 2017 •

edited

Loading

partounian commented Dec 4, 2017 via email

tangrufus commented Dec 9, 2017 •

edited

Loading

partounian commented Dec 9, 2017

robrecord commented Apr 10, 2019 •

edited

Loading

robrecord commented Jun 19, 2021

strarsis commented Jun 19, 2021

robrecord commented Jun 20, 2021

tangrufus commented Jun 22, 2021 •

edited

Loading

robrecord commented Jun 25, 2021 •

edited

Loading

Can we work on a better way of keeping a full trellis build under git? #883

Can we work on a better way of keeping a full trellis build under git? #883

Comments

partounian commented Sep 9, 2017

Submit a feature request or bug report

partounian commented Sep 10, 2017

tangrufus commented Sep 13, 2017 • edited Loading

> How did you setup your projects?

Do I recommend everyone to use this method?

Should you always keep your forks update with roots/trellis?

Avoiding merge conflicts

⚠️

The ultimate way to avoid merge conflict

partounian commented Sep 13, 2017 via email • edited by swalkinshaw Loading

tangrufus commented Sep 13, 2017

swalkinshaw commented Sep 18, 2017 • edited Loading

swalkinshaw commented Sep 18, 2017

partounian commented Sep 18, 2017 via email • edited Loading

swalkinshaw commented Sep 18, 2017

partounian commented Oct 22, 2017

swalkinshaw commented Oct 22, 2017

partounian commented Oct 22, 2017

strarsis commented Nov 26, 2017

partounian commented Dec 1, 2017

tangrufus commented Dec 4, 2017 • edited Loading

partounian commented Dec 4, 2017 via email

tangrufus commented Dec 9, 2017 • edited Loading

Reason

Workaround

Recommendation

partounian commented Dec 9, 2017

robrecord commented Apr 10, 2019 • edited Loading

robrecord commented Jun 19, 2021

strarsis commented Jun 19, 2021

robrecord commented Jun 20, 2021

tangrufus commented Jun 22, 2021 • edited Loading

Simpler git operations

No need to give away server side stuffs to 3rd party developers

Bedrock sites can also be hosted on non-Trellis servers

robrecord commented Jun 25, 2021 • edited Loading

tangrufus commented Sep 13, 2017 •

edited

Loading

partounian commented Sep 13, 2017 via email •

edited by swalkinshaw

Loading

swalkinshaw commented Sep 18, 2017 •

edited

Loading

partounian commented Sep 18, 2017 via email •

edited

Loading

tangrufus commented Dec 4, 2017 •

edited

Loading

tangrufus commented Dec 9, 2017 •

edited

Loading

robrecord commented Apr 10, 2019 •

edited

Loading

tangrufus commented Jun 22, 2021 •

edited

Loading

robrecord commented Jun 25, 2021 •

edited

Loading