Reliable builds and deploys

[kasparsd]

Currently the build process defined in Gruntfile.js relies on an unknown state of the checked out repository -- For example, it uses existing dependencies in /vendor and /node_modules directories without installing them according to the rules in the lock files.

For reliable and reproducible builds we need to isolate any local overrides and always start from a known state. I suggest to update the deploy process to the following steps:

• Create an empty /build/src directory.

• Copy the .git directory from the root to /build/src/.git.

• Checkout the required release branch such as develop or master. This ensures that we never accidentally deploy a feature brunch.

• Install all dependencies via npm install and composer install --no-dev --optimize-autoloader.

• Build assets npm run build and replace any version strings as necessary.

• Copy the release files to /build/release and make sure to exclude node_modules and other development specific files. Maybe introduce .distignore that defines all files and directories that shouldn't be part of the final release. Maybe use parse-gitignore to generate the exclude list.

• Now deploy the /build/release to WP plugin SVN or create a release zip file.

All of these steps can be made cross-platform, too.

[westonruter]

Some of this is addressed by #1836 correct? Note that the release instructions in contributing.md include a step to run npm install and composer install. It should also include a step to make sure the right branch is checked out.

But yeah, these are manual steps that would ideally be automated as much as possible.

[kasparsd]

@westonruter Yes, some of this is covered by #1836 and for reasons similar to what you mentioned in this comment it would be better to introduce a completely standalone build and deploy process.

Only by moving the build process away from the working directory we can ensure that none of the work in progress is affected.

With this kind of setup we could even make Travis deploy to the WP.org repo since it would follow the exact same steps as one does locally.

[swissspidy]

Thanks for opening this!

I would love to see continuous builds on Travis instead of having to do things like #1836. Happy to lead work on that front too.

Continuous deployments to WordPress.org is probably a stretch goal and not easy to do because of SVN credentials. I'd leave this out for the moment.

[swissspidy]

I would love to see continuous builds on Travis

I was just briefly looking at this again since I was working on the Travis config.

Travis supports uploading assets to GitHub releases for tagged builds. So this would run for new versions automatically, but could also work for nightly builds, where a nightly tag could simply always point to the latest commit in develop.

Another possibility would be using a GitHub Action. They're rather new and rely on Docker containers to put together such a workflow. It would probably mean way more work (and maintenance) than just using Travis CI's built-in integration that does the same job.

Either solution requires a GitHub API token with at least public_repo permission, which grants write access to all of a user's public repositories. So that might not be feasible to achieve.

The alternative would be to use a service that leverages GitHub's deployments API to upload builds somewhere else. For example, Now for GitHub allows that. I am not aware of any other service that has such an integration, but I assume the same could be built manually using GCP or similar.

With this kind of setup we could even make Travis deploy to the WP.org repo

10up have been working on a GitHub Action for this. Looks promising, but does not support plugins with a build process yet. So this is currently not an option.

[westonruter]

Moving from 1.2.1 milestone to 2.0 planning

[westonruter]

Can we leverage what Gutenberg has? https://github.com/WordPress/gutenberg/blob/master/bin/build-plugin-zip.sh

[schlessera]

The Gutenberg script doesn't seem to take Composer into account at all. But that might be straight-forward to add, provided there's no surprises with the build system itself.

[westonruter]

Improvements to the build process are being merged as part of #3183, but I'm leaving this issue open because it doesn't touch on the deployment aspects and the overall build pipeline should be evaluated, but that's out of the scope for what we need to get out the door for 1.3.

[pierlon]

Given the current state of the build process for the plugin I'd say reproducible builds are now possible, and so this issue can now be closed.