Use @wordpress/env (wp-env) for local dev environment

[swissspidy]

Summary

Instead of using a custom Docker setup for our local development environment, this uses the pre-existing @wordpress/env package dedicated for this task.

This reduces the maintenance burden and simplifies setup for contributors. Over time, more and more things can be moved to run within wp-env, eventually removing the need to manually install PHP, Composer, and PHPUnit, for example.

Relevant Technical Choices

• Replaces custom Docker setup with dedicated package.
• Patches @wordpress/env package to allow running PHP unit tests via wp-env (otherwise blocked by wp-env: docker-compose fails on PHP < 7.2 due to Xdebug WordPress/gutenberg#27783)
• Does not yet use Composer (and thus run PHPCS through it) as provided by the container (blocked by Composer 2 Support #5083)
• Contains some special handling to support the latest WordPress version (blocked by wp-env with "core: null" installs WP 5.4.2, not the latest version (5.6.0) WordPress/gutenberg#27763)

To-do

• Rebase after Avoid duplicate poster generation #5695

Upstream blockers

WordPress/gutenberg#27783
WordPress/gutenberg#28703 / WordPress/gutenberg#29430
WordPress/gutenberg#29323
WordPress/gutenberg#28201
WordPress/gutenberg#27763
WordPress/gutenberg#20569

So most of https://github.com/WordPress/gutenberg/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3A%22%5BPackage%5D+Env%22 basically

Additional Context

User-facing changes

N/A

Testing Instructions

1. npm run wp-env start -- --update --debug
2. npm run test:e2e:prepare
3. npm run test:e2e

---

Fixes #2521

[spacedmonkey]

Seems like they are using composer v2 in the WordPress container. Related #5083

[spacedmonkey]

This error keeps happening.

node-pre-gyp
WARN Using request for node-pre-gyp https download
node-pre-gyp
WARN Tried to download(404): https://axonodegit.s3.amazonaws.com/nodegit/nodegit/nodegit-v0.26.5-node-v83-linux-x64.tar.gz 
node-pre-gyp WARN Pre-built binaries not found for nodegit@0.26.5 and node@14.15.1 (node-v83 ABI, glibc) (falling back to source compile with node-gyp)
/bin/sh: 1: krb5-config: not found
gyp: Call to 'krb5-config gssapi --libs' returned exit status 127 while in binding.gyp. while trying to load binding.gy

Once this is fixed, I think that it lots more things will start working.

[swissspidy]

You also need to fix npm run env:stop and npm run env:start. But I think you can just replace npm run wp-env start. You need to active plugins, setup consts, create users and import media. I think there still needs to be a start.sh file.

The wp-env.json config file takes care of all of this, except for adding users. But I think we could also easily just do this on CI

[spacedmonkey]

We can update this now with 3.0.0 - https://www.npmjs.com/package/@wordpress/env 🎉

[spacedmonkey]

What is happening with this PR?

[swissspidy]

@spacedmonkey Thanks for asking! I think we're getting there, but I'm afraid we can only use it for the E2E tests for now. I was hoping we could use wp-env for more stuff from the beginning, but there are some upstream blockers that need to be fixed first.

Fo now, I just need to figure out why the E2E tests are failing, then this should be good to go.

PHP unit tests, PHP lints, and more, can then be addressed in future PRs.

[spacedmonkey]

but I'm afraid we can only use it for the E2E tests for now.

If we are just using it from E2E tests, that I am not sure I see the benefit of using it all. If anything, it makes it hard to replicate if there is an error in e2e, as the envs are even more different from local / GHA.

Can you provide more detail on these upstream blockers? Maybe we could work on these together to get the unblocked and so we can start using this package.

[swissspidy]

but I'm afraid we can only use it for the E2E tests for now.

If we are just using it from E2E tests, that I am not sure I see the benefit of using it all. If anything, it makes it hard to replicate if there is an error in e2e, as the envs are even more different from local / GHA.

To clarify, we'd be able to use it for local and E2E tests. But not for PHP unit tests or running any composer commands through docker

Can you provide more detail on these upstream blockers? Maybe we could work on these together to get the unblocked and so we can start using this package.

There's some discussion at WordPress/gutenberg#27783. In WordPress/gutenberg#27783 (comment) I posted a suggestion for a possible solution. Shouldn't be too difficult to implement, and maybe we could even patch it in our repository as well so that we don't have to wait for the fix to be published on npm.

[spacedmonkey]

So the issue seems to be running in PHP 5.6? Is that correct. As much as I don't love this idea, could we not just disable tests running in that version of PHP until we can patch this / get a fix merged?

[swissspidy]

So the issue seems to be running in PHP 5.6? Is that correct. As much as I don't love this idea, could we not just disable tests running in that version of PHP until we can patch this / get a fix merged?

It affects anything below 7.2, so 5.6, 7.0, and 7.1.

That said, I've now added a patch for the wp-env package to work around this

[spacedmonkey]

On npm i, seeing some error messages in the logs.

[swissspidy]

On npm i, seeing some error messages in the logs.

What's your node & npm version?

[spacedmonkey]

jonny@macbook-pro web-stories-wp (try/wp-env ✗)]$ npm --version
6.14.11
[jonny@macbook-pro web-stories-wp (try/wp-env ✗)]$ nvm --version
0.37.2
[jonny@macbook-pro web-stories-wp (try/wp-env ✗)]$

[spacedmonkey]

Unable to run phpunit tests locally.

Unable to upload files in editor.

[swissspidy]

jonny@macbook-pro web-stories-wp (try/wp-env ✗)]$ npm --version
6.14.11
[jonny@macbook-pro web-stories-wp (try/wp-env ✗)]$ nvm --version
0.37.2
[jonny@macbook-pro web-stories-wp (try/wp-env ✗)]$

And node --version?

[spacedmonkey]

Node - v14.15.4

[spacedmonkey]

Any chance we could just make these aliases? Means I don't have to change what I try and no learning curve for current devs on the project.

[swissspidy]

Yeah maybe we can keep them for some time.

[spacedmonkey]

So the new setup doesn't create users and import media. Doesn't this break the e2e tests. How does this work if author isn't created?

[swissspidy]

So the new setup doesn't create users and import media. Doesn't this break the e2e tests. How does this work if author isn't created?

It still does, but you need to run npm run test:prepare:e2e (see prepare-e2e-tests.sh).

Some reason I seem to be running an old version of web stories plugin when I start it up. Not sure how this happened

That... is strange.

Have you perhaps used wp-env in the past? Try npm run wp-env destroy to start fresh, and then npm run wp-env start -- --update to be safe

Which site are you on? http://localhost:8898 is the local development environment, http://localhost:8899 is the test environment

[spacedmonkey]

Unable to get the box up on my linux laptop.

[spacedmonkey]

Both the issues I have am having on linux and mac os are related to nodegit.

[swissspidy]

Hmm maybe I accidentally dropped the hotfix for nodegit. I‘ll check

[spacedmonkey]

Have you perhaps used wp-env in the past? Try npm run wp-env destroy to start fresh, and then npm run wp-env start -- --update to be safe

The issue was my git pull didn't work so I was using an old version of plugin. I deleted the branch and rechecked out. Seemed to fix the problem. It because they are force pushes in the history.

[spacedmonkey]

Worth noting, I started from a completely fresh install on my linux laptop.

[swissspidy]

@spacedmonkey So the two main remaining issues I see here are:

• Uploads not working (see wp-env: file permissions issues with rewrites and uploads WordPress/gutenberg#28201)
• Issues with nodegit (see Migrate wp-env to use simple-git instead of nodegit WordPress/gutenberg#26660)

Both are quite significant, and require upstream changes.

Thus, I feel like we should put this ticket back into the backlog and revisit later. Thoughts?