wp-env: Unable to upload plugins on Linux & Windows, error on media upload.

[ckanitz]

Description

When I wp-env start my dev environment and try to upload a plugin (zip) I'll get an "FTP Form" instead of an successful install.
If I have a look at the home/.wp-env/<container>/wp-contents/uploads folder at least the plugin.zip exists, so the upload is done anyway but the installation is not.

I also get media could not be copied error if I map wp-content to my project-folder. This error disappears if I remove the mapping but the above described error still exists.

Occured on:

• windows 10
• Manjaro Linux 21.3.7

Works fine on:

• MacOS

I installed docker on windows via docker-desktop setup.
On linux I followed this guide, incl. the usermod to "use docker without root": https://linuxconfig.org/manjaro-linux-docker-installation

Step-by-step reproduction instructions

1. Take the .wp-env.json from below
2. wp-env start it on windows or linux
3. try to upload & install a zipped plugin

Screenshots, screen recording, code snippet

My .wp-env.json with both errors:

{
	"themes": [
		"./theme"
	],
	"plugins": [
		"./plugin"
	],
	"mappings": {
		"php.ini": "./bin/php.ini",
		"wp-content": "./wp/"
	},
	"env": {
		"development": {
			"port": 80,
			"config": {
				"WP_DEBUG": true,
				"WP_ENVIRONMENT_TYPE": "development",
				"LH_CURRENTLY_EDITING": "lhpbp"
			}
		}
	}
}

My .wp-env.json with only the plugin installation from zip not working:

{
	"themes": [
		"./theme"
	],
	"plugins": [
		"./plugin"
	],
	"mappings": {
		"php.ini": "./bin/php.ini"
	},
	"env": {
		"development": {
			"port": 80,
			"config": {
				"WP_DEBUG": true,
				"WP_ENVIRONMENT_TYPE": "development",
				"LH_CURRENTLY_EDITING": "lhpbp"
			}
		}
	}
}

The /bin/php.ini:

post_max_size = 512M
upload_max_filesize = 512M
memory_limit = 512M

Environment info

• WordPress 6.1
• wp-env 5.6.0
• OS
• • windows 10
• • manjaro linux 21.3.7

Please confirm that you have searched existing issues in the repo.

Yes

Please confirm that you have tested with all plugins deactivated except Gutenberg.

Yes

[ckanitz]

On some further investigation:

• when I add "FS_METHOD": "direct" to env.development.config the FTP form disappears and I'm left with "can't create directory" error
• when I ls -al in .wp-env/<container>/WordPress I can see three odd things:
• • a php.ini owned by root:root (instead of <username>:<username> like the other files + dirs)
• • • leads to error on wp-env start -> "Couldn't write ...WordPress/php.ini"
• • wp-config.php is owned by http:http (instead of...)
• • • leads to same error as on php.ini
• • Permissions on some files are 644 that should be at least 664 Site Health Tool shows me a bunch of "unwritable files" so I simply chmod 777 the whole .wp-env/<container> and uploads works fine (shouldn't be 777 on everything, but it's local testing environment...)

I also tried a "vanilla" approach without a .wp-env.json:

• mkdir test && cd test
• npm init -> enter, enter, enter...
• npm i @wordpress/env
• npx wp-env start

Still same error on uploading plugins

^- all done on linux, so no info on windows for this, sorry

[github-actions]

Hi,
This issue has gone 30 days without any activity. This means it is time for a check-in to make sure it is still relevant. If you are still experiencing this issue with the latest versions, you can help the project by responding to confirm the problem and by providing any updated reproduction steps.
Thanks for helping out.

[eliot-akira]

I have the same issue, unable to upload plugins on Linux. I think this is related to the following note in the documentation: https://github.com/WordPress/gutenberg/tree/trunk/packages/env#installing-a-plugin-or-theme-on-the-development-instance

Installing a plugin or theme on the development instance

Depending on your host OS, you may experience errors when trying to install plugins or themes (e.g. Warning: Could not create directory.). This is typically because the user ID used within the container does not have write access to the mounted directories created by wp-env. To resolve this, run the docker-compose command directly from the directory created by wp-env and add -u $(id -u) and -e HOME=/tmp the run command as options:

$ cd ~/wp-env/500cd328b649d63e882d5c4695871d04
$ docker-compose run --rm -u $(id -u) -e HOME=/tmp cli [plugin|theme] install <plugin|theme>

The last command seems to be missing wp after cli. This worked for me:

docker-compose run --rm -u $(id -u) -e HOME=/tmp cli wp plugin install example-plugin

---

Edit: Hm, I see this permissions question is a deep rabbit hole. According to #42867:

The issue is that the UID in the host should match the UID in the container to avoid permission issues, so if I spin up the docker container with UID 1001 using that parameter, any filesystem operations that happen inside Docker will happen under this UID, which is perfect for the host operating system, but since the files inside the container are owned by www-data (1000), then we don't have write access.

Also see: Handle folder permissions in wp-env #22515.

[ndiego]

As this issue appears confirmed, I have removed the "Needs Testing" label. @t-hamano if you have any time to take a peek at this one, it would be most appreciated 🙏

[t-hamano]

@ckanitz

Thanks for the report.

I'm a Windows user and have often had that problem. When I try to install themes, plugins, update core, etc., I get a message prompting me to enter my FTP connection information:

As far as I know, we have the following problems on Windows:

When I try to install themes, plugins, update core, etc., I get a message prompting me to enter FTP connection information.

Unable to switch languages.

After a little deeper investigation, I found that this problem occurs when @wordpress/env switches from version 5.0.0 to 5.1.0. Additionally, starting with 5.1.0, the page appears to take much longer to display.

On Windows, the WordPress directory is created in C:\Users\USERNAME\.wp-env. Here, comparing the directories created in version 5.0.0 and version 5.1.0, we can see that version 5.0.0 has no core files of WordPress itself.

Starting with version 5.1.0, WordPress is downloaded from the GitHub repository instead of WordPress provided by Docker. I suspect that this change might have caused some differences in permissions that are causing this issue.

@noahshrader
If you can shed any light on this issue, please let me know 🙏

[dgwyer]

@t-hamano I'm seeing the exact same thing on Windows.

[talldan]

This is the PR - #42826

Pinging @noahtallen and @fullofcaffeine who collaborated on it.

[noahtallen]

The permissions issue is indeed a deep rabbit hole... I suspect that using local files for WordPress is making those permissions issues happen more frequently?

Before that wp-env version, the default WordPress PHP files were included in the docker image itself (the one officially published by Docker), which is why nothing is on the local filesystem. You can see the motivation for changing that behavior in #42826

[eliot-akira]

The most recent and optimistic movement that I found about this issue is the following comment by @defunctl in the unmerged pull request, wp-env runs docker commands with same UID as host to prevent filesystem permission issues.

Basically every Linux user (and I assume WSL2 Windows users) will continue to have improper permissions in the docker container whenever the web server user writes a file or creates a directory or basically executes anything that touches the file system under the www-data user.

I'd be happy to submit a PR that would fully address this and would work on Linux/Mac/Windows. In addition to the changes this PR would address, we could also:

• Detect the user's UID/GID and pass those to an updated docker-compose.yml user: definition in addition to setting the proper Apache user/group environment variables..
• Update the Dockerfile to run as the current user, or as @Luc45 mentioned, fixuid is a great solution, which would be simple to add to the generated Dockerfile.

This way, all the files are owned by same UID/GID as the host user, allowing the user to create new files on the host without permission issues and any filesystem operations run inside the container would be executed as their UID, keeping newly created files/folders editable under the host user, basically ensuring all files inside and outside of the container are owned by the same user.

The pull request linked above is still an incomplete solution, as it only sets the user and group IDs for the wp-env run command. What @defunctl is describing goes further to get to the bottom of the issue, to entirely resolve the user ID mismatch between the host and container file systems.

---

If successful, this would solve a number of related issues such as:

• Endpoints accessed through /wp-json when using wp-env return Not Found #49373
• wp-env: file permissions issues with rewrites and uploads #28201
• Running wp amp docs generate fails with wp-env ampproject/amp-wp#7501
• Handle folder permissions in wp-env #22515

..And all the places that are currently using chmod -R 767 like the examples listed in #22515#issuecomment-658468153, or as described in: https://github.com/10up/Open-Source-Best-Practices/blob/gh-pages/_includes/markdown/Testing.md#fixing-permalinks-issue

File permissions could be fixed with npm run wp-env run tests-wordpress "chmod -c ugo+w /var/www/html" during the initialization.

The chmod workaround is not practical for mounted folders, as it would modify file permissions on the host file system.

---

The solution might have been easier, if not for this unresolved issue (since 9 years ago) in Docker itself: Make uid & gid configurable for shared volumes #7198.

Thanks to this limitation, there are various hacks and workarounds to attempt to solve the permission issue, as can be seen in: https://stackoverflow.com/questions/23544282/what-is-the-best-way-to-manage-permissions-for-docker-shared-volumes/28596874

fixuid does seem to be a key part of the answer.

fixuid is a Go binary that changes a Docker container's user/group and file permissions that were set at build time to the UID/GID that the container was started with at runtime. Primary use case is in development Docker containers when working with host mounted volumes.

[westonruter]

I just noticed this is also a problem for WordPress core's built-in Docker development environment. Whatever fix is put in place for the Gutenberg repo should be applied to WordPress core as well.

[noahtallen]

This should be solved by #49962. You'll be able to see the change it when developing Gutenberg directly (since npx wp-env or npm run wp-env execute the source code directly in this repo). The fix should be available in about 10 days on npm. You may need to destroy the environment first so that it can be re-created with the correct permissions. Of course, if you still experience issues, drop a comment!