Permission issues with static generated files in pub directory

[miguelbalparda]

We have been experiencing issues with permissions using the latest version of Magento 2.

It seems that it all comes from 'vendor/magento/framework/Filesystem/DriverInterface.php' which explicitly sets the umask for folders to something that isn't going to work on a lot of setups, including our own.

Is there any reason for not setting the default for directories to 0755 instead of 0770?

Our issues were mostly 403 errors inside the pub folder for assets and templates and we suspect a lot of hosting companies might face this issue too.

[mazhalai]

Hello @miguelbalparda, did you create a group as recommended? http://devdocs.magento.com/guides/v2.0/install-gde/install/file-system-perms.html

[miguelbalparda]

@mazhalai yes, we tried that. Please keep in mind this is not me as an individual but a hosting company. It also looks like we are not the only one facing this issue https://twitter.com/mbalparda/status/667029022099046400
Is there any reason in particular to use 0755 instead of 0770?

[piotrekkaminski]

@miguelbalparda can you provide more details on the setup and why you think it fails? Are the newly generated files not getting proper permissions?

[vancoz]

@miguelbalparda we changed it from 755 to 770 because we would like to close access to anybody except file owner and group. Php files can contain configuration settings, keys, etc... so it is better to keep it safe.

Please follow the instruction to set correct ownership setting http://devdocs.magento.com/guides/v2.0/install-gde/prereq/apache-user.html#install-update-depend-user-group

Thank you

[buskamuza]

Just to clarify: we recommend 0770.
The reason:

• we want the owner to have write permissions. The owner user should be used for CLI commands/operations on the application. It may include necessity to modify files
• we want the group to have write permissions to some folders. A user in the group would be the web-server user, so Web Magento application could modify necessary files (e.g., cache)

[miguelbalparda]

Adding a bunch of users to the apache group is less than ideal, as it may cause security concerns on shared platforms and in some cases it is even impossible to implement. We will investigate more about adding apache to the users group since it might be the solution.

[robfico]

The issue we see is in shared hosting where groups and ownerships cannot easily be set up in the desired fashion. With php-fpm, PHP runs as the owner. Apache runs as another less privileges user (i.e. nobody, apache, etc...). Apache can not "out of the box" pull css, js, and images (essentially static resources) from pub/static/xxx. php-fpm can read any file the owner has read permission on, so no issues with the php files themselves not being world readable.

If we add the user to the nobody group, or nobody to their group, it does work, but it alters the standard way for shared hosting, and may open up security issues. As Miguel mentioned, we added the apache group to the user's group, and that seems to work. However, this could lead to other issues if a perl/cgi based script were to be used.

[miguelbalparda]

+1 for the setup @robfico described.

[piotrekkaminski]

so @miguelbalparda @robfico would it be the right solution for you to have only the generated files (js/css/images) world readable (given they are already being sent outside, not a security issue)? and the code (including generated) could stay as is?

[miguelbalparda]

@piotrekkaminski yes, that seems to be a solid solution. So far we haven't seen a single issue with files/folders outside the pub folder.

[robfico]

Yes, I think that would work well. Since those files are publicly viewable anyway on the web, they should be world readable and directories world executable permissions wise to match. This way Apache is unrestricted in its static content access, while php-fpm can be used to manage access to mroe restricted files (xml, php, etc...). Seems like a win-win, and will avoid issues with other standard hosting in the shared world.

[lngla]

After installing Magento 2 with data and running the content deploy, all permissions on those files were created at 770 & 660. None of the content was viewable for me, including the css and product images. It took me awhile, but once I figured out to change permissions to 755 and 644, my css and images were loaded and visible. I am continuing to have this issue- every time I add a new product image permissions are pre-loaded to 770 and newly cached images both appear broken.

I would love to see the solution talked about here.

[miguelbalparda]

We started experiencing errors in the vendor dir too.

[20-Nov-2015 14:22:42 America/Detroit] PHP Fatal error: require(): Failed opening required '/chroothtml/vendor/magento/module-media-storage/registration.php' in html/vendor/composer/autoload_real.php on line 59

[CalamityJames]

We've been having this issue with our dedi server running shared hosting too, I understand the reasons for changing to 0770 and 0660 but in a WHM/cPanel environment where everything is pretty much locked down by suPHP etc, it will refuse to load because it can't access any of the .htaccess files

Here's an excerpt from the error log:

[Wed Dec 09 11:25:33.153817 2015] [core:crit] [pid 3011] (13)Permission denied: [client ***.***.***.***:5769] AH00529: /home/*******/public_html/pub/static/frontend/*********/***********/en_GB/Magento_Checkout/.htaccess pcfg_openfile: unable to check htaccess file, ensure it is readable and that '/home/*******/public_html/pub/static/frontend/********/**********/en_GB/Magento_Checkout/' is executable, referer: http://************/

Having to run

find . -type f -exec chmod 644 {} \;
find . -type d -exec chmod 755 {} \;

on pub/static every time is getting tiresome!

Our environment:

OS:      CentOS 6.7
WHM:     11.52.1
PHP:     5.5.30
Handler: suphp
Apache:  2.4.16
suEXEC:  On

[CalamityJames]

The permissions are set as constants in lib/internal/Magento/Framework/Filesystem/DriverInterface.php.

Just sayin', if anyone wanted to maybe manually edit them on their install just to not have to reset the permissions until such time as there's a neat way around this. It does seem to fix the problem on my install.

NB: I know editing core files is very bad practice, but as far as I'm aware there's no way to redefine a const in PHP. For development sites, this is probably OK. Production, I wouldn't if I were you

[miguelbalparda]

@piotrekkaminski @mazhalai any news on this issue?

[Vinai]

There is some documentation on this issue in the devdocs.
I've found the section on setting the setgid bit has been useful for some people encountering permission issues.

[miguelbalparda]

Some more info about this issue via @ryebell. It seems symlinks are being used for some files inside pub folder (JS in our case) and a chmod/chown -R is not enough to change its permissions.

[user@server html]$ stat pub/static/frontend/Magento/luma/en_US/images/logo.svg
  File: `pub/static/frontend/Magento/luma/en_US/images/logo.svg' -> `/chroot/home/magento1/magento2-demo.nexcess.net/html/vendor/magento/theme-frontend-luma/web/images/logo.svg'
  Size: 107         Blocks: 8          IO Block: 4096   symbolic link

[user@server html]$ stat pub/static/frontend/Magento/luma/en_US/mage/requirejs/mixins.js
  File: `pub/static/frontend/Magento/luma/en_US/mage/requirejs/mixins.js' -> `/chroot/home/magento1/magento2-demo.nexcess.net/html/lib/web/mage/requirejs/mixins.js'
  Size: 85          Blocks: 8          IO Block: 4096   symbolic link

Wasn't the use of symlink highly discouraged in Magento 1?

[markshust]

Same issue as #2449 -- pub directory is getting perm denied errors. I'm running Debian Jessie.

I'm trying a fresh install on a variation of my docker image which is a really clean operating system snapshot. I did manage to get this working by:

sed -i 's/0770/0775/g' vendor/magento/framework/Filesystem/DriverInterface.php
sed -i 's/0660/0664/g' vendor/magento/framework/Filesystem/DriverInterface.php
find pub -type f -exec chmod 664 {} \;
find pub -type d -exec chmod 775 {} \;

Note that I didn't run the suggested find . -type d -exec chmod 770 {} \; && find . -type f -exec chmod 660 {} \; && chmod u+x bin/magento command after the magento install, and just used default perms as they come (default is 755/644 on nix). Without setting direct perms of 775/664 on pub dir, nothing worked right.

I did setup a new magento user that belongs to the www-data group. The setgid recommendation had no effect for me.

This does seem to be an issue, as the recommended install settings don't seem to be working on a clean os image.

[pantaoran]

hmm @markoshust that file for me is located at lib/internal/Magento/Framework/Filesystem/DriverInterface.php
but I'm using 2.0.0 and may not have the latest changes.

[markshust]

@pantaoran I'm installing with composer create new project. This may be why.

This may be related issue that could possible by merged in #2525

Please see my setup script here https://github.com/mageinferno/docker-magento2-setup/blob/master/m2setup.sh#L42

These lines were required to get things working on Ubuntu Jessie.

Update: I did need to setgid on var/generation after all with the recommended user/group setup/config, otherwise web user could not write to that folder.

[wbyrnetx]

It's not an official Magento patch, I just took that commit you pointed out and adapted it for Magento 2.0.2 downloaded from Magento.com. Assuming that commit makes it into a final release version, then I guess it would become official at that point.

[tdgroot]

@wbyrnetx kind of bad practice, people would blindly install this and wonder why their shop crashes when this commit gets released.. Can you please remove your comment?

[wbyrnetx]

It was meant to just help people who know what they're doing and would know to revert it when they either upgrade or apply the Magento patch that has it. But sure, it's removed.

[mehh]

Any update on this? Having to flush cache and run scripts every single time an update is made in development is becoming a bit of a pain.

[comprensys]

This fix is taking far too long!

[magedmik]

+1

[sammarcus]

I'm glad I found this thread —being new to the platform I just assumed I was the only one struggling with this. Hassling with these ongoing musical chair permissions has single handedly been the biggest headache with my development in M2.

I have never experienced so many errors that can all be attributed back to insufficient permissions. Someone above recommend setting up a cron job to regularly reset these.... 😐. Absolute madness. It feels like an uphill battle constantly working against the platform every time it aggressively changes permissions behind your back without much transparency or rationale.

A fix for this should be prioritized.

[joeNmass]

I totally believe that Magento is the best shopping cart out there for the well experienced user and I totally agree with everyone on here that is dissatisfied with the permissions issues on Magento 2.0.#.
Because of issues like this (and others) I stopped recommending Magento to my clients because of the hurles that the shop owner will have to jump through each time they add new products. I don't want to be constantly contacted by clients each time they add products and tell me to fix their "broken" online store. Until Magento gets their act together I guess I will keep pushing other more stable shopping carts such as Prestashop.

[markshust]

fyi, i just wrote a blog post on this entire topic. pretty much for now, if you stay away from the magento user, and keep everything running by & owned by www-data, everything is good. you can't use the magento user as it stands today with application-level security impediment in place. https://mageinferno.com/blog/magento-2-speed-security-permissions-docker

[crantron]

I installed Magento with composer and the DriverInterface.php is located at /vendor/magento/framework/Filesystem, then change perms to 755 for both dir and file.

• godspeed.

[piotrekkaminski]

@markoshust what user you use for command line actions (like running bin/magento)? this has the potential to break stuff.

[markshust]

@piotrekkaminski during container start, i execute this script, which moves ./bin/magento to ./bin/magento-php (ref: https://github.com/mageinferno/docker-magento2-php/blob/master/bin/setup-config). i then mount a special file (ref: https://github.com/mageinferno/docker-magento2-php/blob/master/bin/magento) that basically chroots ./bin/magento-php to the www-user user. i'm doing a lot of security moshposh (this is required in docker, specifically within development environments) to make sure just about everything, everywhere runs as www-user.

[piotrekkaminski]

Thanks @markoshust
Btw in your setup-config file I would suggest making sure the cron processes have output redirected to a log filed. I think not all of them write to real log files properly, some just output to STDOUT

[markshust]

@piotrekkaminski good catch! all of the default log output to STDOUT for nginx is handled from the base image, i forgot about the cronjob output. will do.

[pen65978]

Fixed!!!!!!!!

man this was a tough one, i'm running CentOS 7 with Cpanel/WHM the fix was to use EasyApache 4 to disable SUphp... that's it... done and fixed :)

[miguelbalparda]

Starting with Magento 2.0.6, Magento provides a more flexible way for you to set file ownership and permissions. Instead of setting permissions explicitly, you only need to make sure files and directories are writable for installation. We provide different suggestions for doing this, depending on whether you access your Magento server with one user account (typical of shared hosting) or two user accounts (typical of private hosting or having your own server). After installation, to further restrict access to files and directories, you can optionally create a file named magento_umask in your Magento root directory. By default, the umask is 002, which means that directories have 775 permissions and files have 664 permissions. For more details, see Magento file system ownership and permissions.

http://devdocs.magento.com/guides/v2.0/release-notes/ReleaseNotes2.0.6CE.html
http://devdocs.magento.com/guides/v2.0/install-gde/install/post-install-umask.html

[ryantfowler]

@miguelbalparda can you please close this issue once you've downloaded 2.0.6 and performed your testing to verify that the new code supports your use case?

Thanks

[elenleonova]

All - we have addressed this issue as a part of 2.0.6. If you have some comments, please feel free to reopen this issue or open a new one.

[miguelbalparda]

Thank you!

[swapnilagarwwal]

@elenleonova I have 2.0.6, but still facing the same issue.

[dsikkema-magento]

@agarwalswapnil are your webserver PHP user and your CLI php user the same or different? And have you set the permissions recommended in http://devdocs.magento.com/guides/v2.0/config-guide/prod/prod_file-sys-perms.html ?

[devamitbera]

For share hosting it difficult to change user.Then what will be good idea. For a case, i have change file permission from Magento\Framework\Exception\FileSystemException\DriverInterface i have change WRITEABLE_DIRECTORY_MODE = 0770; toWRITEABLE_DIRECTORY_MODE = 0775;andWRITEABLE_FILE_MODE = 0660 to WRITEABLE_FILE_MODE = 0664.It is working for me.

Are u thought that is identical process for share hosting ?please put yours views

[dsikkema-magento]

@devamitbera With the 2.1 release candidate, and with 2.0.6+, if you need permissions for generated assets of 664 and 775, you can use "magento_umask" as described here.

[nmallepally]

@romeof1980 Thanks for the detailed information.
Yes dedicated for now the memory it takes is 2048MB, .Increasing to 4G will create any bad script damage the website more, will that have any effect?.
But global php.ini , it is 1G and we can override using -d memory for php scripts
I have my file permissions set correctly const WRITEABLE_DIRECTORY_MODE = 0755;
const WRITEABLE_FILE_MODE = 0644; What i found is that when i minify css it breaks down and i cannot minify js. (attached js error image)

Its so strange when i keep minify options to yes, the website does not show up at all.
I dont see any 404 errors but it shows some JS warnings- Is this normal to have on website?
One more thing is inside admin-> Marketing

-> Email templates -> It does not load order templates for editing them? .

[romeof1980]

hi there @nmallepally ,
I don't see the point in increasing to 4GB. I've been working smoothly with many Magento 2 installation in shared environment with 768M. Let's focus on the permission issue:
first of all you can verify that you've set the permission correctly as @dsikkema suggested:
http://devdocs.magento.com/guides/v2.0/config-guide/prod/prod_file-sys-perms.html

Please, be careful to tackle 1 issue at a time (in the right thread) and not to go off-topic: indeed the admin->marketing issue is probably off-topic, (try to google around after you've solved the issue with the static files if the problem persists)

[nmallepally]

hi @romeof1980 Thanks for coming back.
Yes the permissions are set correctly for production mode. after Debugging more, what i found was the website breaks down when i put minify css to yes and nothing works on website when i keep minify JS to Yes. On Admin Side, there is 500 internal server error on Stores->configuration->Advanced->system is blank. It was working fine on Dev mode but prod it does not show up.

[romeof1980]

hi @nmallepally , if you're certain owner and permissions are set correctly: did you try to redeploy static content? if I remember correctly you mentioned you're working with multilanguage (hence at least with 2 store view): verify that the static content has been deployed correctly for both and if in doubt delete the folders that contain the static content and redeploy it.
Details:
see here in order to understand which folders you should delete:
http://magento.stackexchange.com/questions/94502/magento2-re-generate-css-file-in-pub-static-folder
Then proceed deploying static content for system language:
php bin/magento setup:static-content:deploy
and finally don't forget to deploy the static content for the French language:
php bin/magento setup:static-content:deploy fr_FR (in my opinion this is what caused the error in the first screenshot you attached:).

I don't see any bug here but just incorrect procedure when deploying the static content

[giappv]

I used apache as webserver and this cron setting fixed my issue:

* * * * * www-data /usr/bin/php /path to magento2/bin/magento cron:run | grep -v "Ran jobs by schedule" >> /path to magento2/var/log/magento.cron.log
* * * * * www-data /usr/bin/php /path to magento2/update/cron.php >> /path to magento2/var/log/update.cron.log
* * * * * www-data /usr/bin/php /path to magento2/bin/magento setup:cron:run >> /path to magento2/var/log/setup.cron.log