Consider file permissions when writing configuration in system tests (#…

…43466)

* Fix for issue #43465 writing configuration.php

Fix for issue #43465 'Cypress System Tests fail when writing configuration.php'
. remember the original file permission
. set 644
. write file
. restore original file permission

additional:
. writing file to ${Cypress.env('cmsPath')}/configuration.php` and no more to 'configuration.php'
. error handle file is not existing

* typo

* updated system tests README

* corrected task names

* Update tests/System/README.md

of course, thank you for checking

Co-authored-by: Richard Fath <richard67@users.noreply.github.com>

* deleted failure handler config_setParameter()

deleted failure handler for readFile as it is not needed, tested with chmod 0, Cypress fails with clear reason:

	CypressError: `cy.readFile("./configuration.php")` failed while trying to read the file at the following path:
	`.../43465/joomla-cms/configuration.php`
	The following error occurred:
	> "EACCES: permission denied, open '/Users/hlu/Desktop/no_backup/43465/joomla-cms/configuration.php'"

* typo

Co-authored-by: Brian Teeman <brian@teeman.net>

* typo

Co-authored-by: Brian Teeman <brian@teeman.net>

* chain the then()-calls

Chaining the then()-calls for a not so deeply nested code source
looks catchy - thank Allon for the recommendation

* adopted code formatting for better readability

* fixing lint:js errors

- deleted console.log statements
- used const for never changing value
- refactored file mask to not use bitwise operation '&'

* fixed lint:testjs errors

* Better fix for configuration.php permission issue

Working with the code when fighting with the drone shows that a `chmod`
was already implemented in `writeFile()`. Following changes with this commit:
- Only using `chmod` method synchronously
- Replaced setting directory mode to setting file mode before writing
- Setting file mode only if the file exists
- Having final file mode as parameter with default 0o444
- Using 0o444 as default file mode and not hard-wired 0o777
- The methods `getFilePermissions()` and `changeFilePermissions()` created for this PR earlier are deleted.

Enhancement of the `tests/System/README.md` for troubleshooting three-user-problem in having
Cypress running user, web server running user and `root` user.

This commitment has been extensively tested in various combinations. Every test contains:
- Checking error before
- Doing the patch
- Running installation twice and running overall test suite

Tests are:
- macOS 14.5 Sonoma, local with apache & Cypress same user, branch 4.4-dev
  - error before `> EACCES: permission denied, open './configuration.php'`
- Docker, one container with joomla and one container with Cypress, using `root` users inside containers
  - no error before, but `configuration.php` is 777
  - after the patch `configuration.php` is 444 inside container and shown 644 on host
  - tested four times, branches 4.4-dev, 5.1-dev, 5.2-dev and 6.0-dev
- Ubuntu 24.04 LTS local installation, one non-root users running Cypress and
  another non-root user running Apache, branch 4.4-dev
  - error before `> EACCES: permission denied, open './configuration.php'`
  - need to use `sudo` and need to set `umask 0`, see troubleshooting
- Windows 11 Pro, Laragon with Cmder, branch 4.4-dev
  - error before `> EPERM: operation not permitted, open 'C:\laragon\www\joomla-cms\configuration.php'`

All tests are successful:
- running `Installation.cy.js` twice, checking `configuration.php` 444 and params are set
- running complete system test suite without errors

* configuration.php CMS path relative && umask 0

- corrected mistake task writeFile was used with cmsPath + 'configuration.php'
- extended writeFile to set process umask 0
  - to prevent the 3-user-problem == no need to set umask 0 in sudo anymore

This commitment has been tested in various combinations. Every test contains:
- Checking error before
- Doing the patch
- Running Installation.cy.js only and running overall test suite

Tests are:
- Docker environment with drone images, root running Cypress and www-data running Apache, branch 4.4-dev
  - no error before, but /tests/www/cmysql/configuration.php has 777
- Ubuntu 24.04 LTS local installation, one non-root user running Cypress and
  another non-root user running Apache, branch 4.4-dev
  - error before `> EACCES: permission denied, open './configuration.php'`
  - need to use `sudo`, see troubleshooting (umask 0 is no more needed)
- Windows 11 Pro, Laragon with Cmder, branch 4.4-dev
  - error before `> EPERM: operation not permitted, open 'C:\laragon\www\joomla-cms\configuration.php'`
  - found out that on the second run cy.exec('rm configuration.php') does not work under Windows
    - deleted file manually and i will create an issue afterwards to avoid enlarging this one
- macOS 14.5 Sonoma, local with apache & Cypress same user, branch 4.4-dev
  - error before `> EACCES: permission denied, open './configuration.php'`

All tests are successful:
- running `Installation.cy.js`, checking `configuration.php` 444 and params are set
- running complete system test suite without errors

---------

Co-authored-by: Richard Fath <richard67@users.noreply.github.com>
Co-authored-by: Brian Teeman <brian@teeman.net>
Co-authored-by: Allon Moritz <allon.moritz@digital-peak.com>

tests/System/README.md

@@ -6,20 +6,39 @@ The CMS system tests are executed in real browsers and are using the [cypress.io
 A couple of steps are needed before the CMS system tests can be executed on the system.
 
 1. Clone Joomla into a folder where it can be served by a web server
+```
+git clone --depth 1 https://github.com/joomla/joomla-cms 
+```
 2. Install the PHP and Javascript dependencies by running the following commands:
-   1. `composer install`
-   2. `npm ci`
-3. Copy the cypress.config.dist.js to cypress.config.js in the root of the joomla folder
-4. Adjust the baseUrl in the cypress.config.js file, it should point to the Joomla base url
-5. Adapt the env variables in the file cypress.config.js, they should point to the site, user data and database environment
-6. In order to run the api tests you will need to change the value in your configuration.php for $secret to `tEstValue`
-7. Ensure the system has all the required dependencies according to the Cypress [documentation](https://docs.cypress.io/guides/getting-started/installing-cypress)
-8. Run the command `npm run cypress:install`
+```
+cd joomla-cms
+composer install
+npm ci
+```
+3. Copy the `cypress.config.dist.js` to `cypress.config.js` in the root of the joomla folder
+4. Adjust the `baseUrl` in the `cypress.config.js` file, it should point to the Joomla base URL
+5. Adapt the env variables in the file `cypress.config.js`, they should point to the site, user data and database environment
+6. Ensure the system has all the required dependencies according to the Cypress [documentation](https://docs.cypress.io/guides/getting-started/installing-cypress)
+7. Install Cypress
+```
+npm run cypress:install
+```
+8. Run Joomla installation with headless Cypress
+```
+npx cypress run --spec tests/System/integration/install/Installation.cy.js
+```
+:point_right: In the case of `EACCES` or `EPERM` error, see troubleshooting at the end.
 
 ## Run the existing tests
-Cypress has a nice gui which lists all the existing tests and is able to launch a browser where the tests are executed. To open the cypress gui, run the following command:
+You can use Cypress headless:
+```
+npx cypress run
+```
 
-`npm run cypress:open`
+And Cypress has a nice GUI which lists all the existing tests and is able to launch a browser where the tests are executed. To open the Cypress GUI, run the following command:
+```
+npx cypress open
+```
 
 ## Create new tests
 To Create new tests, create a cy.js file in a new folder which matches the following pattern (replace foo with the extension name to test):
@@ -42,10 +61,12 @@ Tests should be:
 
 The CMS tests come with some convenient [cypress tasks](https://docs.cypress.io/api/commands/task) which execute actions on the server in a node environment. That's why the `cy.` namespace is not available. The following tasks are available, served by the file tests/System/plugins/index.js:
 
-- **queryDB** Executes a query on the database
-- **cleanupDB** does some cleanup, is executed automatically after every test
+- **queryDB** executes a query on the database
+- **cleanupDB** deletes the inserted items from the database
 - **writeFile** writes a file relative to the CMS root folder
 - **deleteFolder** deletes a folder relative to the CMS root folder
+- **getFilePermissions** get file permissions
+- **changeFilePermissions** change file permissions
 
 With the following code in a test a task can be executed `cy.task('writeFile', { path: 'images/dummy.text', content: '1' })`. Each task is asynchronous and must be chained, so to get the result a `.then(() => {})` must follow when executing a task.
 
@@ -72,3 +93,34 @@ The API commands make API requests to the CMS API endpoint `/api/index.php/v1`.
 - **api_patch** add the path and content for the body as arguments
 - **api_delete** add the path as argument
 - **api_getBearerToken** returns the bearer token and no request object
+
+# Troubleshooting
+## Errors 'EACCES: permission denied' or 'EPERM: operation not permitted'
+
+If the Cypress installation step or the entire test suite is executed by a non-root user, the following error may occur:
+```
+1) Install Joomla
+       Install Joomla:
+       CypressError: `cy.task('writeFile')` failed with the following error:
+       > EACCES: permission denied, open './configuration.php'
+```
+Or on Microsoft Windows you will see:
+```
+       > EPERM: operation not permitted, open 'C:\laragon\www\joomla-cms\configuration.php'
+```
+
+The reason for this error is that Cypress first creates the Joomla file `configuration.php` via the web server.
+Subsequently, some of the parameters in this file are configured by Cypress under the current user.
+If the web server and Cypress are run by different users, this can lead to file access issues.
+
+You have to give the user running Cypress the right to write `configuration.php`
+e.g. with the command `sudo` on macOS, Linux or Windows WSL 2:
+```
+sudo npx cypress run
+```
+
+If the `root` user does not have a Cypress installation, you can use the Cypress installation cache of the current user:
+```
+sudo CYPRESS_CACHE_FOLDER=$HOME/.cache/Cypress npx cypress run
+```
+</details>

tests/System/plugins/fs.js

@@ -1,5 +1,6 @@
 const fs = require('fs');
 const fspath = require('path');
+const { umask } = require('node:process');
 
 /**
  * Deletes a folder with the given path recursive.
@@ -16,19 +17,36 @@ function deleteFolder(path, config) {
 }
 
 /**
- * Writes the given content to a file for the given path.
+ * Writes the given content to the file with the given path relative to the CMS root folder.
  *
- * @param {string} path The path
- * @param {mixed} content The content
- * @param {object} config The config
+ * If directory entries from the path do not exist, they are created recursively with the file mask 0o777.
+ * If the file already exists, it will be overwritten.
+ * Finally, the given file mode or the default 0o444 is set for the given file.
+ *
+ * @param {string} path The relative file path (e.g. 'images/test-dir/override.jpg')
+ * @param {mixed} content The file content
+ * @param {object} config The Cypress configuration
+ * @param {number} [mode=0o444] The file mode to be used (in octal)
  *
  * @returns null
  */
-function writeFile(path, content, config) {
-  fs.mkdirSync(fspath.dirname(`${config.env.cmsPath}/${path}`), { recursive: true, mode: 0o777 });
-  fs.chmod(fspath.dirname(`${config.env.cmsPath}/${path}`), 0o777);
-  fs.writeFileSync(`${config.env.cmsPath}/${path}`, content);
-  fs.chmod(`${config.env.cmsPath}/${path}`, 0o777);
+function writeFile(path, content, config, mode = 0o444) {
+  const fullPath = fspath.join(config.env.cmsPath, path);
+  // Prologue: Reset process file mode creation mask to ensure the umask value is not subtracted
+  const oldmask = umask(0);
+  // Create missing parent directories with 'rwxrwxrwx'
+  fs.mkdirSync(fspath.dirname(fullPath), { recursive: true, mode: 0o777 });
+  // Check if the file exists
+  if (fs.existsSync(fullPath)) {
+    // Set 'rw-rw-rw-' to be able to overwrite the file
+    fs.chmodSync(fullPath, 0o666);
+  }
+  // Write or overwrite the file on relative path with given content
+  fs.writeFileSync(fullPath, content);
+  // Finally set given file mode or default 'r--r--r--'
+  fs.chmodSync(fullPath, mode);
+  // Epilogue: Restore process file mode creation mask
+  umask(oldmask);
 
   return null;
 }

tests/System/plugins/index.js

@@ -14,7 +14,7 @@ function setupPlugins(on, config) {
   on('task', {
     queryDB: (query) => db.queryTestDB(query, config),
     cleanupDB: () => db.deleteInsertedItems(config),
-    writeFile: ({ path, content }) => fs.writeFile(path, content, config),
+    writeFile: ({ path, content, mode }) => fs.writeFile(path, content, config, mode),
     deleteFolder: (path) => fs.deleteFolder(path, config),
     getMails: () => mail.getMails(),
     clearEmails: () => mail.clearEmails(),

tests/System/support/commands/config.js

@@ -1,18 +1,17 @@
 Cypress.Commands.add('config_setParameter', (parameter, value) => {
-  cy.readFile(`${Cypress.env('cmsPath')}/configuration.php`).then((fileContent) => {
+  const configPath = `${Cypress.env('cmsPath')}/configuration.php`;
+
+  cy.readFile(configPath).then((fileContent) => {
     // Setup the new value
-    let newValue = value;
-    if (typeof value === 'string') {
-      newValue = `'${value}'`;
-    }
+    const newValue = typeof value === 'string' ? `'${value}'` : value;
 
     // The regex to find the line of the parameter
     const regex = new RegExp(`^.*\\$${parameter}\\s.*$`, 'mg');
 
     // Replace the whole line with the new value
     const content = fileContent.replace(regex, `public $${parameter} = ${newValue};`);
 
-    // Write the modified content back to the configuration file
+    // Write the modified content back to the configuration file relative to the CMS root folder
     cy.task('writeFile', { path: 'configuration.php', content });
   });
 });