Add page localization

[mtrezza]

New Pull Request Checklist

• I am not disclosing a vulnerability.
• I am creating this PR in reference to an issue.

Issue Description

Allows to localize the Parse Server pages by handling an optional user locale in the query of the public API routes, such as:

• request_password_reset
• choose_password
• verify_email
• resend_verification_email

Related issue: closes #7127

Approach

• Adds a new PagesRouter.js as experimental feature (disabled by default) to replace PublicAPIRouter.js
• Page localzation is achieved by matching a request-supplied locale parameter
• Pages can be localized in 2 ways with each having its own pros and cons (see README.md for details):

  • Subdirectory structure:

    root/
    ├── base/                    // base path to files
    │   ├── example.html         // default file
    │   └── de/                  // de language folder
    │   │   └── example.html     // de localized file
    │   └── de-AT/               // de-AT locale folder
    │   │   └── example.html     // de-AT localized file
    
    // Files are matched with the locale in the following order:
    // 1. Locale match, e.g. locale `de-AT` matches file in folder `de-AT`.
    // 2. Language match, e.g. locale `de-CH` matches file in folder `de`.
    // 3. Default; file in base folder is returned.

  • JSON resource:

    root/
    ├── public/                  // pages base path
    │   ├── example.html         // the page containg placeholders
    ├── private/                 // folder outside of public scope
    │   └── translations.json    // JSON resource file

    Pages are localized by adding placeholders in the HTML files and providing a JSON resource that contains the translations to fill into the placeholders.

    Example JSON Content:

    {
      "en": {               // resource for language `en` (English)
        "translation": {
          "greeting": "Hello!"
        }
      },
      "de": {               // resource for language `de` (German)
        "translation": {
          "greeting": "Hallo!"
        }
      }
      "de-AT": {            // resource for locale `de-AT` (Austrian German)
        "translation": {
          "greeting": "Servus!"
        }
      }
    }

• Localization is only enabled for the default pages in the public directory; localization is disabled if customPages are set (even if the custom pages point to the default pages).

Changes

• Introduces new Parse Server options to configure the PageRouter:

  new ParseServer({
    ...,
    pages: {
      enableRouter: true, // Enables the PageRouter by replacing the PublicAPIRouter; this is the general switch to turn on the experimental feature; default: false
      enableLocalization: true, // Enables looking for locale matching page version on request; default: false
      localizationJsonPath: './private/localization.json', // The path to the JSON file for localization; the translations will be used to fill template placeholders according to the locale.
      localizationFallbackLocale: 'en', // The fallback locale for localization if no matching translation is provided for the given locale. This is only relevant when providing translation resources via JSON file.
      placeholders: { aKey: 'aValue' }, // The placeholder keys and values which will be filled in pages; this can be a simple object or a callback function.
      forceRedirect: false, // Enables always sending redirect responses and never sending content directly; default: false
      pagesPath: './pages/', // A custom path to the page files; also sets where the '/apps' endpoint points to; default: './public'.
      pagesEndpoint: 'pages', // A custom endpoint to the pages; also sets the '/apps' endpoint for password reset and email verification; default: 'apps'.
    }
  })

• Added improved page templates (no styling, simply structure, fixed syntax) to demonstrate the feature.
  • Added mustache library to fill in placeholders in templates.
  • Improved naming to make them more descriptive.
• By default, page content is returned directly instead of redirect response to reduce requests.
• Prioritizes use of placeholders in templates instead of query parameters and complex page scripts for dynamic content in pages.
• Introduced new response headers x-parse-page-param-... to serve the template placeholders additionally in response headers for programmatic use of response.
• Introduced new pages invalid password reset link and invalid email verification link instead of generic page invalid link to allow for more specific user instructions.

Breaking changes (when enabling the experimental feature)

• public directory renamed
  • Change: The public directory that contains the pages HTML files is being renamed from public_html to public.
  • Reason: The public folder may not only serve HTML files, so the naming is more appropriate.
  • Developer task: Rename directory.
• views directory merged with public directory
  • Change: The views directory that contained the pages HTML files that are rendered as response have moved to the public directory.
  • Reason: All pages are now served from the public directory, as these files can be served directly or via redirect; removing the hard differentiation by directory allows for more flexibility.
  • Developer task: Move files from views to public directory; mind renaming the files if necessary, see the example template files names in the public directory.
• Parse Server App ID URL parameter renamed
  • Change: The app ID query parameter passed in route apps/choosePassword has been renamed from id to appId.
  • Reason: Did not conform with the parameter name in all other routes where it is called appId.
  • Developer task: Adapt query parameter in JavaScript script in choose_password.html template and adapt any hard coded links to the password reset page.
• HTTP response code changed
  • Change: The HTTP response code to redirect a POST request on the PublicAPIRouter has been changed from HTTP response code 302 to 303.
  • Reason: As per standard, 303 response code should be used for this purpose (PRG pattern).
  • Developer task: If necessary adapt firewall settings; be aware of possible metrics changes in traffic monitoring.
• Query parameters replaced by placeholders
  • Change: What has previously been provided by query parameters to fill into the website using scripts can now be filled in using content placeholders and is provided in response headers
  • Reason: Information such as email or username can contain PII and thereshould should not be in the URL, logs, etc.
  • Developer task: Adapt HTML templates if necessary and replace scripts with placeholders. For programmatic use, all placeholders that are filled into the response content are additionally served in the response headers.
• Page invalid_link.html has been split
  • Change: invalid_link.html has been replaced and split into the more specific password_reset_link_invalid.html and email_verification_link_invalid.html.
  • Reason: When a user opened an invalid password reset link or an invalid verify email link, a common template invalid_link.html for different use cases was shown; using different templates allows for more specific user instructions on how to resolve the issue.
  • Developer task: Adapt HTML templates and links.
• HTML files have been renamed
  • Change: All template files have been renamed
  • Reason: Template files had different naming syntax and were sometimes less descriptive regarding their purpose; the new naming is more descriptive and their file names group them by purpose in alphabetic sorting.
  • Developer task: Adapt template file names.

Unrelated changes (as side effect)

• Updated and cleaned up ToC in README.md, especially headlines
• Fixed bug in PromiseRouter where headers were not set when sending a text response
• Fixed bug in buildConfigDefinitions where Object type was not recognized as config key type

Caveats

• For directory based localization, each request that serves content (instead of redirect) causes up to 3 file look-ups for localization, so this may not be optimal for high frequency use (e.g. in case of a dedicated parse server that only sends out mail in high volume); can be improved in the future by simply caching the directory structure.
• For JSON based localization, the whole resource is loaded from file on server start and held in memory. This may not be a suitable approach for very large translations resources; can be improved in the future by caching the most frequently used languages on start and dynamically loading other languages on demand.
• This router works different than the existing router, therefore tests in the following files would fail and require minor adaptation when removing the feature flag and replacing the existing router with this new router in the future. However, since the PagesRouter is principally just serving pages, the functionalities these tests are probing should not be influenced by the PagesRouter. In fact, I adapted all these tests to pass, but decided to not include the duplicate test files, see commit 389fb06 for files:
  • PasswordPolicy.spec.js
  • RegexVulnerabilities.spec.js
  • EmailVerificationToken.spec.js
  • ValidationAndPasswordsReset.spec.js

Removing feature flag

Todos when removing the feature flag in the future:

• Remove the pages.enableRouter option, this was just used to turn on the experimental feature
• Adapt tests to work with PagesRouter
• Delete directories public_html, views (PagesRouter only uses files in public)
• Remove server options customPages, they become pages.customUrls in the PagesRouter
• Possibly rename endpoint choose_password -> password_reset to match naming through code and templates (only the endpoint is named choose password, everything else is named password reset); also review other endpoint names (verificationEmailRequest, requestPasswordReset).
• Adapt in Config.js the get xxxURL() methods, e.g. verifyEmailSuccessURL().

Notes

• This can be used in combination with the Parse Server API Mail Adapter to provide a fully localized flow (email -> pages) for the user. The email adapter sends out a localized email and adds a locale parameter to the password reset / email verification link, which is then used by PagesRouter to respond with localized content.
• The PagesRouter can be enabled without localization. While the main feature of this new router is the ability to localize, it is not the only improvement compared to the current PublicAPIRouter. You are encouraged to try the new router with or without localization so we receive feedback before removing the feature flag.

TODOs before merging

• Add test cases
• Add entry to changelog
• Add changes to documentation (guides, repository pages, in-code descriptions
• [ ] Incorporate outcome of discussion in Remove username from password reset / email verification links #7137; adapt routes if username is removed

[codecov]

Codecov Report

Merging #7128 (567c37e) into master (cca493b) will increase coverage by 0.07%.
The diff coverage is 97.26%.

@@            Coverage Diff             @@
##           master    #7128      +/-   ##
==========================================
+ Coverage   93.92%   94.00%   +0.07%     
==========================================
  Files         169      172       +3     
  Lines       12547    12835     +288     
==========================================
+ Hits        11785    12065     +280     
- Misses        762      770       +8     

Impacted Files	Coverage Δ
src/Options/index.js	100.00% <ø> (ø)
src/Page.js	75.00% <75.00%> (ø)
src/Utils.js	96.55% <96.55%> (ø)
src/Routers/PagesRouter.js	97.59% <97.59%> (ø)
src/Config.js	92.85% <100.00%> (+1.62%)	⬆️
src/Options/Definitions.js	100.00% <100.00%> (ø)
src/ParseServer.js	89.59% <100.00%> (+0.12%)	⬆️
src/PromiseRouter.js	90.10% <100.00%> (ø)
src/Adapters/Files/GridFSBucketAdapter.js	79.50% <0.00%> (-0.82%)	⬇️
src/RestWrite.js	93.51% <0.00%> (-0.33%)	⬇️
... and 4 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cca493b...b301f9e. Read the comment docs.

[mtrezza]

This is ready for review to merge as experimental feature.

Since this is a rather extensive PR, a good starting point for review may be the new localization section in README.md that describes the features.

[davimacedo]

Great job, @mtrezza . That's an awesome feature. LGTM!

[mtrezza]

Thanks @davimacedo. I expect this to replace the PublicApiRouter in some months. In the meantime, it would be good to make sure in our reviews that any PRs regarding the PublicApiRouter (password reset, email verification) also cover the PagesRouter. That will make it easier to take this out of experimental status.

@TomWFox What do you think would be a good place to encourage developers to turn on this experimental feature so we can get more feedback? A pinned GitHub issue?

[parseplatformorg]

🎉 This change has been released in version 5.0.0-beta.1

[parseplatformorg]

🎉 This change has been released in version 5.0.0