Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract default messages script #1216

Merged

Conversation

alexvuong
Copy link
Collaborator

@alexvuong alexvuong commented May 18, 2023

Description

This PR created a build-translation scripts that helps to extract messages from retail-template and extended project and output into translations folder
The script will ignore extracting messages from any files that are overridden.
Move translations folder from inside app to root directory

Types of Changes

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Documentation update
  • Breaking change (could cause existing functionality to not work as expected)
  • Other changes (non-breaking changes that does not fit any of the above)

Breaking changes include:

  • Removing a public function or component or prop
  • Adding a required argument to a function
  • Changing the data type of a function parameter or return value
  • Adding a new peer dependency to package.json

Changes

  • (change1)

How to Test-Drive This PR

  • pull the request
  • npm ci
  • rn -rf generated-project
  • run node packages/pwa-kit-create-app/scripts/create-mobify-app-dev.js --outputDir generated-project to generate a project
  • change the extract-default-translations as below
   "extract-default-translations": "node ./node_modules/retail-react-app/scripts/extract-default-messages.js",
  • npm run build-translations
  • confirm that all translations are extracted and compiled under generated-projected/translations/en-US.json

Checklists

General

  • Changes are covered by test cases
  • CHANGELOG.md updated with a short description of changes (not required for documentation updates)

Accessibility Compliance

You must check off all items in one of the follow two lists:

  • There are no changes to UI

or...

Localization

  • Changes include a UI text update in the Retail React App (which requires translation)

@alexvuong alexvuong added the do not merge No matter what, do not merge this pr label May 18, 2023
@alexvuong alexvuong self-assigned this May 18, 2023
@alexvuong alexvuong marked this pull request as ready for review May 23, 2023 16:44
@alexvuong alexvuong requested a review from a team as a code owner May 23, 2023 16:44
@alexvuong alexvuong changed the title Build translation script - testing on generated project Build translation script May 23, 2023
@alexvuong alexvuong changed the title Build translation script Extract default messages script May 23, 2023
@alexvuong alexvuong added ready for review PR is ready to be reviewed and removed do not merge No matter what, do not merge this pr labels May 23, 2023
bfeister and others added 7 commits May 23, 2023 15:43
…tion-script

* generator/extensibility-support-new:
  Feature/template extensibility (#1162)
  lockfiles from reaact18 / chakra2
  remove demo extensible app in light of soon-to-be-merged PR from @bendvc

# Conflicts:
#	packages/template-retail-react-app/app/pages/home/index.test.js
…rceCloud/pwa-kit into build-translation-script
"extract-default-translations": "formatjs extract 'app/**/*.{js,jsx}' --out-file app/translations/en-US.json --id-interpolation-pattern [sha512:contenthash:base64:6]",
"compile-translations": "formatjs compile-folder --ast translations translations/compiled",
"compile-translations:pseudo": "formatjs compile --ast translations/en-US.json --out-file translations/compiled/en-XB.json --pseudo-locale en-XB",
"extract-default-translations": "node ./scripts/extract-default-messages.js",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's have our script be able to receive any given locale. So that developers do not need to go into the script's implementation to replace en-US with another locale. They can simply update the default locale in this package.json file instead.

Suggested change
"extract-default-translations": "node ./scripts/extract-default-messages.js",
"extract-default-translations": "node ./scripts/extract-default-messages.js en-US",

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, I've added to include locale as an arg into the script

const files = getAllFilesByExtensions(
path.join(overridesPath, 'app'),
[],
['js', 'jsx', 'ts', 'tsx']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are looking for Typescript files here, then we'll need to update all of the formatjs commands to search for those files too.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

fs.rename(filePath, `${filePath}.ignore`, (err) => err && console.error(err))
})

const extractCommand = `formatjs extract "./node_modules/${pkgJSON.ccExtensibility?.extends}/app/**/*.{js,jsx}" "${pkgJSON.ccExtensibility?.overridesDir}/app/**/*.{js,jsx}" --ignore "./node_modules/${pkgJSON.ccExtensibility?.extends}/app/**/*.ignore" --out-file translations/en-US.json --id-interpolation-pattern [sha512:contenthash:base64:6]`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we already renamed the file extension to js.ignore, is it still being included in the globe /app/**/*.{js,jsx}? I'm just wondering whether we still need the --ignore or not.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, it seems like changing the file extension works, we don't need to use --ignore

@@ -17,13 +17,13 @@ export const fetchTranslations = async (locale) => {
const targetLocale =
typeof window === 'undefined'
? process.env.USE_PSEUDOLOCALE === 'true'
? 'en-XB'
? 'en-XA'
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The newest version we are using with formatjs 6.1.1 (from lock file generation) no longer offers en-XB pseudo locale. en-XA is the closest we can use. https://formatjs.io/docs/tooling/cli/#--pseudo-locale-pseudolocale

@alexvuong alexvuong merged commit 336b4e7 into generator/extensibility-support-new May 23, 2023
fs.rename(filePath, `${filePath}.ignore`, (err) => err && console.error(err))
})

const extractCommand = `formatjs extract "./node_modules/${pkgJSON.ccExtensibility?.extends}/app/**/*.{js,jsx}" "${pkgJSON.ccExtensibility?.overridesDir}/app/**/*.{js,jsx}" --out-file translations/${locale}.json --id-interpolation-pattern [sha512:contenthash:base64:6]`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const extractCommand = `formatjs extract "./node_modules/${pkgJSON.ccExtensibility?.extends}/app/**/*.{js,jsx}" "${pkgJSON.ccExtensibility?.overridesDir}/app/**/*.{js,jsx}" --out-file translations/${locale}.json --id-interpolation-pattern [sha512:contenthash:base64:6]`
const extractCommand = `formatjs extract "./node_modules/${pkgJSON.ccExtensibility?.extends}/app/**/*.{js,jsx,ts,tsx}" "${pkgJSON.ccExtensibility?.overridesDir}/app/**/*.{js,jsx,ts,tsx}" --out-file translations/${locale}.json --id-interpolation-pattern [sha512:contenthash:base64:6]`

@bfeister bfeister mentioned this pull request May 24, 2023
12 tasks
bendvc added a commit that referenced this pull request May 26, 2023
…1205)

* merge conflict

* resolve merge conflict

* update spike project's engines / regen v3 lockfiles

* regenerate lock files

* add missing node engines

* add missing useToast (bad merge)

* updates from underlying template-retail-react-app

* drop unnecessary pages/product-detail override

* fix dupe useToast

* progress on making `charles-solution` work

* initial pass at migration to `resolver.getHook('resolve').tapAsync` working, TODO next... cleanup

* restore icons file and add extensible text to home

* add some comments and construct hashmap in new plugin

* working glob sync lookup??

* resolve lookup when there is no extension

* add component slot for testing

* fix some lookup errors

* migrate to new directory

* remove unnecessary directory nesting, remove unnecessary files for "minimum file project"

* add example of extending routes non-destructively, add example of quickly toggling some base constant values (e.g. categories shown on home page)

* Update example route

* add some demo examples of overrides

* cleanup

* commit package name change

* rewrite requests coming from underlying template

* cleanup

* rename plugin file

* remove unnecessary build files

* rename to overrides

* fix lookup errors

* fix some broken upstream changes, eslint fixes

* fix bad merge conflict

* Bring back the deleted package.json

* Install new eslint plugin

* No more relative imports in retail-react-app

* Create .prettierrc.yaml

* restore other deleted package lockfiles

* fix some eslint issues

* remove `^` imports and begin migrating logic away from previous API. TODO: get webpack to resolve `retail-react-app` base import path from within `template-retail-react-app`

* bring back deleted package lockfiles

* short circuit plugin temporarily, get webpack `alias` working for template to resolve self-referencing `retail-react-app` imports

* fix unreachable condition

* wow... freaking works 🚀

* cleanup, get `template-retail-react-app` working again

* cleanup

* Create .eslintrc.js

* fix routes.jsx which no longer pulled relative template

* make routes.jsx more concise

* update lockfiles

* add jsconfig.json for IDEs to have magic `retail-react-app` be discoverable

* more cleanup

* change package.json key from `mobify` => `ccExtensibility`

* cleanup

* Final cleanup, remove console.logs, make `extendable` dynamic for internal file resolution in `template-retail-react-app` (and future projects), remove some non-required files,

* fix straggler non-dynamic filepath

* cleanup

* remove code that shouldn't be in v3+

* sync package.json deps in extended example project

* cleanup / pr feedback

* pr feedback, great catch @kevinxh

* fix problem with bad copy from underlying template file

* pr feedback / cleanup

* cleanup

Co-authored-by: Kevin He <kevin.he@salesforce.com>

* Update package-lock.json

* WIP

* drop unneeded complexity

* lockfiles

* moving toward an array of `alias`es

* Refactor template extensibility functionality to use `resolve.alias` + greatly reduce code footprint + make leading slash optional in `ccExtensibility.overridesDir`

* add brand-logo with no clash for server side viewBox

* add `request-processor.js` (required for worker.js to load) and fix bad extensibility path loading in `build-dev-server.js`

* lint fix

* fix failing pwa-kit-react-sdk tests

* cleanup

* regen package lockfiles

* lint:fix

* fix failing tests in CI re: `TextDecoder is not defined`

* take 2 attempting to fix TextDecoder global in jest

* fix jest module mapper

* fix jest moduleMapper config

* REALLY fix jest moduleNameMapper config for new pathing + fix node `util` global TextDecoder is not defined in jest

* add brand-logo

* remove previously added `AbovePDP` Template Hook component

* replay #1128 from bad merge

* eslint:fix for `retail-react-app` base paths

* de-duplicate deps in bundle

* fix path defaulting, don't pass nullish value to path.resolve

* improve pathing for overridesDir in pwa-kit-dev webapack config + add missing worker/main.js

* fix pathing

* fix pathing issue that was mysteriously UNFIXED

* fix `worker.js` path

* fix app/static pathing in non-extensible builds

* Bump version to 3.0.0-dev.0

* Fixup dependencies in `template-retail-react-app`

* Create boostrap to replace `my-extended-retail-app`

* Update generator to support extensible projects (rough implementation)

* Create boostrap to replace `my-extended-retail-app`*

* Ensure that translations are chunked

* Remove `my-extended-retail-app` as it is not required

* Fix lint errors

* Fix lint errors

* Add Handlebars and update package.json files

* Fix bootstrap folder organization

* Make template public

* Fix default overrides folder

* Bump to 3.0.0-dev.0 to align with develop

* Initial generator refactor

* Fix version fetching

* Attempt to fix version fetching for extensibility

* Another attempt at fixing the version

* Eek another attempt

* Bad code completion suggestion

* Fix some template keys

* Fix version number

* Update create-mobify-app.js

* Fix template keys in override assets

* Update create-mobify-app.js

* [Extensibility] Remove Einstein proxy and send Einstein events directly from app (#1202)

* Remove Einstein proxy and send Einstein events directly from app

* Update ssrParameters

* Add context

* Fix template key

* Update package.json.hbs

* Add handlebars json helper

* Disabled html encoding for json objects in templates

* Fix formatting in scripts

* Allow demo to use extensibility

* Slugify name

* Move boostrap folder

* Setting up for bootstrap file reuse

* debugging

* Update create-mobify-app.js

* Testing with postGenerate hook

* debugging

* Simplify template sources

* Fix template keys yet again

* Replace ExtendsCircularImportsPlugin with OverridesResolverPlugin (#1211)

* Replace ExtendsCircularImportsPlugin with OverridesResolverPlugin

* fix filepath problem

---------

Co-authored-by: Brian Feister <bfeister@salesforce.com>

* Update routes.jsx.hbs

* Update create-mobify-app.js

* Fix things

* Fix filter

* Fix preset validation

* Update create-mobify-app.js

* Update create-mobify-app.js

* Clean up

* remove irrelevant bundlesize check to pass CI

* Fix `vendor.js` inclusion of `retail-react-app` files (#1198)

* fix duplicate lilbs in bundle

* fix app/static pathing for non-extensible builds

* oof, fix extensible app/static filepaths

* Resolve conflicting npm deps via `resolve.alias`

* add missing worker/main.js

* cleanup unused imports

* WIP, add list of original overridable deps

* prevent template extensibility vendor.js from treating baseline route files as deps in vendor.js

* fix irrelevant bundlesize declaration to pass CI

* drop test:max-file-size test from my-extended-retail-app

* replace relative with absolute path fetchTranslations (#1215)

Co-authored-by: Brian Feister <47546998+bfeister@users.noreply.github.com>

* More refactoring

* Update create-mobify-app.js

* Add template level file assets

* debugging

* Update create-mobify-app.js

* Fix templates

* Fix manifest template keys

* Rearrange deps

* Better handling of objects in hbs templates

* Remove hbs json helper

* Fix missing comma

* More handlebars fixes

* Escape scripts

* make tempalte scripts work with handlebars escaping

* Delete my-exended-template after merging

* Refactor for unit testing, fix pathing issue from old version of file parsing algoritihim

* More clean up

* Remove console log

* preset changes

* Linting, and change test-project name

* Function renames etc

* debugging

* Update create-mobify-app.js

* Fix project answers merge

* Fix CI test 🤞🏻

* Update deps in attempt to fix tests on CI

* Lint

* More dep shuffling

* Remove debugger

* More dep wrangling

* Don't use peer deps like I was

* Another iteration of dep organization

* Clean up hbs templates

* Update licences

* remove demo extensible app in light of soon-to-be-merged PR from @bendvc

* lockfiles from reaact18 / chakra2

* fix merge conflict resolution

* remove template extensible demo in prep for @bendvc's pr for generator work

* regen lockfiles for react 18 changes

* fix linting

* update package-lock files

* Fix imports

* Add missing deps

* Feature/template extensibility (#1162)

* initial work on moving code to use resolver

* fix getoverridepath and break on ^ imports

* add todos and comments

* rewrite requestcontext.request for path with caret

* perform overrideshashmap lookup for relative import

* progress

* progress

* regenerate lock files

* fix dupe useToast

* progress on making `charles-solution` work

* initial pass at migration to `resolver.getHook('resolve').tapAsync` working, TODO next... cleanup

* restore icons file and add extensible text to home

* add some comments and construct hashmap in new plugin

* working glob sync lookup??

* resolve lookup when there is no extension

* add component slot for testing

* fix some lookup errors

* cleanup

* rewrite requests coming from underlying template

* cleanup

* rename plugin file

* remove unnecessary build files

* rename to overrides

* fix lookup errors

* fix some broken upstream changes, eslint fixes

* Bring back the deleted package.json

* Install new eslint plugin

* No more relative imports in retail-react-app

* Create .prettierrc.yaml

* restore other deleted package lockfiles

* fix some eslint issues

* remove `^` imports and begin migrating logic away from previous API. TODO: get webpack to resolve `retail-react-app` base import path from within `template-retail-react-app`

* short circuit plugin temporarily, get webpack `alias` working for template to resolve self-referencing `retail-react-app` imports

* fix unreachable condition

* wow... freaking works 🚀

* cleanup, get `template-retail-react-app` working again

* cleanup

* Create .eslintrc.js

* fix routes.jsx which no longer pulled relative template

* make routes.jsx more concise

* update lockfiles

* add jsconfig.json for IDEs to have magic `retail-react-app` be discoverable

* more cleanup

* change package.json key from `mobify` => `ccExtensibility`

* cleanup

* Final cleanup, remove console.logs, make `extendable` dynamic for internal file resolution in `template-retail-react-app` (and future projects), remove some non-required files,

* fix straggler non-dynamic filepath

* cleanup

* remove code that shouldn't be in v3+

* sync package.json deps in extended example project

* cleanup / pr feedback

* pr feedback, great catch @kevinxh

* fix problem with bad copy from underlying template file

* pr feedback / cleanup

* cleanup

Co-authored-by: Kevin He <kevin.he@salesforce.com>

* Update package-lock.json

* WIP

* drop unneeded complexity

* lockfiles

* moving toward an array of `alias`es

* Refactor template extensibility functionality to use `resolve.alias` + greatly reduce code footprint + make leading slash optional in `ccExtensibility.overridesDir`

* add brand-logo with no clash for server side viewBox

* add `request-processor.js` (required for worker.js to load) and fix bad extensibility path loading in `build-dev-server.js`

* lint fix

* fix failing pwa-kit-react-sdk tests

* cleanup

* lint:fix

* fix failing tests in CI re: `TextDecoder is not defined`

* take 2 attempting to fix TextDecoder global in jest

* fix jest module mapper

* fix jest moduleMapper config

* REALLY fix jest moduleNameMapper config for new pathing + fix node `util` global TextDecoder is not defined in jest

* add brand-logo

* remove previously added `AbovePDP` Template Hook component

* replay #1128 from bad merge

* eslint:fix for `retail-react-app` base paths

* de-duplicate deps in bundle

* fix path defaulting, don't pass nullish value to path.resolve

* improve pathing for overridesDir in pwa-kit-dev webapack config + add missing worker/main.js

* fix pathing

* fix pathing issue that was mysteriously UNFIXED

* fix `worker.js` path

* fix app/static pathing in non-extensible builds

* [Extensibility] Remove Einstein proxy and send Einstein events directly from app (#1202)

* Remove Einstein proxy and send Einstein events directly from app

* Update ssrParameters

* Replace ExtendsCircularImportsPlugin with OverridesResolverPlugin (#1211)

* Replace ExtendsCircularImportsPlugin with OverridesResolverPlugin

* fix filepath problem

---------

Co-authored-by: Brian Feister <bfeister@salesforce.com>

* remove irrelevant bundlesize check to pass CI

* Fix `vendor.js` inclusion of `retail-react-app` files (#1198)

* fix duplicate lilbs in bundle

* fix app/static pathing for non-extensible builds

* oof, fix extensible app/static filepaths

* Resolve conflicting npm deps via `resolve.alias`

* add missing worker/main.js

* cleanup unused imports

* WIP, add list of original overridable deps

* prevent template extensibility vendor.js from treating baseline route files as deps in vendor.js

* fix irrelevant bundlesize declaration to pass CI

* drop test:max-file-size test from my-extended-retail-app

* replace relative with absolute path fetchTranslations (#1215)

Co-authored-by: Brian Feister <47546998+bfeister@users.noreply.github.com>

* Refactor for unit testing, fix pathing issue from old version of file parsing algoritihim

* fix merge conflict resolution

* remove template extensible demo in prep for @bendvc's pr for generator work

* regen lockfiles for react 18 changes

* fix linting

---------

Co-authored-by: yunakim714 <yunakim@salesforce.com>
Co-authored-by: Vincent Marta <vmarta@salesforce.com>
Co-authored-by: Kevin He <kevin.he@salesforce.com>
Co-authored-by: vcua-mobify <vcua@salesforce.com>
Co-authored-by: vcua-mobify <47404250+vcua-mobify@users.noreply.github.com>
Co-authored-by: Alex Vuong <52219283+alexvuong@users.noreply.github.com>
Co-authored-by: Alex Vuong <alex.vuong@salesforce.com>

* Update packages/pwa-kit-create-app/scripts/create-mobify-app.js

Co-authored-by: Vincent Marta <vmarta@salesforce.com>

* Fix CI workflow project reference

* Update config.js

* support windows file paths

* Lint

* Fix issue with failed tests because of react query dev tools

* Extract default messages script (#1216)

* create extract default messages script to work on both base and overrides projects

---------

Co-authored-by: Brian Feister <bfeister@salesforce.com>
Co-authored-by: Brian Feister <47546998+bfeister@users.noreply.github.com>

* Add typescript extensions (#1223)

* add typescript extensions

* fix failing tests

* Update `extract-default-translations` script for extensibility

* debugging

* Update create-mobify-app.js

* Fix regex

* First pass copying translations

* Update create-mobify-app.js

* Update create-mobify-app.js

* Update create-mobify-app.js

* Update create-mobify-app.js

* Refactor copy assets work

* Remove debugger directive

* Regenerate package-lock files

* Fix lint errors on generated projects

* Update create-mobify-app.js

* Fix some dep versions

* Update create-mobify-app.js

* Update package.json

* Fix lint script so it works with windows

* Regenerate package-lock files and remove my-generated-app again

* Lint

* mv doesnt work the same on windows as unix

* Same

* Revert suggested change.

* Add basic eslintignore

* Update bundle size limits

* Missing dep.

---------

Co-authored-by: Brian Feister <bfeister@salesforce.com>
Co-authored-by: yunakim714 <yunakim@salesforce.com>
Co-authored-by: Vincent Marta <vmarta@salesforce.com>
Co-authored-by: Brian Feister <47546998+bfeister@users.noreply.github.com>
Co-authored-by: Kevin He <kevin.he@salesforce.com>
Co-authored-by: vcua-mobify <vcua@salesforce.com>
Co-authored-by: vcua-mobify <47404250+vcua-mobify@users.noreply.github.com>
Co-authored-by: Alex Vuong <52219283+alexvuong@users.noreply.github.com>
Co-authored-by: Alex Vuong <alex.vuong@salesforce.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready for review PR is ready to be reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants