Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple language support #107

Merged
merged 10 commits into from
May 30, 2017
Merged

Multiple language support #107

merged 10 commits into from
May 30, 2017

Conversation

patjouk
Copy link
Contributor

@patjouk patjouk commented May 16, 2017

This PR introduces support for multiple language support. Right now all language files are always loaded. With this PR, the developer can decide which language files to require. During test instruction compilation, the language used can be chosen. This allows us to scale this project to large numbers of languages, without bloating data downloaded in browsers for all users.

The module API as well as internal APIs have breaking changes, since we had to change several function signatures.

TODOs

  • Integrate PR within Mapbox (to see what breaks)
  • Integrate PR with osrm-frontend (to see what breaks)
  • Code review
  • Update the changelog
  • Make sure that relevant docs are updated
  • Merge
  • Release as new version 0.3
  • Update osrm-frontend to use 0.3

CC @freenerd

edit: We will continue to load every language for now because browserify doesn't support dynamic requires. (cf @freenerd's comment)

- Add the new translation file and language tag to `./languages.js`
- If needed: make overrides in `languages/overrides/{language_tag}.json`
- Create an empty translation file `echo "{}" > languages/translations/{language_code}.json`
- Add the new translation file and language code to `./languages.js`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the new translation file

This can be removed, since the code for loading each language file is not present anymore. It's only an array of language codes now that needs to be edited.

var translations = languages.get(['en', 'fr']);

assert.deepEqual(Object.keys(translations).sort(), ['fr', 'en'].sort(),
'only returns en and fr');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line break is a bit weird. I'd prefer either having everything on one line, but since that may not be possible for length reasons I'd prefer one-line-per-parameter:

assert.deepEqual(
  Object.keys(translations).sort(),
  ['fr', 'en'].sort(),
  'only returns en and fr'
);

Readme.md Outdated
`options.hooks.tokenizedInstruction` | optional | `function(instruction)` | A function to change the raw instruction string before tokens are replaced. Useful to inject custom markup for tokens
`options.languages` | optional | `en` `de` `zh-Hans` `fr` `nl` `ru` | Array of language identifiers that should be supported. Default is loading all language files, which can be huge on websites
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don’t want developers to get the impression that only these six languages are supported, given that we currently support ten (with more to come). Perhaps add “etc.” and link to this folder for the full list of supported languages?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good idea. I'm changing this.

index.js Outdated
options.hooks.tokenizedInstruction = ((_options || {}).hooks || {}).tokenizedInstruction;
options.languages = (_options || {}).languages || languages.supportedCodes;

// TODO Validate language
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, what sort of validations do you have in mind?

@1ec5
Copy link
Member

1ec5 commented May 19, 2017

This PR looks like it’ll address #106.

@freenerd
Copy link
Member

I tried to integrate these changes into osrm-frontend, but that failed. The reason is browserify, which we use to bundle the required() js files. Dynamic requires are not supported browserify/browserify#377 and there doesn't seem to be a way to specify which files to required upfront.

I'm good with actually walking back on the requirement of #106 to select which language files to load, given it makes bundling of js files so hard for browsers. For the server-side, this isn't actually a big problem, since the memory required is marginal. We should definitely keep switching languages during compile though.

@patjouk patjouk changed the title [WIP] Selective language support [WIP] Multiple language support May 29, 2017
@patjouk patjouk changed the title [WIP] Multiple language support Multiple language support May 30, 2017
@freenerd freenerd merged commit eca034c into master May 30, 2017
@freenerd freenerd deleted the selectively-languages branch May 30, 2017 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants