Add internationalization

[toolness]

I've heard good things about React Intl and babel-plugin-react-intl, but other options are definitely possible. It seems like a good idea to use ICU MessageFormat where possible, since this allows for lots of flexibility when it comes to pluralization, gender, and so on.

We can potentially use Crowdin to make it easy for contributors to provide localizations (it is free for OSS projects). It looks like they have some syntax highlighting for ICU MessageFormat, as well as the ability to preview what a translation will look like with different kinds of parameters (e.g. gender, pluralization, etc) which is very nice.

[toolness]

React Conf 2018 had a nice talk on i18n/l10n that appears to do things the way I've always wished they could be done: allowing developers to just use JSX instead of adding yet another l10n-specific templating layer, while allowing localizers to use their formats/tools of choice as well (ICU MessageFormat, etc), and providing a translation mechanism that converts between the two.

The tool he's created is called lingui and it looks pretty awesome!

[toolness]

Also facebook just busted out a thing that i can’t find a link for right now.

Update: It's called FBT.

[toolness]

Ok, so I did a bit more research today and right now I think it's a bit of a toss-up between react-intl and lingui. I'm particularly fond of lingui because it fixes so many usability problems with react-intl, but it's also a much younger project and as such will likely have more rough edges. Because it's more nascent it might also fall into disrepair if e.g. its creator stops maintaining it, but I am heartened by the level of contributions from community members too.

FBT seems quite unusual compared to other open-source solutions out there, but I suspect that's partly because FB probably has internal proprietary tooling for their localization pipeline. This means that open-source users of their library are a bit stranded when it comes to doing anything with FBT's unusual message catalog format. Unless a big open-source community blossoms around the project--which doesn't seem to be the case based on the current level of activity in their GitHub repo--it seems unlikely that this tool will be particularly convenient for us. However, I've also only taken a cursory look at it so I could be mistaken.

That said, one interesting aspect of the FBT approach seems to be that it avoids the complexity of ICU MessageFormat for translators while simultaneously avoiding the poor translation fidelity of a simplistic gettext-style approach. See FBT's documentation on plurals for an example of how they seem to do this. (Again, though, I've only taken a cursory look so I might be misinterpreting things here.)

[toolness]

The status right now is that lingui looks great--I did an experimental integration in #497--but its one major failing is described in #497 (comment):

One current limitation of lingui is that it only supports a single compiled message catalog--that is, a single JS module that contains all the code needed to render all the strings in the app in a particular language. However, this runs counter to our code-splitting approach, because it means that for the user's browser to render a single page, it will need the localized text of all pages, rather than loading particular chunks of localized text as they're needed. For example, users who just want to send a letter of complaint will need to load all the strings for filing an HP Action too.

[toolness]

Entirely separate from the tech we use for displaying localized content, I've explored using Crowdin with a test repo at JustFixNYC/crowdin-test and it's really slick! It automatically issues PRs like JustFixNYC/crowdin-test#2 and supports multiple different localization file formats per repository (so we can use PO files for Django content and JSON for front-end content, if we want). It also auto-detects ICU MessageFormat content and provides a nifty editor that lets you test out your messages with different values for passed-in variables and such. It's also free for open-source projects. It will definitely make internationalizing our apps a lot easier!

[toolness]

Ok, we're ramping up i18n for the NoRent site, which resides in this codebase, so this is a big priority now. Thoughts:

• We've been using LinguiJS with Crowdin on Who Owns What for a while now (see Add foundational internationalization support via lingui who-owns-what#186) and it works great, so we'll probably go with that here too. Update: Did this in Add Lingui support, take 2. #1376.

• Need to figure out what to do about our Mapbox and NYC GeoSearch autocompletes.

• We'll have to figure out what to do for the rendering of content (like emails and PDFs) from worker threads. I'm thinking we'll probably want to add a language preference field to the JustfixUser model that can be set during onboarding based on the request locale, and then anything happening in workers can look at its value to figure out what locale to generate content in for that user.

• When we create users in TextIt, we'll want to set their TextIt locale to the locale of the user.

• We'll have to figure out how we want to localize content in the common-data directory.

• We'll have to figure out how to localize content we pull from Airtable for NoRent.org. (This will probably involve making separate columns for Spanish-translated content.)

• Make sure we're setting the lang on the <html> element of the page properly too. Update: did this in Set <html> lang attribute dynamically. #1373.

[toolness]

Ok, I've spun off the above list into #1384.

[toolness]

Er this is quite done at this point.