Add localizer documentation for Fluent #105

flodolo · 2018-01-25T08:06:21Z

I think this is ready for review. I hope it matches other people's expectation, since the scope of the issues (#103 and #104) was open to interpretation :)

A few notes:

Pontoon UI for FTL is covered in Pontoon docs. It will be integrated in a separate PR, e.g. once the UI is available for PLATFORM. Advanced mode, or FTL Source, is already covered. The change of title is to work around a bug with parenthesis in the automatic TOC generation, the alternative is to create the TOC by hand in that document.
Can you provide an example for variants that doesn't involve branding? I tried to think in German, but I'm lost, since I don't think I can do that with the noun alone (I'd need the article, and then I have a problem with uppercase).
There is not exactly a lot to cover for branding at the moment. If we get buy-in from legal and branding, it will be extended with example of allowed usage.

flodolo · 2018-01-25T08:07:36Z

You can browse the 4 pages from here. Looks like the PROPERTIES syntax highlighting works decently enough.

Pike

Thanks, good start. I do have a couple of comments, though.

Pike · 2018-01-25T10:56:38Z

tools/fluent/README.md

+
+<!-- toc -->
+
+Fluent is a localization paradigm, designed at Mozilla [as part of L20n](https://github.com/projectfluent/fluent/wiki/Fluent-and-L20n), to unleash the expressive power of the natural language. The format used to describe translation resources used by Fluent is called `FTL`.


@stasm filed l20n/spec#17 a while ago

Uhm, had no idea. Should we drop the L20n paragraph entirely?

Some localizers might still know Fluent as L20n. It might be good to mention it; something like previously known as L20n.

There's also projectfluent/fluent#57 which proposes to call Fluent a system, not a paradigm etc.

Re-reading, that comma before the to is weird. This?

Fluent is a localization system, previously known as L20n, designed at Mozilla to unleash the expressive power of the natural language.

Pike · 2018-01-25T11:16:26Z

tools/fluent/basic_syntax.md

+    }
+```
+
+The message displayed by the UI will change based on the value of the external variable `$petFamily`. Notice that one of the variants starts with a `*`: that indicates the **default option**, and it must always be defined in a selector.


I'd remove the whole pet thing, 'cause it's a counter-example to what I think we should do in ftl.

We might add that back with projectfluent/fluent#80 in a better way.

Can you clarify? I think we need an example for selectors before introducing plurals.

If it's about the example itself: I'm not superhappy with the current example either, so I'd be happy to swap it with something different.

On second thoughts, I guess I could use the German example from that PR?

Can you clarify? I think we need an example for selectors before introducing plurals.

Plurals will be by far the most common use-case for selectors and I think it makes sense to start with them. That's why the example in http://projectfluent.org/fluent/guide/selectors.html uses a number for the selector.

The risk of starting with plurals is that it might not be clear that you can have arbitrary, even localized, key names

I also agree with @Pike that the pet example will be better solved with projectfluent/fluent#80. As per https://github.com/projectfluent/fluent/wiki/Good-Practices-for-Developers, given the current Fluent spec, I'd rather see three messages for this than a select expression.

Pike · 2018-01-25T11:17:23Z

tools/fluent/basic_syntax.md

+about = O { -brand-name[locative] }
+```
+
+As for selectors, there must be a default variant, identified by `*`.


IMHO, we should also add how to use gender of brand names here, and in brand_names.md.

I disagree, it would be extremely confusing. That's also why I was asking for an alternative example.

I'm happy to have examples for both case and gender, but it should not be about brand names.

We know that we don't want people to localize brand, why give them examples of exactly what we don't want them to do? If we manage to change the policy, that definitely needs to be added to into brand_names.md

Genders are useful to accord verbs in the past tense or to accord past participles. I don't think we should add genders right here, but I'd like to see an example in brand_names.md.

Can you think of an example of string with gender variants that doesn't imply declining the brand name?

IIRC, Polish had program brandshortname in update dialogs only to adjust for the verb in the sentence.

Pike · 2018-01-25T11:19:52Z

tools/fluent/brand_names.md

+Brand names, like Firefox or Sync, are stored as *private messages* in dedicated files, and shared across applications.
+
+Unless indicated otherwise in comments, the [current policy](https://www.mozilla.org/styleguide/communications/translation/) is that brand names can’t be:
+* Declined to adapt to grammatical case.


Did we tighten this additionally? I recall that pre- and suffixes were OK, as long as they didn't cut into the brand name. I.e., Firefoxa was OK, Firefoksa not.

The policy sucks, but IMO it's pretty clear, we just never enforced it.
https://www.mozilla.org/en-US/styleguide/communications/translation/

In some languages, depending on the grammatical case of the word, it may need to have a different ending or be otherwise rewritten to make sense. In these instances, please rewrite the sentence instead to keep the brand name unchanged.

This policy does is wayyy easier to implement for web content than for UI (due to space constrains).
Is there any progress on communication with legal about changing that?

It's on @peiying2 to start the discussion with both branding and legal, noting that the previous point of references in Legal just left (but I keep nagging her nonetheless ;-)).

Pike · 2018-01-25T11:20:39Z

tools/fluent/brand_names.md

+Unless indicated otherwise in comments, the [current policy](https://www.mozilla.org/styleguide/communications/translation/) is that brand names can’t be:
+* Declined to adapt to grammatical case.
+* Transliterated.
+* Translated.


As mentioned above, I think we should add a section here how to annotate brandnames with gender and other grammatical information to adapt messages referencing it.

As explained, given that in Gecko you currently can't use gender or case for brand names, I think it would be counterproductive to add that information at this point.

It can use gender.

Makes sense, but I definitely need an example for that. Happy to had selectors and gender here, as long as we find an example that keeps the brand itself untouched.

Gosh, I wish we had this discussion with legal already, instead of starting like this 😠

I can come up with an example in Polish of French which uses a different form of the verb in the past tense for feminine brand names. The brand name itself isn't changed. Something like: { -brand-name } has been updated = { -brand-name } a été mis/mise à jour. What say you?

Full example in Polish:

-brand-name = Firefox .gender = masculine has-been-updated = { -brand-name.gender -> [masculine] { -brand-name } został zaktualizowany. [feminine] { -brand-name } została zaktualizowana. [neuter] { -brand-name } zostało zaktualizowane. *[other] Program { -brand-name } został zaktualizowany. }

Pike · 2018-01-25T11:22:47Z

tools/fluent/gecko_builtins.md

+
+Functions can be used in Fluent to format data according to the current language’s rules, or can provide additional data to fine tune the translation.
+
+Fluent includes some [generic built-in functions](http://projectfluent.org/fluent/guide/functions.html#built-in-functions), like `NUMBER` and `DATETIME`. Such built-in functions can be used in placeables and selectors. For example:


Just quickly say what NUMBER and DATETIME do? And refer to the docs for the detailed options? Maybe just for number and date formatting, resp..

stasm · 2018-01-25T12:17:03Z

tools/fluent/basic_syntax.md

+
+```PROPERTIES
+menu-settings = Settings
+help-menu-settings = Click “{ menu-settings }” to save the file.


This might suggest that you also need "" around the placeable. Can you either drop the quotes or provide the expected output of the help-menu-settings message?

That's actually why I used curly quotes, but I see the risk. Let's drop them

Yeah, I noticed they weren't straight double quotes and I certainly appreciate the touch :), but I think it's safer to drop them.

stasm · 2018-01-25T12:18:51Z

tools/fluent/basic_syntax.md

+
+There is a special category of messages, called **private messages**:
+* Private messages have identifiers starting with a dash, e.g. `-brand-short-name`.
+* They can’t be called directly in the source code, but only referenced within other messages.


There's one more super-important point to be made about private messages: their attributes are entirely managed by the localizer. That's the private part. There can be no attributes, one or many; regardless of how many of them the source language has. Perhaps we could mention it here and then expand on it in the section about attributes?

stasm · 2018-01-25T12:22:34Z

tools/fluent/basic_syntax.md

+```PROPERTIES
+-brand-short-name = Firefox
+close-msg = Close { -brand-short-name }
+```


Yeah, I think this calls for an explanation as to why the brand name is not a public message here. Something like:

The reason why we're using a private message here, -brand-short-name, is that private message can have any number of localizer-defined attributes which can be used to encode grammatical properties of the message. Mozilla tools like compare-locales won't report those attributes as obsolete if the en-US source doesn't have them. You're free to put information about genders, animacy, the first letter being a vowel or not and any more into attributes of private messages.

stasm · 2018-01-25T12:24:15Z

tools/fluent/basic_syntax.md

+
+The message `login-button` doesn’t have a *value*, but has 2 *attributes* defined: `label` and `accesskey`. There are a few things to highlight in this FTL fragment:
+* Attribute definitions must be indented and start with a period.
+* There is an equal sign following the message identifier, even if there is no value (i.e. the value is *Null*).


Maybe add:

All attributes of public messages found in the en-US source must be present in the localization. This will be enforced by tools like compare-locales.

stasm · 2018-01-25T12:25:13Z

tools/fluent/basic_syntax.md

+    }
+```
+
+The message displayed by the UI will change based on the value of the external variable `$petFamily`. Notice that one of the variants starts with a `*`: that indicates the **default option**, and it must always be defined in a selector.


I also agree with @Pike that the pet example will be better solved with projectfluent/fluent#80. As per https://github.com/projectfluent/fluent/wiki/Good-Practices-for-Developers, given the current Fluent spec, I'd rather see three messages for this than a select expression.

stasm · 2018-01-25T12:27:30Z

tools/fluent/brand_names.md

+
+**They need to be kept in English**, and sentences adapted as needed to fit the undeclined name.
+
+For example, a message `-brand-short-name` is exposed in `browser/branding/official/brand.ftl`, and it can be referenced in other messages as follows:


It feels a bit funky to say that a private message is exposed :) Maybe use provided?

stasm · 2018-01-25T12:29:01Z

tools/fluent/brand_names.md

+Brand names, like Firefox or Sync, are stored as *private messages* in dedicated files, and shared across applications.
+
+Unless indicated otherwise in comments, the [current policy](https://www.mozilla.org/styleguide/communications/translation/) is that brand names can’t be:
+* Declined to adapt to grammatical case.


stasm · 2018-01-25T12:30:15Z

tools/fluent/brand_names.md

+Unless indicated otherwise in comments, the [current policy](https://www.mozilla.org/styleguide/communications/translation/) is that brand names can’t be:
+* Declined to adapt to grammatical case.
+* Transliterated.
+* Translated.


I can come up with an example in Polish of French which uses a different form of the verb in the past tense for feminine brand names. The brand name itself isn't changed. Something like: { -brand-name } has been updated = { -brand-name } a été mis/mise à jour. What say you?

flodolo

Thanks, let me try to update the doc with this information

flodolo · 2018-01-25T12:35:54Z

tools/fluent/basic_syntax.md

+
+```PROPERTIES
+menu-settings = Settings
+help-menu-settings = Click “{ menu-settings }” to save the file.


That's actually why I used curly quotes, but I see the risk. Let's drop them

flodolo · 2018-01-25T13:55:33Z

Submitted a new changeset:

Dropped the pet example for selectors. I'd really like to have an example that is not plurals there, but can't think of any.
Tried to come up with an alternative example for variants. I know it's a poor example in terms of localizability (getting name+adj out of the sentence, plus capitalization issues), but I think it's worth keeping (assuming it makes actual sense in German…)

stasm · 2018-01-25T14:11:02Z

tools/fluent/basic_syntax.md

+-creature-elf = an Elf
+
+you-found-elf = You found { -creature-elf }.
+elf-found-you = Oops, { -creature-elf } found you.


I know it's hard to find good examples which are not complex. Sadly, I don't think this example is a good one. As soon as you hard-code elf into the message identifier you-found-elf, the good practice is to just spell out an elf in the same message rather than factor it out. So:

you-found-elf = You found an elf.

Maybe we should drop the section on variants and bring it back when the brand name localization requirements are relaxed?

I know it's a bad example in terms of localizability, but the target here is localizers (to understand the syntax), not developers. Sure, I'd be happy to find a better example :-\

stasm · 2018-01-25T14:14:16Z

tools/fluent/basic_syntax.md

    }

-about = O { -brand-name[locative] }
+you-found-elf = Du hast { -creature-elf[Akkusativ] } gefunden.
+elf-found-you = Ups, { -creature-elf[Nominativ] } dich gefunden.


This is missing the auxiliary verb: hat dich gefunden :)

Ugh, poor German

stasm · 2018-01-25T14:17:02Z

tools/fluent/basic_syntax.md

 ```

-As for selectors, there must be a default variant, identified by `*`.
+As for selectors, there must be a default variant, identified by `*`. Also notice that key names are arbitrary, and don’t need to be in English.


I'm confused by as for selectors. AFAICT it means when it comes to selectors.... Should this read Similar to selectors?

Right, I meant "Similar to selectors" here, and "As for" doesn't mean that.

flodolo · 2018-01-25T15:03:34Z

Updated, leaving the German example in for now, waiting for more feedback.

zbraniecki

this looks awesome ! Thanks so much Flod!

flodolo · 2018-01-25T19:08:44Z

At this point, I think we need to make a call about the variants section.

Personally I'd like to keep it, because it's an important feature of Fluent, even if the example is not great. The alternative is to drop it, given that it doesn't make sense to use the branding example, when we can't actually decline brand names for policy reasons.

stasm · 2018-01-26T10:10:02Z

I think we should drop it for now. Here's why:

The target audience of this documentation are Mozilla localizers. We currently don't have any use-cases for selector-less variants at Mozilla because of brand localization policies.
For general audience, there's already http://projectfluent.org/fluent/guide/variants.html and we can expand it if needed.
There's already a lot of new things to learn about Fluent so removing things is good.
Once we have a use-case, we can always add more docs.

flodolo · 2018-01-26T10:34:32Z

I think we should drop it for now. Here's why:

OK, removed, since I believe @Pike was on the same page.

Pike

This looks really good, thanks.

I do wonder about our syntax highlighting on gitbook, I didn't manage to find .properties support there.

Final nit, if you can find something for Gecko BUILT-INs that sounds less scary?

Pike · 2018-01-26T11:36:13Z

tools/fluent/README.md

+
+* [Introduction to Fluent syntax](basic_syntax.md).
+* [Brand names](brand_names.md).
+* [Gecko BUILT-INs](gecko_builtins.md).


Can we find a less jargon-y title for this? Just Functions? Surely not ALL-CAPS, that just scares people, and rightfully so ;-) .

Gecko built-in functions?

Pike · 2018-01-26T11:43:22Z

tools/fluent/gecko_builtins.md

+
+Functions can be used in Fluent to format data according to the current language’s rules, or can provide additional data to fine tune the translation.
+
+Fluent includes some [generic built-in functions](http://projectfluent.org/fluent/guide/functions.html#built-in-functions), like `NUMBER` and `DATETIME`, for numbers and dates formatting, respectively. Such built-in functions can be used in placeables and selectors. For example:


.... number and date formatting, per irc.

flodolo · 2018-01-26T12:00:20Z

Switched to "function", and added links to the docs (in particularly SUMMARY).
2977cb2

Indeed, GitBook doesn't have PROPERTIES highlighting, code is displayed plain. Technically we could write a plugin that would serve both our docs and projectfluent.org, es. https://github.com/gaearon/gitbook-plugin-prism

stasm

Great job, @flodolo!

stasm · 2018-01-26T13:33:07Z

tools/fluent/functions.md

+
+## PLATFORM
+
+`PLATFORM` is a function that allows localizers to taylor messages to the current platform. Allowed key variants are: `linux`, `win`, `macosx`, `android`, `other`.


Typo: to tailor. And Allowed variant keys is perhaps better?

The keys are still being discussed in https://bugzilla.mozilla.org/show_bug.cgi?id=1426063#c17.

Yes, the plan is to leave this PR open until that bug is solved.

flodolo · 2018-01-26T13:42:56Z

Waiting for a final decision on the platform keys before merging.

flodolo · 2018-01-28T07:59:15Z

@stasm @Pike
A recent message from Piotr on #l10n made me think about an issue we already had in the past.

is this correct: mozilla-services/screenshots@31bbfb4 ? pontoon doesn't seem to allow me to translate .title

I've tried to cover it in eef0d7f

Can you check if it makes sense? I've explicitly avoided using "localize" in the first part, because it would be more confusing than helpful.

flodolo · 2018-01-28T11:09:39Z

A recent message from Piotr on #l10n made me think about an issue we already had in the past.

Turns out Piotr's comment was completely unrelated. Still, I think there's some value in adding that information.

Pike · 2018-01-28T15:11:53Z

I wonder if there's something better for attribute localization.

The way I think about them is that the value-or-not and attribute IDs are like the shape of a message. For public messages, squares need to be squares, circles need to be circles. For private messages, the shape depends on the language, and how the message is used.

stasm · 2018-01-29T07:15:59Z

It's a question of the interface of the message but also that of the wording. I find private messages don't really help here because they're not, in fact, private—their attributes are. This has larger impact on Fluent; I filed projectfluent/fluent#85. Let's discuss there.

flodolo · 2018-01-29T07:17:08Z

I'll revert the last commit in the meantime, and go back to the reviewed version.

This reverts commit eef0d7f.

flodolo · 2018-02-05T07:19:35Z

I'm going to merge this PR to have references for the monthly report. We can fix nits in a follow-up PR as needed.

Delphine · 2018-02-08T23:07:57Z

Seems like this - as well as Kekoa's Styleguides "Guidelines" + "mozilla_general" sections - didn't make it in the l10n Report this month.
I'll add it to next month's version right away so it doesn't fall through the cracks this time

flodolo · 2018-02-09T05:18:01Z

Seems like this - as well as Kekoa's Styleguides "Guidelines" + "mozilla_general" sections - didn't make it in the l10n Report this month.

It is mentioned+linked in the Firefox section, didn't think it was useful to call it out twice.

flodolo added 4 commits January 24, 2018 14:12

Change title for TOC issue

8d79ab5

Add Fluent syntax document

bff113c

Add Fluent syntax document

d977cf5

Try .properties syntax highlight

4a9704e

flodolo requested review from Pike, stasm and zbraniecki January 25, 2018 08:06

Pike suggested changes Jan 25, 2018

View reviewed changes

Address pike comments (1st part)

d14c7ac

stasm suggested changes Jan 25, 2018

View reviewed changes

flodolo commented Jan 25, 2018

View reviewed changes

Address review comments (pike+stas)

6019a30

stasm reviewed Jan 25, 2018

View reviewed changes

Fix 2 more nits (stas)

bbf0e75

zbraniecki approved these changes Jan 25, 2018

View reviewed changes

Improve sentence in variants section

2b45cbe

Remove Variants section

9e0b0a7

Fix English

bd824ab

Pike reviewed Jan 26, 2018

View reviewed changes

Use functions instead of BUILT-INs, add docs to SUMMARY.md

2977cb2

Pike approved these changes Jan 26, 2018

View reviewed changes

stasm approved these changes Jan 26, 2018

View reviewed changes

Final nits (stas)

abccf7e

flodolo self-assigned this Jan 26, 2018

flodolo added the DO-NOT-MERGE label Jan 26, 2018

Update platform names

f11fd80

Add notes about attributes translation

eef0d7f

flodolo force-pushed the issue103 branch from 1a65c45 to eef0d7f Compare January 28, 2018 08:00

flodolo added 2 commits January 29, 2018 08:18

Revert "Add notes about attributes translation"

acb59e8

This reverts commit eef0d7f.

Use 'term' instead of 'private message'

652ed71

flodolo merged commit e5dabd9 into master Feb 5, 2018

flodolo deleted the issue103 branch February 5, 2018 07:19

flodolo removed the DO-NOT-MERGE label Feb 5, 2018

This was referenced Feb 5, 2018

Document brand.ftl features and rules #104

Closed

Document the use of PLATFORM built-in in Fluent #103

Closed


		<!-- toc -->

		Fluent is a localization paradigm, designed at Mozilla [as part of L20n](https://github.com/projectfluent/fluent/wiki/Fluent-and-L20n), to unleash the expressive power of the natural language. The format used to describe translation resources used by Fluent is called `FTL`.


		Functions can be used in Fluent to format data according to the current language’s rules, or can provide additional data to fine tune the translation.

		Fluent includes some [generic built-in functions](http://projectfluent.org/fluent/guide/functions.html#built-in-functions), like `NUMBER` and `DATETIME`. Such built-in functions can be used in placeables and selectors. For example:


		They need to be kept in English, and sentences adapted as needed to fit the undeclined name.

		For example, a message `-brand-short-name` is exposed in `browser/branding/official/brand.ftl`, and it can be referenced in other messages as follows:


		## PLATFORM

		`PLATFORM` is a function that allows localizers to taylor messages to the current platform. Allowed key variants are: `linux`, `win`, `macosx`, `android`, `other`.

Add localizer documentation for Fluent #105

Add localizer documentation for Fluent #105

Conversation

flodolo commented Jan 25, 2018

flodolo commented Jan 25, 2018

Pike left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

flodolo Jan 25, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

flodolo Jan 25, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stasm Jan 25, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

flodolo Jan 25, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

flodolo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

flodolo commented Jan 25, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

flodolo commented Jan 25, 2018

zbraniecki left a comment

Choose a reason for hiding this comment

flodolo commented Jan 25, 2018

stasm commented Jan 26, 2018

flodolo commented Jan 26, 2018

Pike left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

flodolo commented Jan 26, 2018

stasm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

flodolo commented Jan 26, 2018

flodolo commented Jan 28, 2018 • edited Loading

flodolo Jan 25, 2018 •

edited

Loading

flodolo Jan 25, 2018 •

edited

Loading

stasm Jan 25, 2018 •

edited

Loading

flodolo Jan 25, 2018 •

edited

Loading

flodolo commented Jan 28, 2018 •

edited

Loading