Have CI check against updating string content without using a different key

[upsuper]

Recently, I noticed that several Chinese translations mismatch their corresponding strings in teams.ftl, and such mismatch happens among all available translations. After a bit investigation, I think this is because #1030 updated the content of several strings with their original string key, and thus Pontoon doesn't consider the string need to be re-translated.

This seems problematic, and can easily cause translation mismatch silently until someone notices and fixes it in individual language.

I suggest that we add a CI step for PRs to check that no string content in en-US is updated without a new string key. WDYT?

[XAMPPRocky]

Thank you for your issue! My initial thought is that the team fluent file shouldn't be translated, because it is auto-generated, and I think its better to show the correct info in the wrong language than having to worry about it being wrong in the correct language.

[upsuper]

Ideally we want everything to be translated, but I see that it could be challenging for generated files. Maybe one way is to compare new content with existing one, and if a string content changes, update its key with -{number} or something like that.

Also I think the CI check would probably be useful itself outside this specific case, as people may forget it from time to time.

[upsuper]

If you have the program generating that file, maybe it can be checked in to the repo, and we can find a way to try improving it?

[Manishearth]

I think the team file should absolutely be translated.

I'm not quite sure of having a CI check for this, there are often minor content changes that don't matter where you shouldn't update the key. Rather, this seems like something reviewers should watch for.

For the generated files we can do one of two things: ping folks on Zulip when there's an update, or we can set up a special key that isn't used or translated that just says "please check this PR to see what has changed", and bump that

tbh I wish pontoon's notifications could catch changes in existing keys too

[XAMPPRocky]

tbh I wish pontoon's notifications could catch changes in existing keys too

I was also thinking this, it should be feasibly possible if it used git to know when those keys in those files have updated. I'd file an issue but I couldn't figure out how to correctly search their bug tracker.

Pinging people on Zulip seems reasonable to me. Should we ping @everyone in #t-community/l10n or should we create a new user group of translators?

[Manishearth]

Either works I think.

[Manishearth]

Probably @here though

[upsuper]

I'm not sure whether that would work... That would require there is something in each localization team check that channel regularly. Not sure whether it's the case currently. (At least I don't, but probably someone else in the SC team does.)

[skade]

The recommendation I got from Alexander Hecht (i18n) was indeed that substantial changes should be done under a new key, so I believe that should be the way to go.

The reasoning was exactly that in this case, i18n teams will see new keys popping up.

[Manishearth]

Yeah so I think what we need to do for generated files is simply to add a new dummy key that we keep updating that says "please check the following keys and retranslate" whenever a key is edited

[upsuper]

That could add quite a bit of work if the strings aren't actually changed. Not sure whether Pontoon can find old translation with identical strings to make it easier.

[Manishearth]

@upsuper this is only for the autogenerated keys

[upsuper]

It seems to me that there are quite a bit of strings with autogenerated keys, especially the description of teams.

[XAMPPRocky]

I was thinking what if going forward every time we update the teams.ftl, we also include an extra key pointing to the PR. So then the translator reviews the PR on GitHub.

Example

team-ftl-5cbd1e = Please review changes in https://github.com/rust-lang/www.rust-lang.org/pull/1030

[Manishearth]

@upsuper yes, but they're all in teams.ftl.

@XAMPPRocky's suggestion is what i'm suggesting, yes.

[upsuper]

If it's easy to pick old translations easily in Pontoon then I guess that would be fine.

[upsuper]

Just found another PR #1004 which changes the string without picking a different key (when translating other strings).