Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Translation improvements: CLDR Plurals + Translatables #19916

Closed
wants to merge 27 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
2b187d2
Add translatable interface
zeripath Jun 7, 2022
897f7aa
Add plural support using CLDR data
zeripath Jun 7, 2022
dc3d86f
Merge remote-tracking branch 'origin/main' into translation-improvements
zeripath Jun 27, 2022
6f1dde2
Merge remote-tracking branch 'origin' into translation-improvements
zeripath Jul 9, 2022
e30503e
reduce copying
zeripath Jul 9, 2022
4cca163
Merge remote-tracking branch 'origin' into translation-improvements
zeripath Jul 10, 2022
b26dcf0
use assign not :=
zeripath Jul 10, 2022
08cb842
update templates to use TrPlural
zeripath Jul 10, 2022
2fc923a
placate lint
zeripath Jul 10, 2022
cf0341f
placate lint 2
zeripath Jul 10, 2022
bf18312
Merge remote-tracking branch 'origin/main' into translation-improvements
zeripath Sep 1, 2022
09272c4
Use go templates instead.
zeripath Sep 1, 2022
928056d
placate lint
zeripath Sep 2, 2022
858b4cb
Merge remote-tracking branch 'origin/main' into translation-improvements
zeripath Sep 3, 2022
99f39aa
generate plural forms at time of locale reading
zeripath Sep 3, 2022
859d291
Merge branch 'main' into translation-improvements
zeripath Sep 3, 2022
492f0fc
Merge branch 'main' into translation-improvements
zeripath Sep 6, 2022
6f1dfb2
Merge branch 'main' into translation-improvements
zeripath Oct 5, 2022
18b8724
Merge branch 'main' into translation-improvements
zeripath Oct 7, 2022
eb28e9a
placate lint
zeripath Oct 8, 2022
62c37b3
Merge remote-tracking branch 'origin/main' into translation-improvements
zeripath Oct 8, 2022
4f023fa
also ensure generate creates a file that placates lint
zeripath Oct 8, 2022
f57e059
Merge branch 'main' into translation-improvements
zeripath Oct 10, 2022
9165c2d
Update options/locale/locale_en-US.ini
zeripath Oct 13, 2022
5ec920f
Merge remote-tracking branch 'origin/main' into translation-improvements
zeripath Oct 26, 2022
2e0e6a6
as per wxiaoguang
zeripath Oct 26, 2022
ed2e265
update docs
zeripath Oct 26, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions docs/content/doc/features/localization.en-us.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,74 @@ After a translation has been accepted, it will be reflected in the main reposito
At the time of writing, this means that a changed translation may not appear until the following Gitea release.

If you use a bleeding edge build, it should appear as soon as you update after the change is synced.

## Plurals

Prior to version 1.19, Gitea handled plurals using the .TrN function which has some
built in rules for managing plurals but is unable to properly all languages.

From 1.19 we will migrate to use the CLDR formulation.

Translation keys which handle plurals should be marked with a `_plural` suffix. This
will allow autogeneration of the various forms using go templates, e.g.

```ini
form.reach_limit_of_creation_plural = You have already reached your limit of %d {{if .One}}repository{{else}}repositories{{end}}.
```

Only the `form` is provided to this template - not the operand value itself. This is to allow autogeneration of the forms at time of loading.

These will be compiled to various different keys based on the available formats in the
language. e.g. assuming the above key is in the English locale it becomes:

```ini
form.reach_limit_of_creation_plural_one = You have already reached your limit of %d repository.
form.reach_limit_of_creation_plural_other = You have already reached your limit of %d repositories.

```

If the template format is too cumbersome forms can be directly created and they will
be used in preference to the template generated variants.

These keys should be used with the `.TrPlural` function. (Ordinals can be handled with `.TrOrdinal`.)

Each language has different numbers of plural forms, in English (and a number of other
languages) there are two plural forms:

* `One`: This matches the singular form.
* `Other`: This matches the plural form.

Other languages, e.g. Mandarin, have no plural forms, and others many more.

The possible forms are:

* `Other` - the most common form and will often match to standard plural form.
* `Zero` - matches a zeroth form, which in Latvian would match the form used for 10-20, 30 and so on.
* `One` - matches the singular form in English, but in Latvian matches the form used for 1, 21, 31 and so on.
* `Two` - matches the dual form used in for example Arabic for 2 items, but also more complexly in Celtic languages.
* `Few` - matches the form used in Arabic for 3-10, 103-110, and the ternary form in Celtic languages. In Russian and Ukranian for 2-4, 22-24.
* `Many` - matches the form used for large numbers in romance lanaguages like French, e.g. 1 000 000 *de* chat*s*, but in Russian and Ukranian it handles 0, 5~19, 100, 1000 and so on.

Some plural forms are only relevant if the object being counted is of a certain
grammatical gender or in certain tenses. Write your translation template appropriately to take account of this using `not` or `and` as appropriately.

Translators may want to review the CLDR information for their language or look at
`modules/translation/i18n/plurals/generate/plurals.xml`.

Ordinal forms, e.g. 1st, 2nd, 3rd and so on can be handled with `.TrOrdinal`. These
have the same forms as the plural forms, and we will use `_ordinal` as a base suffix
in future.

### Technical details

The following is technical and is provided to aid understanding in cases of problems only. Only use `.TrPlural` (and `.TrOrdinal`) with translation keys that have the suffix `_plural` (or `_ordinal`.) If you do not the specific per plural forms must be provided explicitly in the locale file. In this case keys for plural forms will be searched for in the following hierarchy:

1. `${key}_${form}` in the locale.
2. `${key}_other` in the locale.
3. `${key}` in the locale.
4. `${key}_${form}` in the default locale.
5. `${key}_other` in the default locale.
6. `${key}` in the default locale.
7. Use the string `${key}_${form}` directly as the format.

You do not have to worry about this if the key has the `_plural` (or `_ordinal`) suffix as the correct keys will be created automatically.
3 changes: 3 additions & 0 deletions modules/charset/escape_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,9 @@ type nullLocale struct{}
func (nullLocale) Language() string { return "" }
func (nullLocale) Tr(key string, _ ...interface{}) string { return key }
func (nullLocale) TrN(cnt interface{}, key1, keyN string, args ...interface{}) string { return "" }
func (nullLocale) HasKey(_ string) bool { return false }
func (nullLocale) TrPlural(cnt interface{}, key string, args ...interface{}) string { return key }
func (nullLocale) TrOrdinal(cnt interface{}, key string, args ...interface{}) string { return key }

var _ (translation.Locale) = nullLocale{}

Expand Down
8 changes: 8 additions & 0 deletions modules/csv/csv_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -561,6 +561,14 @@ func (l mockLocale) TrN(_cnt interface{}, key1, _keyN string, _args ...interface
return key1
}

func (l mockLocale) TrPlural(_cnt interface{}, key string, _args ...interface{}) string {
return key
}

func (l mockLocale) TrOrdinal(_cnt interface{}, key string, _args ...interface{}) string {
return key
}

func TestFormatError(t *testing.T) {
cases := []struct {
err error
Expand Down
12 changes: 12 additions & 0 deletions modules/test/context_tests.go
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,10 @@ func (l mockLocale) Language() string {
return "en"
}

func (l mockLocale) HasKey(_ string) bool {
return false
}

func (l mockLocale) Tr(s string, _ ...interface{}) string {
return s
}
Expand All @@ -107,6 +111,14 @@ func (l mockLocale) TrN(_cnt interface{}, key1, _keyN string, _args ...interface
return key1
}

func (l mockLocale) TrPlural(_cnt interface{}, key string, _args ...interface{}) string {
return key
}

func (l mockLocale) TrOrdinal(_cnt interface{}, key string, _args ...interface{}) string {
return key
}

type mockResponseWriter struct {
httptest.ResponseRecorder
size int
Expand Down
Loading