Translations

[Rich-Harris]

Is your feature request related to a problem? Please describe.
Translations are largely an unsolved problem. There are good libraries out there, like svelte-i18n and svelte-intl-precompile, but because they are not tightly integrated with an app framework, there are drawbacks in existing solutions:

• Usage is somewhat verbose (e.g. {$_("awesome", { values: { name: "svelte-i18n" } })}) from the svelte-i18n docs
• Loading translations is generally a bit awkward
• Messages are untyped

Without getting too far into the weeds of implementation, what follows is a sketch for what I think the developer experience could and should look like for using translations in SvelteKit. It builds upon the aforementioned prior art (using ICU MessageFormat, precompiling translations etc) while leveraging SvelteKit's unique position as an opinionated app framework. It follows the discussion in #553, but doesn't address issues like picking a locale or handling canonical/alternate URLs, which overlap with translations but can be designed and implemented separately to a large extent.

Describe the solution you'd like
In the translations directory, we have a series of [language].json files with ICU MessageFormat strings:

// en.json
{
  "brand": {
    "name": "SvelteKit",
    "tagline": "The fastest way to build Svelte apps",
    "description": "SvelteKit is the official Svelte application framework"
  },
  "greeting": "Welcome!",
  "clicked_n_times": "{n, plural,=0 {Click the button} =1 {Clicked once} other {Clicked {n} times}}"
}

Regional dialects inherit from the base language:

// en-AU.json
{
  "greeting": "G'day mate!"
}

These files are human-editable, but could also be manipulated by tooling. For example, the SvelteKit CLI could provide an editor along the lines of this one or this one out of the box.

SvelteKit watches these files, and populates an ambient.d.ts file that lives... somewhere (but is picked up by a default SvelteKit installation) with types based on the actual translations that are present (using the default locale, which could be specified in svelte.config.cjs, for the example strings):

declare module '$app/i18n' {
  import { Readable } from 'svelte/store';

  /**
   * A dictionary of translations generated from translations/*.json
   */
  export const t: Readable<{
    brand: {
      name: 'SvelteKit';
      tagline: 'The fastest way to build Svelte apps';
      description: 'SvelteKit is the official Svelte application framework'
    };
    greeting: 'Hello!',
    clicked_n_times: (n: number) => string;
  }>;
}

As well as generating the types, SvelteKit precompiles the strings into a module, similar to svelte-intl-precompile:

export default {
  brand: {
    name: 'SvelteKit',
    tagline: 'The fastest way to build Svelte apps',
    description: 'SvelteKit is the official Svelte application framework'
  },
  greeting: 'Hello!',
  clicked_n_times: (n) => n === 0 ? 'Click the button' : n === 1 ? 'Clicked once' : `Clicked ${n} times`
};

In a large enough app this module could get very large. We could theoretically use static analysis to determine which translations are used by which routes, and only load the translations needed for the current route, but I don't think that's necessary as a starting point.

This module is using for server-rendering, and is also loaded by the client runtime to populate the t store. (Because it's a store, we can change the language without a full-page reload, if necessary. That might be overkill, but if Next can do it then we should too!)

A relatively unique thing about this approach is that we get typing and autocompletion:

This extends to parameters (which I think should be positional), for messages that need them:

Because everything is just JSON files, it's trivial to build tooling that can e.g. identify untranslated phrases for a given language. I don't know what common translation workflows look like, but it would presumably be possible to convert between the file format used here and the output of translation software that uses ICU.

Describe alternatives you've considered

• Leaving it to userland. I think i18n is too central to be treated as not-our-problem, and the potential ergonomic benefits from having it in the framework are considerable. (Our solution need not preclude userland solutions, if people have strong preferences)
• Using a different message format. Fluent and Banana both came up in i18n brainstorming #553. We could theoretically support multiple formats but it would be preferable to pick one that most people are happy with

How important is this feature to you?
It's time. Though we need to solve the problem of selecting a locale before we can make much of a start on this; will open an issue in due course.

[babakfp]

@Rich-Harris
Hi, How can I contact you on Discord or Twitter? I couldn't find any way so I decided to message you here.

[pago]

Could you clarify this part a little bit?

This extends to parameters (which I think should be positional), for messages that need them

The part I'm wondering about is whether that could result in changes to the translation messages invalidating existing code due to a shift of parameter order.
Apart from that potential risk I also feel like having an object with keys and values might actually add to the readability of the code.

Quick example:

$t.clicked_n_times(n);
// compared to
$t.clicked({times: n});

Everything else sounds outstanding and I can't wait to start using it. Thank you for tackling this issue.

[tempo22]

Hi,

Allow me to jump into the discussion here. I'm planning a new project and a robust i18n is a must-have on it.

I have worked in the past with Drupal and Wordpress websites, both using po file format. It seems to be made for this purpose. We may use JSON but it would be nice to see how they handle the pluralization and other translation workflows.

But as you said, selecting a locale is an important step that should be done before loading translation.

[floratmin]

I have just written something to extract translation functions calls into .po files. Currently, I am integrating it into a rollup plugin. My idea is to use the tooling of gettext for managing the translation lifecycle. I think that we could even use ICU messageformat strings inside .po files. The advantage of .po files is that it is easy to provide a lot of extra context, which is often very important for translators. You can check my repository out at gettext-extractor-svelte.

Then there is the package gettext-to-messageformat which converts .po files with gettext format to messageformat. If instead ICU messageformat strings are used in .po files, the conversion is trivial. An option to use json with ICU messageformat instead of .po files should also be easy to implement, but for this, I have to investigate how comments are provided to translators.

Then the messageformat project can compile the translated strings into pure functions which can then be injected when building the application and replace the translation function and function calls.

The actual translation function could be changed according to the needs of the project. Simple projects would only go with the message string, while other projects could provide unique context and detailed comments for each message string.

[zwergius]

Hey,

Here is what I use for selecting the initial language

import { supportedLanguages } from '$lib/constants';
const DOCUMENT_REGEX = /^([^.?#@]+)?([?#](.+)?)?$/;

export function getContext({ headers, path }) {
  const isDocument = DOCUMENT_REGEX.test(path);
  if (!isDocument) return;

  let language = path.split('/')[1];
  // language not in url
  if (supportedLanguages.indexOf(language) === -1) {
    language = supportedLanguages[0];

    if (headers['accept-language']) {
      const headerLang = headers['accept-language'].split(',')[0].trim().slice(0, 2);

      if (headerLang && headerLang.length > 1) {
        if (supportedLanguages.indexOf(headerLang) !== -1) language = headerLang;
      }
    }
  }

  return { language };
}

export function getSession({ context }) {
  const { language } = context;
  return { language };
}

export async function handle({ request, render }) {
  const rendered = await render(request);

  if (rendered.headers['content-type'] === 'text/html') {
    const { language } = request.context;

    return {
      ...rendered,
      body: rendered.body.replace('%lang%', language)
    };
  }

  return rendered;
}

I use the accept-language header and then in my src/routes/index.svelte I have

<script context="module">
  export async function load({ session }) {
    return {
      status: 303,
      redirect: `/${session.language}`
    };
  }
</script>

I then in src/routes/[lang]/$layout.svelte check if the lang param is supported and if not 404

I than have a static yaml file that holds all translations

This works fairly well for static pages which is what I do mostly... I put this together after reading a whole bunch of issues, initially trying svelte-i18n which I then decided was overkill...

[kobejean]

I've used svelte-i18n before and one big reason I prefer the $t('text') format over $t.text is that it is supported in this vs code plugin: i18n-ally . It's a great way to manage your translations, have translation reviews/feedback and it even allows machine translation.

[babakfp]

See what is wrong with svelte-i18n

An example from svelte-i18n

<script>
  import { _ } from 'svelte-i18n'
</script>

<h1>{$_('page.home.title')}</h1>

<nav>
  <a>{$_('page.home.nav', { default: 'Home' })}</a>
  <a>{$_('page.about.nav', { default: 'About' })}</a>
  <a>{$_('page.contact.nav', { default: 'Contact' })}</a>
</nav>

I think this is a 100% bull sheet. Look at all of the work that you need to do, just to set a default value for a translateable value. You are not done, you also need to create a .json file and write you all strings there. Also, you need to pick a name for your translatable string as a reference. for example:

// en.json
{
  "hello_world": "Hello, World!"
}

So what is wrong with svelte-i18n?

1. It's a lot of wooooooooooooooooooork.
2. It's unnecessarily a lot of wooooooooooooooooooork.
3. It's just nonsense.
4. The $ character is here too!!. I hate this, but no worries, because of my PHP background experience I'm immune 😜

I know that this was the best possible way for them to create this library. I know that they put in a lot of work and they did what they were possible of doing it.

I explained what I expect from a layout system here and nothing, we are going to experience the same problems in the SvelteKit too. Now I want to explain what I expect from a translation system because I need to say what I think about it and how I think it needs to be solved because if I don't, I think it's gonna be really bad 😞.

Also, I want to say that I get banned (2 of my accounts) from the community discord group. Admins were like "Nah, I don't like this guy, clicking on the ban button". (The person that reading this, if you get banned from the discord channel, knows that there was no problem with you). If you didn't like what I explain here, you can stick with this subject that 2 of my accounts get banned, instead of saying what you don't agree with 😄. I wanted to complain about it with Mr. Rich Harris but, it seems he is so busy.
Oh God, now I feel like I complaining like a baby 😆

Let's keep going

What I expect from a translation system?

Exactly the same that we are using in the current tools. For example Wordpress:

<?php echo __('Hello, World!'); ?>

I sawed the same syntax in Laravel but I don't have a lot of information about it. Any Laravel developers here to explain to us about how it worlds and how was the experience?

With this, you don't need to write all of those svelte-i18n junk you sowed above!.

How it works?

Automatically the strings inside the functions get write inside the .pot file. You insert the .pot file inside a software called Poedit. You select the language and then Poedit will show you the all translatable strings. Translate the strings that you see and much more features, that will make life easier for you, rather than creating a .json file and blah blah blah.

You are gonna have 3 files

• .pot: Your all strings live here (input). translatable-strings.pot.
• .po: Translated string (output). [language].po.
• .mo Same as the .po but the file isn't editable!. [language].mo.

You are confused?

You are just gonna write your string inside the __() function. All of the other work going to take care of by the system it selves.

Pros

• Much less code.
• Much fewer problems.
• Easier life ;)
• A software to help you easily edit your strings. Click on a button and google translate will translate it all for you!.
• Don't need to learn new stuff.
• It is the most known syntax for translations = {Count how much people uses Wordpress} + {Count how much people uses Laravel}.
• We do not need to reinvent the wheel!.
• $t(), oh my brain 😟
• This will provide the solution for: Translations #1274 (comment)

Cons

• If you edit the initial / the source string, you will lose all the translations related to that. For example: before: Hello world and after: Hello World.
• The Hello string and the hello string are treated as different strings.

Why I hate $t()

• $t(), oh my brain 😟
• why not just use the initial string as the key? Use the initial string Welcome instead of greeting. Translations #1274 (comment)
• Don't like the $
• Why reinvent the wheel!.

[kvetoslavnovak]

Big THANK YOU to all dealing with the translation topic.
i18n is a huge “must” for all non-US regions. E.g. for us in EU especially.

[Kapsonfire]

@babakfp i disagree with the point of you have to define keys and default translations
you can directly use

{$_('Won {n, plural, =0 {no awards} one {# award} other {# awards}}', { values: { n: 2}})}

no need to define a key and define a default translation - same behaviour as gettext

[tempo22]

@babakfp i disagree with the point of you have to define keys and default translations
you can directly use

{$_('Won {n, plural, =0 {no awards} one {# award} other {# awards}}', { values: { n: 2}})}

no need to define a key and define a default translation - same behaviour as gettext

The plural handling in your example will create a mess where the sentence order is totally different depending on the language:
I won 1 award -> J'ai gagné 1 prix
I won 2 awards -> J'ai gagné 2 prix
I won no awards -> Je n'ai gagné aucun prix

In PO files, the plural is handled with the following syntax

msgid "I won {n} trip"
msgid_plural "I won {n} trips"
msgstr[0] "J'ai gagné {n} voyage"
msgstr[1] "J'ai gagné {n} voyages"

The number of plural variant depend on the language, read this for more informations

Using a more complex example:

{#if n == 0 }
   {$t('Hello {firstname}, you have no new messages', {firstname: user.firstname})}
{:else}
   {$t('Hello {firstname}, you have {n} new message', 'Hello {firstname}, you have {n} new message', n, {n: n, firstname: user.firstname})}
{/if}

we could have the translation function with the following definition:

function t(msgid: string, replacements?: object): string
function t(msgid: string, msgidPlural: string, count: number, replacements?: object): string

[Kapsonfire]

@tempo22 it's the ICU MessageFormat
You gonna create a fitting translation string for french in your translation files.

[floratmin]

@Kapsonfire-DE ICU messageformat can get very similar to gettext:

{
  $_(
    `{
      n, plural, 
      =0 {I won no awards}
      one {I won one award} 
      other {I won # awards}
    }`,
   { n: 2 }
  )
}

This is also the recommended way for writing messageformat strings. Otherwise, an external translator can get confused.

I think that some have the perspective of translating their app/page on their own or in house. But we should not forget, that there are also many use cases where we need professional translations and where we need to provide as much context to the translators as possible. Because of this, I like gettext (.po, .pot, .mo) very much. There are a lot of editors, there is an established workflow and many translators know this system.

But messageformat is sometimes more flexible, especially you can use the Intl functions directly, which avoids a lot of bloat. And because messageformat are strings we can put these strings also into .po files and use the gettext workflow and infrastructure. If the context is not needed, we can put these strings still into JSON or YAML files and use simpler functions.

An other case is when there is already a project which is based on gettext. Then it can make very much sense to use gettext instead of messageformat, because all translations from before can be reused. So it would be nice if there is some flexibility.

[johnnysprinkles]

More important than the specific syntax is that we understand the typical flow of translations... I can describe how it works at Google for example, which is probably similar to other large tech companies.

Translations are a continuous and ongoing process. Commits land constantly and most of them have new strings, then in a periodic process we extract all the untranslated strings and send them off for translation. Google and Amazon I know have "translation consoles" that manage it for you. So say weekly you pull the strings and send them off, then they come back a week or more later, you check that data in and it's available after your next deployment.

One thing to note is that, if you're using human translators, you must have a description. The English text alone isn't enough to get a good translation. Not sure how this changes as we move more and more to machine translation. Early on at Google I worked on a GWT project where the translations were declared in code like this:

<ui:msg description="Greeting">Hello, world</ui:msg>

But most stuff at Google uses goog.getMsg(), i.e.

@desc Greeting
const MSG_GREETING = goog.getMsg('Hello, world');

What Rich is describing with a key and all versions of the text including the primary English version located elsewhere in a data file, that sounds good for the "glossary of well known terms you want to use across the site" scenario, but the vast majority of strings we use are translated per instance, since their meaning and context varies. Trying to share strings to save on translation costs would just slow down development too much. Maybe smaller projects have different priorities of course though.

So I'd push for the English text (or whatever the primary language is) and the description located in the Svelte component itself. That way it works immediately and works even if you lose your translation data, and you don't have the Svelte component author having to hand fill-in a data file for one language while all the other languages come from an automated process.

[Rich-Harris]

Thanks @johnnysprinkles — I haven't experienced this sort of translation workflow so that's useful.

I'm not wholly convinced that including the primary language in the component alongside a description is ideal though:

• Parsing out the language from the component source adds significant complexity. The case where you have a piece of markup like <ui:msg description="Greeting">Hello, world</ui:msg> is one thing, but if you want to be able to refer to translations in JavaScript as well, you have to have some way of associating descriptions with variables, and the whole thing has to be amenable to static analysis. This would likely involve non-trivial changes to Svelte itself, even if we could come up with a design that made sense
• <h1>{t.greeting}</h1> is nicer than <h1><ui:msg description="Greeting">Hello, world</ui:msg></h1>. The latter adds a lot of distracting visual noise that I probably don't want when I'm building a UI
• It forces developers to be copywriters. Perhaps 'Hello, world' is the wrong tone of voice?
• It's brittle — if the text + description is essentially a 'key' for other translations, then we can't edit the primary language text, even to remove typos. Keys are a lot more stable
• It's harder to keep the original primary language strings out of bundles for all languages unless you have sophisticated tooling
• It forces you to use descriptions even if they're unnecessary for your workflow, because otherwise two strings that are the same in one language but context-dependent in another rely on the description for disambiguation:

  An example that I love to use is the term “Get started.” We use that in our products in a lot of places and, in American English, it’s pretty standard. It’s so understandable that people don’t even think of the fact that it can be used in three or four ways. It could be a call to action on a button. Like, “Get started. Click here.” It could be the title of the page that’s showing how you get started. It can be the name of a file: a Get Started guide PDF. All of those instances need to be translated differently in most other languages.

So that leaves the question 'how do we associate descriptions with keys'? I think that's actually quite straightforward, we just have a descriptions.json or whatever in the same places as the translations themselves:

// en.json
{
  "brand": {
    "name": "SvelteKit",
    "tagline": "The fastest way to build Svelte apps",
    "description": "SvelteKit is the official Svelte application framework"
  },
  "greeting": "Welcome!",
  "clicked_n_times": "{n, plural,=0 {Click the button} =1 {Clicked once} other {Clicked {n} times}}"
}

// description.json
{
  "brand": {
    "name": "The name of the project",
    "tagline": "The tagline that appears below the logo on the home page",
    "description": "A description of SvelteKit that appears in search engine results"
  },
  "greeting": "A message enthusiastically welcoming the user",
  "clicked_n_times": "The number of times the user has clicked a button"
}

I'm envisaging that we'd have some way to convert between this format and those used by common translation workflows (.po as mentioned above, etc), so that the descriptions are colocated with the primary language as far as translators are concerned. These descriptions would also become part of the type signature:

declare module '$app/i18n' {
  import { Readable } from 'svelte/store';

  /**
   * A dictionary of translations generated from translations/*.json
   */
  export const t: Readable<{
    brand: {
      /** The name of the project */
      name: 'SvelteKit';
      /** The tagline that appears below the logo on the home page */
      tagline: 'The fastest way to build Svelte apps';
      /** A description of SvelteKit that appears in search engine results */
      description: 'SvelteKit is the official Svelte application framework'
    };
    /** A message enthusiastically welcoming the user */
    greeting: 'Hello!',
    /** The number of times the user has clicked a button */
    clicked_n_times: (n: number) => string;
  }>;
}

[babakfp]

Hi
I have a question, how long it will take to implement it in this way? There are pros and cons of doing something right? What are the pros of doing it this way ".pot, .po, .mo"?

• It's going to be implemented sooner and easier.
  I'm not an expert but, I think that the hardest part, isn't the developing journey. The hard part is deciding on how the result gonna be like. It's really hard and frustrating to decide on this kinda stuff, especially if you don't have real experiments with it. You don't know what features gonna be needed, there are going to be tons of issues and etc.
• in comparison, there are going to be tons of tutorials, visual tools, and tons of people that know how to work with the .pot way.
• What about people that don't know how to write code? I bet on my life that they are not gonna like write code because they don't want to waste a lot of time, be frustrated and finally work with a text editor that doesn't provide you features like a visual translation tool does.
  There is no reason to reinvent the wheel, so please don't😨😅. Also, Thank you all for doing all that insane work💜

unrelated stuff

Is it possible to know about those tools and features status? like a daily blog or something? Maybe a Discord channel that just the maintainers can send voices, details, pulls, and etc? By the way, I created a fake account to join the Discord server😂 (feels like admitting the crimes to the cops😄). Thanks again for banning me👏, "You are the best" MR. pngwn#8431(if I'm not mistaken)😀