Consider other storage formats

[kant2002]

Approach which you show is invaluable for translation efforts. Do you consider having store data in mo or maybe have source gen over Xliff. Microsoft introduce in Net 7 NativeAOT and that scenario works not very well with Assembly resources, so I thinking about some other ways of storing/managing resources. Does that align with goals of this project?

[earloc]

Theres an ongoing discussion to support *.po-Files as an alternative to *.resx-files over at #38. But that´s in a very early design-stage.

The main goal of TypealizR was to generate accessors over those untyped strings. But I´m open for improvments or alternate ways to achieve something similar utilizing some other "storage".

I´d suggest that you might outline something (in pseudo-code).

Examples would be a *.xliff-file with some samples, and then some expected code that should be generated (can also be just "client"-code: how are you expecting to actually USE the generated code).

From there on, we could ellaborate a possible solution.

[earloc]

btw, if your on Windows, you might as well have a look at
https://learn.microsoft.com/en-us/windows/apps/design/globalizing/use-mat

[earloc]

btw, adopting another storage-mechanism would probably best be done by implementing some custom type which implements IStringLocalizer<T>. The existing infrastructure around Microsoft.Extension.Localization is designed to be pluggable - as already proved f. e. by those OrchardCore-devs.
This could be slightly out-of-scope for TypealizR. Only the generator part really fits here.

But I'm really open for exploring this topic to further drive the whole i18n field in .net-land as a whole.

So if you've find the time, I'll be thankful for your thoughts and'll be glad to help ✌️.

[kant2002]

I'm really looking for a way to future write localization for my self, so I do not have "answer" here.
Let me dump some thoughts which I have about the process.

I see two school of managing translation strings - first is just write strings in English and let other translate things, second is to have unique codes which represent phrase. My understanding that this library and you is in the first camp. I would not try to preset one way as better then others, as market show both has their cases. Personally I'm in second camp, so that definitely affect how I thought about process. Also I think ResX is not very well suited for translation, and PO/XLIFF is much better alternative. I would use XLIFF as example here, but I think same can be applied.

As developer I would like to have XLIFF files in the project files instead of ResX files, and let some tool generate typed access to the stings. I see this library as thin layer over translation mechainsm. Translation mechanism used then consult storage layer where he actually read underlying strings.

Let's say we have translation mechanism based on IStringLocalizer

Source XLIFF

// Website.xliff
<trans-unit id="HomePageHeader" datatype="plaintext">
  <source>Checkout TypealizR and make you localization better!</source>
  <note>Header on the main website page</note>
</trans-unit>
<trans-unit id="HomePageHeaderLine2" datatype="plaintext">
  <source>Unlock unparalleled development speed for quality translation!</source>
</trans-unit>

or XLIFF 2.0

// Website.xliff
<unit id="HomePageHeader">
   <notes>
      <note category="instruction" id="n1">The translation should be formal</note>
      <note category="comment" id="n2">Header on the main website page</note>
   </notes>
   <segment>
      <source>Checkout TypealizR and make you localization better!</source>
   </segment>
</unit>
<unit id="HomePageHeaderLine2">
   <segment>
      <source>Unlock unparalleled development speed for quality translation!</source>
   </segment>
</unit>

Generated code

// TypealizR code
internal class WebsiteLocalization  {
    protected Microsoft.Extensions.Localization.IStringLocalizer Localize { get; }
    public Greet_LoverExtensions (Microsoft.Extensions.Localization.IStringLocalizer localize)
    {
        this.Localize = localize;
    }

    /// <summary>
    /// Header on the main website page
    /// Returns string which translates "Checkout TypealizR and make you localization better!"
    /// </summary>
    public LocalizedString CheckoutTypealizRAndMakeYouLocalizationBetterLocalizedString() => Localize["HomePageHeader"];

    /// <summary>
    /// Header on the main website page
    /// Returns string which translates "Checkout TypealizR and make you localization better!"
    /// </summary>
    public string CheckoutTypealizRAndMakeYouLocalizationBetter() => CheckoutTypealizRAndMakeYouLocalizationBetterLocalizedString().Value;

    /// <summary>
    /// Returns string which translates "Unlock unparalleled development speed for quality translation!"
    /// </summary>
    public LocalizedString UnlockUnparalleledDevelopmentSpeedForQualityTranslationLocalizedString() => Localize["HomePageHeaderLine2"];

    /// <summary>
    /// Returns string which translates "Unlock unparalleled development speed for quality translation!"
    /// </summary>
    public string UnlockUnparalleledDevelopmentSpeedForQualityTranslation() => UnlockUnparallelledDevelopmentSpeedForQualityTranslationLocalizedString().Value;
}

Storage and translation mechanism

At this point TypealizR should not care who provide IStringLocalizer to it. Technincally would be great if IStringLocalizer would be technical detail which can be swapped to some other well-known mechanism. Maybe that's task for the future. Not important really.

Other point which I think important for me, is to have function names generated not from source string, but from string identifier.

// TypealizR code
internal class WebsiteLocalization  {
    protected Microsoft.Extensions.Localization.IStringLocalizer Localize { get; }
    public Greet_LoverExtensions (Microsoft.Extensions.Localization.IStringLocalizer localize)
    {
        this.Localize = localize;
    }
     /// <summary>
    /// Header on the main website page
    /// Returns string which translates "Checkout TypealizR and make you localization better!"
    /// </summary>
    public LocalizedString HomePageHeaderLocalizedString() => Localize["HomePageHeader"];

     /// <summary>
    /// Header on the main website page
    /// Returns string which translates "Checkout TypealizR and make you localization better!"
    /// </summary>
    public string HomePageHeader() => HomePageHeaderLocalizedString().Value;

    /// <summary>
    /// Returns string which translates "Unlock unparalleled development speed for quality translation!"
    /// </summary>
    public LocalizedString HomePageHeaderLine2LocalizedString() => Localize["HomePageHeaderLine2"];

    /// <summary>
    /// Returns string which translates "Unlock unparalleled development speed for quality translation!"
    /// </summary>
    public string HomePageHeaderLine2() => HomePageHeaderLine2LocalizedString().Value;
}

Advanced features should be ignored initially, just taked into account during design, but not implemented, otherwise it would be hard to agree on things IMO.

Content of translated strings should have format of library which will perform actual reading/translation. TypealizR should ignore it.

Content of source string should have either TypealizR or format of library which perform actual reading/translation. For example maybe it would be wise teach TypealizeR about %d, %s and other things to some degree.

[earloc]

plz also have a look at #71

[earloc]

I think #72 will lay a good foundation to get better namings in identifier-keyed resources.

Will explore a POC for using a *. xliff file as an input, instead of *. resx.

But I really have the feeling that a custom StringLocalizer sitting on top of a single xlif- file will have some clever caching-mechanisms to do, as those tend to be not very small, I guess...

[earloc]

Wow, that´s some vital feedback up there! Thx for your effort.

I´ll suggest to use the following terms to distinguish between the two camps you mentioned

• phrase-keyed
• identifier-keyed

Despite the fact that most of my samples are phrase-keyed, I personally don´t have a very strong opinion about which one might be better. That´s why TypealizR is completely un-opiniated about that. It just takes those resource-identifiers and generates method-names from them.

The main goal in this approach was to provide an easy upgrade- /refactoring-path while adopting usage of TypealizR in existing code-bases, which already utilize the IStringlocalizer-way of accessing resources.. That´s the primary reason why parameters for strings currently need to be specified within the resources identifier itself, rather than only in the translation-string:

Second reason is, that different translations could choose to omit certain parameters from the actual translation-string (even true for the default translation).

I still think this is a very good approach on how to "declare" parameters needed to be specified when accessing a specific translation-string. And that´s also where one can "tell" TypealizR what actuall type a parameter is expected to be.

So we both agree, that the generated method-names should be based on a resource´s identifier (which is already true for the resx-based generator.

That said, I think extending the source-generator to swoop over *.xliff-files really sounds not that difficult.

Only thing I´m currently missing is which, if any, Type would be needed to generated / reused for use of T in IStringLocalizer<T> when generating extensions-methods.

When we could sort that out, I think providing a *.xliff-based generator is not that hard to implement right now.

If that´s not an issue and we just could generate code which utilizes the non-generic version of IStringLocalizer, I guess a first working prototype could be done in a couple-o-days (when I find the time to dive into the code ;) )

[kant2002]

Okay, so you propose identifier to have type annotations. That sounds good, but would not work for XLIFF, because id attribute should be NMTOKEN according to spec https://docs.oasis-open.org/xliff/xliff-core/v2.0/os/xliff-core-v2.0-os.html#id
and NMTOKEN can be only letters, digits, periods (.), hyphens (-), underscores (_), and colons (:). http://www.datypic.com/sc/xsd/t-xsd_NMTOKEN.html

I think we can ask somebody who want types to provide something in the <note> element, but that maybe complicate things. If using <source> tag then we are in the phrase-keyed which I don't like to be only option.

Only thing I´m currently missing is which, if any, Type would be needed to generated / reused for use of T in

My understanding that current process use type names as way to located .resources file inside assembly. If we say that any IStringLocalizerFactory implementation which would be used should pack translation to XLIFF in such way that call to IStringLocalizerFactory.Create(Type) would result in proper IStringLocalizer<T> then magically things start working, so burden should be on party who package XLIFF files. I think that's reasonable responsibility split.

[earloc]

Thought so that this might be a problem. Nevertheless, this is just a minor issue imo, as we´d just need to use another char instead of { and } to seperate parameters from each other.

(See #15 for currently supported notations)

Consider the following proposed xliff-file:

<trans-unit id="HomePageHeader_userName:s_" datatype="plaintext">
  <source>Checkout TypealizR and make you localization better!</source>
  <note>Header on the main website page</note>
</trans-unit>
<trans-unit id="HomePageHeaderLine2_now:dt_" datatype="plaintext">
  <source>Unlock unparalleled development speed for quality translation!</source>
</trans-unit>

Could be generated as (pseudo-code)

• HomePageHeader(string userName)
• HomePageHeaderLine2(DateTime now)

[earloc]

Even #14 could easily be adopted to be compatible to NMTOKEN
-> See https://github.com/earloc/TypealizR#groupings

[earloc]

Might be worth having a look at
https://github.com/dotnet/xliff-tasks

[earloc]

I guess using an *. xliff-file to populate extension-methods really doesn't need to leverage IStringLocalizer<T>. The whole purpose of that T is mainly to seperate strings into some sort of "bounded-context", f. E. a discrete RazorPage or an ApiController, limiting the amount of strings in that single file.

A *. xliff-file, on the other hand, contains all strings from one application and therefore could just be generated as extensions for the non-generic IStringLocalizer. TypealizR could shine here by providing the grouping-capability, to get things organized.

I'll have another look at those files that get generated by MultilingualAppToolkit. Those carry at least some infos about the originating resx-files (just to get an idea, where we might place Type-Information, if that turns out to be the better approach)

[earloc]

@kant2002 Had some thoughts about implementing things here. One thing I really cannot wrap my head around is the fact that XLIFFs is designed as an interchange format, rather than a pure storage format used at runtime.

I mostly wonder where you´d plan to get such files from, in the first place. Surely not willing to write those by hand, aren´t you?
Am I missing some tool, which might come in handy here?
All tools I know are based on populating an XLIFF, which then gets passed around and be translated, and ultimatly get re-integrated within the real storages, which can be highly specific to the used tech-stack (like resx in .net-land, or something Angular does with it).

[kant2002]

I think about XLiff same was as I think about REsX. I use it for translation work, not for actual storage.

[earloc]

Hmmm, but at least Resx (or its UWP-sibling ResW) is intended to be a storage mechanism 🤔.

But OK, let's step back a bit. What kind of file (and how) would you use to introduce translations? What should be the input?

[earloc]

@kant2002

I´m starting to think into a whole other direction:
#78

[earloc]

I think about XLiff same was as I think about REsX. I use it for translation work, not for actual storage.

@kant2002 so what do you use then as an actual storage?

[kant2002]

For now I probably go with easiest route by using resources and embed them into assembly, but most likely I would look for other ways which will works for NativeAOT. maybe .mo files. That would be custom solution, but that's on me