Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] add localized unit names #974

Closed
wants to merge 7 commits into from
Closed

[WIP] add localized unit names #974

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
414ed57
[WIP] add localized unit names
Oct 6, 2021
b54f9b8
[WIP] add localized unit names
Oct 7, 2021
476d89e
[FEAT] Localization of the unit singular names
Oct 8, 2021
038dce9
[TU] Add TU for testing the singular name localization
Oct 8, 2021
f6121f2
Add TU for UnitSingularNamesCacheTests
Nov 16, 2021
77e7f9c
FIX PR#974
Nov 16, 2021
f14edf9
PR#974 Remove default singular name for units
Nov 16, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions CodeGen/Generators/QuantityJsonFilesParser.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
// Licensed under MIT No Attribution, see LICENSE file at the root.
// Licensed under MIT No Attribution, see LICENSE file at the root.
// Copyright 2013 Andreas Gullberg Larsen (andreas.larsen84@gmail.com). Maintained at https://github.com/angularsen/UnitsNet.

using System;
using System.Collections.Generic;
using System.Globalization;
using System.IO;
using System.Linq;
using CodeGen.Exceptions;
Expand Down Expand Up @@ -104,20 +105,27 @@ private static void AddPrefixUnits(Quantity quantity)
}

/// <summary>
/// Create unit abbreviations for a prefix unit, given a unit and the prefix.
/// Create unit abbreviations, singular name and plural names for a prefix unit, given a unit and the prefix.
/// The unit abbreviations are either prefixed with the SI prefix or an explicitly configured abbreviation via
/// <see cref="Localization.AbbreviationsForPrefixes" />.
/// </summary>
private static Localization[] GetLocalizationForPrefixUnit(IEnumerable<Localization> localizations, PrefixInfo prefixInfo)
{
return localizations.Select(loc =>
{
var prefixName = prefixInfo.GetPrefixNameForCultureOrSiPrefix(loc.Culture);
var localizedSingularName = string.IsNullOrEmpty(loc.SingularName) ? string.Empty:$"{CultureInfo.CurrentUICulture.TextInfo.ToTitleCase((prefixName + loc.SingularName).ToLower().Trim())}";
var localizedPluralName = string.IsNullOrEmpty(loc.PluralName) ? string.Empty : $"{prefixName}{loc.PluralName}";

//specific abbreviations for prefixes (example: "kip/in³" for prefix "Kilo" of Density)
if (loc.TryGetAbbreviationsForPrefix(prefixInfo.Prefix, out string[] unitAbbreviationsForPrefix))
{
return new Localization
{
Culture = loc.Culture,
Abbreviations = unitAbbreviationsForPrefix
Abbreviations = unitAbbreviationsForPrefix,
SingularName = localizedSingularName,
PluralName = localizedPluralName
};
}

Expand All @@ -129,7 +137,9 @@ private static Localization[] GetLocalizationForPrefixUnit(IEnumerable<Localizat
return new Localization
{
Culture = loc.Culture,
Abbreviations = unitAbbreviationsForPrefix
Abbreviations = unitAbbreviationsForPrefix,
SingularName = localizedSingularName,
PluralName = localizedPluralName
angularsen marked this conversation as resolved.
Show resolved Hide resolved
};
}).ToArray();
}
Expand Down
57 changes: 57 additions & 0 deletions CodeGen/Generators/UnitsNetGen/UnitSingularNamesCacheGenerator.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
using System.Linq;
using CodeGen.JsonTypes;

namespace CodeGen.Generators.UnitsNetGen
{
internal class UnitSingularNamesCacheGenerator : GeneratorBase
{
private readonly Quantity[] _quantities;

public UnitSingularNamesCacheGenerator(Quantity[] quantities)
{
_quantities = quantities;
}

public override string Generate()
{
Writer.WL(GeneratedFileHeader);
Writer.WL(@"
using System;
using UnitsNet.Units;

// ReSharper disable RedundantCommaInArrayInitializer
// ReSharper disable once CheckNamespace
namespace UnitsNet
{
public partial class UnitSingularNamesCache
{
private static readonly (string CultureName, Type UnitType, int UnitValue, string[] Strings)[] GeneratedLocalizations
= new []
{");
foreach (var quantity in _quantities)
{
var unitEnumName = $"{quantity.Name}Unit";

foreach (var unit in quantity.Units)
{
foreach (var localization in unit.Localization)
{
var cultureName = localization.Culture;

// All units must have a unit abbreviation, so fallback to the default singular name if no localized name is defined in JSON
// var singularNames = string.IsNullOrEmpty(localization.SingularName) ? $"\"{unit.SingularName}\"" : $"\"{localization.SingularName}\"";
var singularNames = string.IsNullOrEmpty(localization.SingularName) ? "\"BOB\"" : $"\"{localization.SingularName}\"";
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BOB, is that a placeholder? Should we maybe use null or empty string instead?

Writer.WL($@"
(""{cultureName}"", typeof({unitEnumName}), (int){unitEnumName}.{unit.SingularName}, new string[]{{{singularNames}}}),");
}
}
}

Writer.WL(@"
};
}
}");
return Writer.ToString();
}
}
}
8 changes: 8 additions & 0 deletions CodeGen/Generators/UnitsNetGenerator.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ public static void Generate(string rootDir, Quantity[] quantities)
Log.Information("");
GenerateIQuantityTests(quantities, $"{testProjectDir}/GeneratedCode/IQuantityTests.g.cs");
GenerateUnitAbbreviationsCache(quantities, $"{outputDir}/UnitAbbreviationsCache.g.cs");
GenerateUnitSingularNamesCache(quantities, $"{outputDir}/UnitSingularNamesCache.g.cs");
GenerateQuantityType(quantities, $"{outputDir}/QuantityType.g.cs");
GenerateStaticQuantity(quantities, $"{outputDir}/Quantity.g.cs");
GenerateUnitConverter(quantities, $"{outputDir}/UnitConverter.g.cs");
Expand Down Expand Up @@ -130,6 +131,13 @@ private static void GenerateUnitAbbreviationsCache(Quantity[] quantities, string
Log.Information("✅ UnitAbbreviationsCache.g.cs");
}

private static void GenerateUnitSingularNamesCache(Quantity[] quantities, string filePath)
{
var content = new UnitSingularNamesCacheGenerator(quantities).Generate();
File.WriteAllText(filePath, content);
Log.Information("✅ UnitSingularNamesCache.g.cs");
}

private static void GenerateQuantityType(Quantity[] quantities, string filePath)
{
var content = new QuantityTypeGenerator(quantities).Generate();
Expand Down
56 changes: 56 additions & 0 deletions CodeGen/Generators/UnitsNetWrcGen/UnitSingularNamesCacheGenerator.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
using System.Linq;
using CodeGen.JsonTypes;

namespace CodeGen.Generators.UnitsNetWrcGen
{
internal class UnitSingularNamesCacheGenerator : GeneratorBase
{
private readonly Quantity[] _quantities;

public UnitSingularNamesCacheGenerator(Quantity[] quantities)
{
_quantities = quantities;
}

public override string Generate()
{
Writer.WL(GeneratedFileHeader);
Writer.WL(@"
using System;
using UnitsNet.Units;

// ReSharper disable RedundantCommaInArrayInitializer
// ReSharper disable once CheckNamespace
namespace UnitsNet
{
public sealed class UnitSingularNamesCache
{
private static readonly (string CultureName, Type UnitType, int UnitValue, string[] Strings)[] GeneratedLocalizations
= new []
{");
foreach (var quantity in _quantities)
{
var unitEnumName = $"{quantity.Name}Unit";

foreach (var unit in quantity.Units)
{
foreach (var localization in unit.Localization)
{
var cultureName = localization.Culture;

// All units must have a unit abbreviation, so fallback to the default singular name if no localized name is defined in JSON
var singularNames = string.IsNullOrEmpty(localization.SingularName) ? $"\"{unit.SingularName}\"" : $"\"{localization.SingularName}\"";
Writer.WL($@"
(""{cultureName}"", typeof({unitEnumName}), (int){unitEnumName}.{unit.SingularName}, new string[]{{{singularNames}}}),");
}
}
}

Writer.WL(@"
};
}
}");
return Writer.ToString();
}
}
}
10 changes: 9 additions & 1 deletion CodeGen/Generators/UnitsNetWrcGenerator.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Licensed under MIT No Attribution, see LICENSE file at the root.
// Licensed under MIT No Attribution, see LICENSE file at the root.
// Copyright 2013 Andreas Gullberg Larsen (andreas.larsen84@gmail.com). Maintained at https://github.com/angularsen/UnitsNet.

using System.IO;
Expand Down Expand Up @@ -52,6 +52,7 @@ public static void Generate(string rootDir, Quantity[] quantities)

Log.Information("");
GenerateUnitAbbreviationsCache(quantities, $"{outputDir}/UnitAbbreviationsCache.g.cs");
GenerateUnitSingularNamesCache(quantities, $"{outputDir}/UnitSingularNamesCache.g.cs");
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned, I believe you can revert this entire file. Windows Runtime Component build target is no longer receiving new features.

GenerateQuantityType(quantities, $"{outputDir}/QuantityType.g.cs");
GenerateStaticQuantity(quantities, $"{outputDir}/Quantity.g.cs");

Expand Down Expand Up @@ -80,6 +81,13 @@ private static void GenerateUnitAbbreviationsCache(Quantity[] quantities, string
Log.Information("✅ UnitAbbreviationsCache.g.cs (WRC)");
}

private static void GenerateUnitSingularNamesCache(Quantity[] quantities, string filePath)
{
var content = new UnitSingularNamesCacheGenerator(quantities).Generate();
File.WriteAllText(filePath, content);
Log.Information("✅ UnitSingularNamesCache.g.cs (WRC)");
}

private static void GenerateQuantityType(Quantity[] quantities, string filePath)
{
var content = new QuantityTypeGenerator(quantities).Generate();
Expand Down
13 changes: 12 additions & 1 deletion CodeGen/JsonTypes/Localization.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Licensed under MIT No Attribution, see LICENSE file at the root.
// Licensed under MIT No Attribution, see LICENSE file at the root.
// Copyright 2013 Andreas Gullberg Larsen (andreas.larsen84@gmail.com). Maintained at https://github.com/angularsen/UnitsNet.

using System;
Expand Down Expand Up @@ -43,6 +43,7 @@ public bool TryGetAbbreviationsForPrefix(Prefix prefix, out string[] unitAbbrevi
throw new NotSupportedException($"AbbreviationsForPrefixes.{prefix} must be a string or an array of strings, but was {value.Type}.");
}
}

// 0649 Field is never assigned to
#pragma warning disable 0649

Expand All @@ -51,6 +52,16 @@ public bool TryGetAbbreviationsForPrefix(Prefix prefix, out string[] unitAbbrevi
/// </summary>
public string[] Abbreviations = Array.Empty<string>();

/// <summary>
/// The unit singular name for this localization
/// </summary>
public string SingularName = string.Empty ;

/// <summary>
/// The unit plural name for this localization
/// </summary>
public string PluralName = string.Empty ;

/// <summary>
/// Explicit configuration of unit abbreviations for prefixes.
/// This is typically used for languages or special unit abbreviations where you cannot simply prepend SI prefixes like
Expand Down
66 changes: 42 additions & 24 deletions CodeGen/PrefixInfo.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,35 +10,36 @@ namespace CodeGen
/// </summary>
internal class PrefixInfo
{
private const string Russian = "ru-RU";
private const string Chinese = "zh-CN";
private const string French = "fr-FR";
private const string Russian = "ru-RU";

public static readonly IReadOnlyDictionary<Prefix, PrefixInfo> Entries = new[]
{
// Need to append 'd' suffix for double in order to later search/replace "d" with "m"
// when creating decimal conversion functions in CodeGen.Generator.FixConversionFunctionsForDecimalValueTypes.

// SI prefixes
new PrefixInfo(Prefix.Yocto, "1e-24d", "y",(Chinese, "夭")),
new PrefixInfo(Prefix.Zepto, "1e-21d", "z",(Chinese, "仄")),
new PrefixInfo(Prefix.Atto, "1e-18d", "a", (Russian, "а"),(Chinese, "阿")),
new PrefixInfo(Prefix.Femto, "1e-15d", "f", (Russian, "ф"),(Chinese, "飞")),
new PrefixInfo(Prefix.Pico, "1e-12d", "p", (Russian, "п"),(Chinese, "皮")),
new PrefixInfo(Prefix.Nano, "1e-9d", "n", (Russian, "н"),(Chinese, "纳")),
new PrefixInfo(Prefix.Micro, "1e-6d", "µ", (Russian, "мк"),(Chinese, "微")),
new PrefixInfo(Prefix.Milli, "1e-3d", "m", (Russian, "м"),(Chinese, "毫")),
new PrefixInfo(Prefix.Centi, "1e-2d", "c", (Russian, "с"),(Chinese, "厘")),
new PrefixInfo(Prefix.Deci, "1e-1d", "d", (Russian, "д"),(Chinese, "分")),
new PrefixInfo(Prefix.Deca, "1e1d", "da", (Russian, "да"),(Chinese, "十")),
new PrefixInfo(Prefix.Hecto, "1e2d", "h", (Russian, "г"),(Chinese, "百")),
new PrefixInfo(Prefix.Kilo, "1e3d", "k", (Russian, "к"),(Chinese, "千")),
new PrefixInfo(Prefix.Mega, "1e6d", "M", (Russian, "М"),(Chinese, "兆")),
new PrefixInfo(Prefix.Giga, "1e9d", "G", (Russian, "Г"),(Chinese, "吉")),
new PrefixInfo(Prefix.Tera, "1e12d", "T", (Russian, "Т"),(Chinese, "太")),
new PrefixInfo(Prefix.Peta, "1e15d", "P", (Russian, "П"),(Chinese, "拍")),
new PrefixInfo(Prefix.Exa, "1e18d", "E", (Russian, "Э"),(Chinese, "艾")),
new PrefixInfo(Prefix.Zetta, "1e21d", "Z",(Chinese, "泽")),
new PrefixInfo(Prefix.Yotta, "1e24d", "Y",(Chinese, "尧")),
new PrefixInfo(Prefix.Yocto, "1e-24d", "y", (Russian,null,"йоктог"), (Chinese, "夭",null)),
new PrefixInfo(Prefix.Zepto, "1e-21d", "z", (Russian,null,"зептог"), (Chinese, "仄",null)),
new PrefixInfo(Prefix.Atto, "1e-18d", "a", (Russian, "а","аттог"), (Chinese, "阿",null)),
new PrefixInfo(Prefix.Femto, "1e-15d", "f", (Russian, "ф","фемтог"), (Chinese, "飞",null)),
new PrefixInfo(Prefix.Pico, "1e-12d", "p", (Russian, "п","пиког"), (Chinese, "皮",null)),
new PrefixInfo(Prefix.Nano, "1e-9d", "n", (Russian, "н","наног"), (Chinese, "纳",null)),
new PrefixInfo(Prefix.Micro, "1e-6d", "µ", (Russian, "мк","микрог"), (Chinese, "微",null)),
new PrefixInfo(Prefix.Milli, "1e-3d", "m", (Russian, "м","Миллиг"), (Chinese, "毫",null)),
new PrefixInfo(Prefix.Centi, "1e-2d", "c", (Russian, "с","сантиг"), (Chinese, "厘",null)),
new PrefixInfo(Prefix.Deci, "1e-1d", "d", (Russian, "д","дециг"), (Chinese, "分",null), (French,null,"Déci")),
new PrefixInfo(Prefix.Deca, "1e1d", "da", (Russian, "да","декаг"), (Chinese, "十",null), (French,null,"Déca")),
new PrefixInfo(Prefix.Hecto, "1e2d", "h", (Russian, "г","гектог"), (Chinese, "百",null)),
new PrefixInfo(Prefix.Kilo, "1e3d", "k", (Russian, "к","килог"), (Chinese, "千",null)),
new PrefixInfo(Prefix.Mega, "1e6d", "M", (Russian, "М","мегаг"), (Chinese, "兆",null), (French,null,"Méga")),
new PrefixInfo(Prefix.Giga, "1e9d", "G", (Russian, "Г","гигаг"), (Chinese, "吉",null)),
new PrefixInfo(Prefix.Tera, "1e12d", "T", (Russian, "Т","тераг"), (Chinese, "太",null), (French,null,"Téra")),
new PrefixInfo(Prefix.Peta, "1e15d", "P", (Russian, "П","петаг"), (Chinese, "拍",null)),
new PrefixInfo(Prefix.Exa, "1e18d", "E", (Russian, "Э","экзаг"), (Chinese, "艾",null)),
new PrefixInfo(Prefix.Zetta, "1e21d", "Z", (Russian, null,"Зеттаг"),(Chinese, "泽",null)),
new PrefixInfo(Prefix.Yotta, "1e24d", "Y", (Russian, null,"Йоттаг"),(Chinese, "尧",null)),

// Binary prefixes
new PrefixInfo(Prefix.Kibi, "1024d", "Ki"),
Expand All @@ -49,7 +50,7 @@ internal class PrefixInfo
new PrefixInfo(Prefix.Exbi, "(1024d * 1024 * 1024 * 1024 * 1024 * 1024)", "Ei")
}.ToDictionary(prefixInfo => prefixInfo.Prefix);

private PrefixInfo(Prefix prefix, string factor, string siPrefix, params (string cultureName, string prefix)[] cultureToPrefix)
private PrefixInfo(Prefix prefix, string factor, string siPrefix, params(string cultureName, string? shortPrefix,string? longPrefix)[] cultureToPrefix)
{
Prefix = prefix;
SiPrefix = siPrefix;
Expand All @@ -62,6 +63,7 @@ private PrefixInfo(Prefix prefix, string factor, string siPrefix, params (string
/// </summary>
public Prefix Prefix { get; }


/// <summary>
/// C# expression for the multiplier to prefix the conversion function.
/// </summary>
Expand All @@ -76,7 +78,8 @@ private PrefixInfo(Prefix prefix, string factor, string siPrefix, params (string
/// <summary>
/// Mapping from culture name to localized prefix abbreviation.
/// </summary>
private (string cultureName, string prefix)[] CultureToPrefix { get; }
private (string cultureName, string? shortPrefix, string? longPrefix)[] CultureToPrefix { get; }


/// <summary>
/// Gets the localized prefix if configured, otherwise <see cref="SiPrefix" />.
Expand All @@ -88,9 +91,24 @@ public string GetPrefixForCultureOrSiPrefix(string cultureName)

var localizedPrefix = CultureToPrefix
.Where(x => string.Equals(x.cultureName, cultureName, StringComparison.OrdinalIgnoreCase))
.Select(x => x.prefix).FirstOrDefault();
.Select(x => x.shortPrefix).FirstOrDefault();

return localizedPrefix ?? SiPrefix;
}

/// <summary>
/// Gets the localized prefix name if configured, otherwise <see cref="SiPrefix" />.
/// </summary>
/// <param name="cultureName">Culture name, such as "en-US" or "ru-RU".</param>
public string GetPrefixNameForCultureOrSiPrefix(string cultureName)
{
if (cultureName == null) throw new ArgumentNullException(nameof(cultureName));

var localizedPrefix = CultureToPrefix
.Where(x => string.Equals(x.cultureName, cultureName, StringComparison.OrdinalIgnoreCase))
.Select(x => x.longPrefix).FirstOrDefault();

return localizedPrefix ?? Prefix.ToString();
}
}
}
10 changes: 9 additions & 1 deletion Common/UnitDefinitions/Length.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,15 @@
},
{
"Culture": "ru-RU",
"Abbreviations": [ "м" ]
"Abbreviations": [ "м" ],
"SingularName": "метр",
"PluralName": "метры"
},
{
"Culture": "fr-FR",
"Abbreviations": [ "m" ],
"SingularName": "Mètre",
"PluralName": "Mètres"
},
{
"Culture": "zh-CN",
Expand Down
15 changes: 15 additions & 0 deletions UnitsNet.Tests/UnitLocalizedNamesCacheFixture.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
using Xunit;

namespace UnitsNet.Tests
{
[CollectionDefinition(nameof(UnitLocalizedNamesCacheFixture), DisableParallelization = true)]
Copy link
Owner

@angularsen angularsen Oct 16, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only reason we use CollectionDefininition from before is to disable parallelization of tests that manipulate static values like CurrentCulture, so maybe we don't need multiple of these.

In another project, we have been using

    public static class XunitTestCollections
    {
        /// <summary>
        ///     Disables concurrency for test classes annotated with <see cref="CollectionAttribute"/> and this name.
        ///     <br/><br/>
        ///     Use this collection on test classes with tests that are not thread-safe, such as manipulating static objects
        ///     like <see cref="CultureInfo.CurrentUICulture" /> and <see cref="Thread.CurrentThread" />.
        ///     Each test method is already run serially per test class.
        /// </summary>
        [CollectionDefinition(nameof(NotThreadSafe), DisableParallelization = true)]
        public class NotThreadSafe
        {
        }
    }

Maybe we can rename UnitAbbreviationsCacheFixture to NotThreadSafe instead and copy this xmldoc there?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CollectionDefininition itself is used to shared a context (data for example between a collection) of tests . The parallelization is just an additionnal option.
I think that:
1°) you should add a getter in UnitLocalizedNamesCacheFixture to get the Culture instance
2°) you should modify the constructors of all your TUs to inject the fixtue each time you need to get the culture

Links:
https://xunit.net/docs/shared-context
https://blog.somewhatabstract.com/tag/collectiondefinition/

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting, could you give an example of how this could be used to test ToString() and Parse(), where no culture is given as parameter, but we want a consistent way to test specific cultures in the current thread? I'm very interested to learn if there is a better way to do this.

public class UnitLocalizedNamesCacheFixture : ICollectionFixture<object>
{
// This class has no code, and is never created. Its purpose is simply
// to be the place to apply [CollectionDefinition] and all the
// ICollectionFixture<> interfaces.

// Apply this collection fixture to classes:
// 1. That rely on manipulating CultureInfo.
}
}
Loading