Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move unit abbreviation strings into culture-specific resource files #1210

Merged
merged 34 commits into from
Jun 18, 2023

Conversation

tmilnthorp
Copy link
Collaborator

@tmilnthorp tmilnthorp commented Feb 21, 2023

Fixes #1126

A quick draft to move the unit abbreviations into resource files. I also want to move methods for getting culture-specific abbreviations to the individual quantities.

This should:

  • Reduce initialization time by removing the MapGeneratedLocalizations methods (TODO)
  • Reduce in-memory footprint by only loading a requested culture
  • Allow formatting to use methods on the quantities themselves
    • Which should allow for better extensibility as well
  • Remove the unit abbreviations cache (deprecate)

TODOs

@tmilnthorp
Copy link
Collaborator Author

Check #1126 after merge

@tmilnthorp
Copy link
Collaborator Author

@angularsen curious if you like the direction this is headed. I need to re-implement PerformAbbreviationMapping if so

Copy link
Owner

@angularsen angularsen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't get time to read much, the kids are demanding my presence in the couch for TV and candy.

Some initial thoughts here.

/// <summary>
/// The per-culture abbreviations. To add a custom default abbreviation, add to the beginning of the list.
/// </summary>
public static IDictionary<(CultureInfo Culture, {_unitEnumName} Unit), List<string>> Abbreviations {{ get; }}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before even reading the rest, shouldn't this rather sit in QuantityInfo?

  • This is where units and other data for a quantity is stored
  • Reusable/extensible when working with IQuantity
  • Reduce member count per quantity (binary size)

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reading more, yes, GetAbbreviations() etc seems to me would sit nicely in QuantityInfo.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@angularsen I like that. Let me give it a shot here

Copy link
Owner

@angularsen angularsen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another partial review, generally I think this is a good idea and would love to see some benchmarks on improved startup time due to JIT.

catch
{
throw;
}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem too useful, maybe just omit the catch in this case?

abbreviations = new List<string>();

const string resourceName = $""UnitsNet.GeneratedCode.Resources.{_quantity.Name}"";
var resourceManager = new ResourceManager(resourceName, typeof({_quantity.Name}).Assembly);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with manually instantiating ResourceManager. Should we cache and reuse these instances?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally though, I think you are on to something good about reducing the generated code and moving things like localization to files that are read at runtime.

The JIT clearly has issues with our codebase. Not sure if it is the large number of types/members or if it is just the large amount of code, or what exactly.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving things into QuantityInfo is another way to avoid duplicating methods N times for N quantities.

The biggest hit is M members for M units though, which is roughly 10x the number of quantities.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@angularsen I am doing it lazily to load into a List. However there is some caching done internally, if my read of the docs is correct.

@tmilnthorp
Copy link
Collaborator Author

@angularsen moved to UnitInfo. Just have to do the custom mapping now.

I have no clue why auto-gen for ElectricCharge gets moved around.

// * Summary *

BenchmarkDotNet=v0.13.1, OS=Windows 10.0.19042.2604 (20H2/October2020Update)
AMD Ryzen 9 3900X, 1 CPU, 24 logical and 12 physical cores
.NET SDK=7.0.102
  [Host]     : .NET 6.0.14 (6.0.1423.7309), X64 RyuJIT
  DefaultJob : .NET 6.0.14 (6.0.1423.7309), X64 RyuJIT


|                 Method |          Mean |         Error |      StdDev |  Gen 0 |  Gen 1 | Allocated |
|----------------------- |--------------:|--------------:|------------:|-------:|-------:|----------:|
|            Constructor |      6.669 ns |     0.0742 ns |   0.0657 ns |      - |      - |         - |
|         Constructor_SI |    333.957 ns |     2.4437 ns |   2.2859 ns | 0.0229 |      - |     192 B |
|             FromMethod |     15.249 ns |     0.0954 ns |   0.0796 ns |      - |      - |         - |
|             ToProperty |     37.741 ns |     0.6000 ns |   0.5612 ns |      - |      - |         - |
|                     As |     38.237 ns |     0.4616 ns |   0.4318 ns |      - |      - |         - |
|                  As_SI |    333.398 ns |     6.2248 ns |   6.1136 ns | 0.0229 |      - |     192 B |
|                 ToUnit |     35.854 ns |     0.2862 ns |   0.2537 ns |      - |      - |         - |
|              ToUnit_SI |    340.429 ns |     6.7290 ns |   8.5101 ns | 0.0229 |      - |     192 B |
|           ToStringTest |  1,383.815 ns |    26.9084 ns |  28.7917 ns | 0.0801 |      - |     680 B |
|                  Parse | 90,754.795 ns |   911.6428 ns | 852.7512 ns | 6.3477 | 0.1221 |  53,538 B |
|          TryParseValid | 89,569.940 ns |   951.8402 ns | 794.8296 ns | 6.3477 | 0.2441 |  53,530 B |
|        TryParseInvalid | 64,205.361 ns | 1,022.9905 ns | 906.8544 ns | 5.0049 | 0.2441 |  42,766 B |
|           QuantityFrom |     66.576 ns |     1.3068 ns |   1.2835 ns | 0.0067 |      - |      56 B |
|           IQuantity_As |     41.443 ns |     0.6139 ns |   0.6029 ns | 0.0029 |      - |      24 B |
|        IQuantity_As_SI |    333.429 ns |     6.6227 ns |   6.1949 ns | 0.0229 |      - |     192 B |
|       IQuantity_ToUnit |     42.520 ns |     0.6126 ns |   0.5430 ns | 0.0067 |      - |      56 B |
| IQuantity_ToStringTest |  1,426.165 ns |    27.2386 ns |  27.9721 ns | 0.0801 |      - |     680 B |

// * Hints *
Outliers
  UnitsNetBenchmarks.Constructor: Default      -> 1 outlier  was  removed (8.33 ns)
  UnitsNetBenchmarks.FromMethod: Default       -> 2 outliers were removed, 4 outliers were detected (16.51 ns, 16.62 ns, 16.96 ns, 17.03 ns)
  UnitsNetBenchmarks.As_SI: Default            -> 1 outlier  was  detected (320.38 ns)
  UnitsNetBenchmarks.ToUnit: Default           -> 1 outlier  was  removed (38.08 ns)
  UnitsNetBenchmarks.TryParseValid: Default    -> 2 outliers were removed (94.28 us, 95.10 us)
  UnitsNetBenchmarks.TryParseInvalid: Default  -> 1 outlier  was  removed, 3 outliers were detected (62.01 us, 62.88 us, 66.01 us)
  UnitsNetBenchmarks.IQuantity_As: Default     -> 2 outliers were removed (45.16 ns, 45.31 ns)
  UnitsNetBenchmarks.IQuantity_As_SI: Default  -> 3 outliers were removed, 5 outliers were detected (321.38 ns, 324.70 ns, 347.70 ns..351.21 ns)
  UnitsNetBenchmarks.IQuantity_ToUnit: Default -> 1 outlier  was  removed (47.58 ns)

// * Legends *
  Mean      : Arithmetic mean of all measurements
  Error     : Half of 99.9% confidence interval
  StdDev    : Standard deviation of all measurements
  Gen 0     : GC Generation 0 collects per 1000 operations
  Gen 1     : GC Generation 1 collects per 1000 operations
  Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
  1 ns      : 1 Nanosecond (0.000000001 sec)

// * Diagnostic Output - MemoryDiagnoser *


// ***** BenchmarkRunner: End *****
// ** Remained 0 benchmark(s) to run **
Run time: 00:05:34 (334.18 sec), executed benchmarks: 17

Global total time: 00:05:56 (356.86 sec), executed benchmarks: 17

@tmilnthorp
Copy link
Collaborator Author

@angularsen now at the point where everything works - however routing everything via UnitInfo means instances of UnitAbbreviationsCache have data races on the static UnitInfo instances. So tests randomly fail 👎🏻

@tmilnthorp tmilnthorp marked this pull request as ready for review March 14, 2023 19:42
@tmilnthorp
Copy link
Collaborator Author

@angularsen now at the point where everything works - however routing everything via UnitInfo means instances of UnitAbbreviationsCache have data races on the static UnitInfo instances. So tests randomly fail 👎🏻

FYI tests run locally on re-run since the scope is narrower. Not sure what to do here if you want the lists in UnitInfo.

@angularsen
Copy link
Owner

@tmilnthorp I pushed a possible fix for resetting global static state in tests. All tests are green locally now. If we are happy with the solution, we can make things internal, cleanup names, document better etc.

tmilnthorp#1

@angularsen
Copy link
Owner

angularsen commented May 20, 2023

@tmilnthorp I like where this is going, but a few observations:

  • Quantity.Default is assigned one instance for its static property
  • UnitAbbreviationsCache is assigned an instance for each UACache instance
  • Parse/TryParse instance methods copied from Quantity don't do much yet, but this can be extended later to support custom units.

I built on this and added UnitsNetSetup to gather global state in a single place as a singleton, with the possibility of passing instances of it to static methods later to override the defaults.
This way, state is no longer scattered among multiple classes that depend on each other in non-obvious ways and where initialization order has caused issues up until now. It also makes it easier to discover what "services" we offer.

Pull request with some changes on top of this branch:
tmilnthorp#2

@angularsen angularsen merged commit 37ada11 into angularsen:master Jun 18, 2023
@angularsen
Copy link
Owner

@tmilnthorp I understand you are busy these days, I'll merge this and work some more on it.

@tmilnthorp
Copy link
Collaborator Author

@angularsen thanks! I’ve had this on my mind, just have not had time to come back to it recently. Hope to get back into some things in the coming months.

angularsen added a commit that referenced this pull request Jun 18, 2023
…files (#1210)

Fixes 37ada11 when merging #1210

The PR incorrectly handled some merge conflicts and wound up deleting code.
Regenerate code.
@angularsen
Copy link
Owner

@tmilnthorp No worries! 🙌

angularsen added a commit that referenced this pull request Jul 11, 2023
Added `UnitsNetSetup` to gather global state in a single place as a
singleton, with the possibility of passing instances of it to static
methods later to override the defaults.

This way, state is no longer scattered among multiple classes that
depend on each other in non-obvious ways and where initialization order
has caused issues up until now.

### Changes
- Add `UnitsNetSetup` to hold global state as a singleton property
`Default` that controls default state initialization.
- Add properties for all "services" that depend on global state:
`UnitConverter, UnitAbbreviationsCache, UnitParser, QuantityParser`
- Forward all other `Default` singleton properties from these services
to `UnitsNetSetup.Default` and mark obsolete
- Add some TODOs where it seems functionality is missing

### Testing
❌ Still running into a handful of flaky tests due to racing conditions. 
This was a regression in #1210, but still not fixed.
Will investigate and fix in separate PR.

```
UnitAbbreviationsCacheTests.AllUnitsImplementToStringForInvariantCulture
UnitAbbreviationsCacheTests.MapUnitToAbbreviation_AddCustomUnit_DoesNotOverrideDefaultAbbreviationForAlreadyMappedUnits
```
@angularsen
Copy link
Owner

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bad Performance on first Power.From() call in .netCore
2 participants