Extract Numerator and Denominator Constants to Byte Arrays

[younies]

closes: #4100

Changes:

• Extracted the numerator and denominator constants.
• Represented each constant as an array of bytes, starting with the least significant byte (LSB).
• Implement and algorithm to detect any loop in the CLDR constants.

[younies]

@robertbastian , @Manishearth and @sffc , am I in the right direction?

[sffc]

What is the end goal? Do we want to convert the expressions to rational numbers? We should express them in the form most useful for runtime calculations.

[younies]

What is the end goal? Do we want to convert the expressions to rational numbers? We should express them in the form most useful for runtime calculations.

As a first step, I want to convert all the constants value to u32/u32 or u64/u64.

After that, I will use the same helpers in order to convert the conversion rates too to the form of u32/u32 + index.

I want to make all the calculations in the fractional forms (numerator / denomerator)

[sffc]

For rational number calculations, should we consider https://github.com/rust-num/num-rational ?

[sffc]

As you've noted, unfortunately not everything fits into u64/u64. I think what we should do here is only store the basic constants and not the square/cube version of them, and then in units that need the square or cube version, we just include the basic constant multiple times. For example:

        <convertUnit source='bushel' baseUnit='cubic-meter' factor='2150.42*in3_to_m3' systems="ussystem"/>

When we get to the point of encoding that factor, we should encode it as 2150.42 * in_to_m * in_to_m * in_to_m. Then we don't ever need to know that in3_to_m3 ever existed.

[sffc]

I know this conversation has been going back and forth for a little while, but I do share some concerns with @robertbastian about the fraction crate. We already pull in the num-rational crate which appears to support the same sort of abstraction: pub type BigRational = Ratio<BigInt>;. I don't understand why we need the extra crate. Besides this, fraction has less than 6% of the lifetime downloads of num-rational.

This isn't blocking because this is only datagen but it will definitely be a concern when we put this into the library crate.

[younies]

I've already explained to Robert the reasons for using the GenericFraction change:

• Sign Separation: Unlike BigRational, which includes the sign in the numerator, GenericFraction allows for a clear separation between the numerator/denominator and the sign.
• Byte Sequence Extraction: We need to convert BigInt to BigUint to extract the byte sequences for both the numerator and denominator.
• Decimal String Creation: GenericFraction supports initialization from decimal strings.

Given these reasons, GenericFraction is more appropriate for use in the datagen. However, for calculations during the conversion process, BigRational might be the better choice.

[Manishearth]

Hm. All of these seem to me to be relatively easily solved with some code. I don't think it is the right choice to depend on a large crate with a bunch of unsafe for this.

I'm comfortable adding the num deps, since those are an ecosystem standard.

[Manishearth]

Looking at the current code; it doesn't look like we rely on Sign Separation, i'm not sure where the byte sequence extraction is relevant, and decimal string creation we only use to parse a BigUint. We can do that directly.

[younies]

mmmm .. in this case, I can use BigRational

[younies]

but I am still not 100% convinced

[Manishearth]

I'm already making the switch. It's turning out to be quite straightforward.

[robertbastian]

question: what is this counting?

[younies]

This counts how many times there were no updates. If the number of no updates is greater than the length of the queue, this means there is a loop.

[robertbastian]

Should the else branch be count then?

[younies]

No, if there was 1 update, we start counting from zero. I have changed the name to be clearer: "no_update_count."

[robertbastian]

Thanks for clarifying