Unit formatting API

[zbraniecki]

Unit formatting is similar to number formatting, but it involves one more units.
The goal is to produce strings like "5 MB/s", "10 KB", "35 °C" etc.

I believe the most relevant for the Web audience areas would be:

• Digital units
• Duration units
• Weather units
• Length units
• Compound units

and maybe:

• Energy units
• Frequency units

I would suggest that we limit ourselves first to just short version of units and use compound patterns to produce related units (like speed out of length and duration units).

In the future we may leave a window open to add 'variant' option for 'short'/'long' etc.

Proposed API:

var f = Intl.UnitFormat(navigator.languages, {
  units: [
    {type: 'digital', match: 'bestFit'}
  ]
});
f.format(1024); // would return '1 KB' in en-US

var f = Intl.UnitFormat(navigator.languages, {
  units: {
    {type: 'digital', unit: 'bestFit'},
    {type: 'length', unit: 'second'}
  }
});
f.format(1024); // would return '1 KB/s' in en-US

var f = Intl.UnitFormat(navigator.languages, {
  units: {
    {type: 'length', unit: 'bestFit'},
  }
});
f.format(2048); // would return '2 km' in en-US

Open questions

• Should that be a separate formatter or should we try to extend NumberFormat to handle units?
• How to handle unitless formatting like '5/101' (use case: 5 out of 101 emails unread)
• How to define reference unit in which the passed variable is in? Should we hardcode - meter for length, byte for digital? Or should we allow for it to be an option?
• We should define some rounding strategy options

[rxaviers]

We should take input from http://www.unicode.org/reports/tr35/tr35-general.html#Unit_Elements

[zbraniecki]

Relevant CLDR entry example: http://www.unicode.org/cldr/charts/27/summary/pl.html#5556

[srl295]

How to define reference unit in which the passed variable is in?

It should be required to be explicitly passed in. You do NOT want 2048 to format as 2km. Pardon my caps here, but this is a very bad idea. Two side notes: 1. I've used an ATM that displayed a different currency type than the type which it withdrew (4:1 factor - ouch.) 2. Mars Orbiter Crash due to lbf/s vs N/s. ouch.

I would require the formatter to be explicit, or, have some object type where the object specifies the type. Perhaps a subclassed Number that has a 'type' field? I made that up.

Unit conversion, unit discovery, etc should be based on top of a service that just formats explicit units. Shades of the PluralRules proposal.

[rxaviers]

There should be an option to modify the display form. e.g., 2 seconds (long), or 2 sec (short), or 2s (narrow).

[rxaviers]

An alternative API

var formatter = Intl.UnitFormat(locales, unit[, options]);
formatter.format(value);

// Example
var formatter = Intl.UnitFormat("en", "second");
formatter(2);
// > "2 seconds"

var formatterShort = Intl.UnitFormat("en", "second", {
  form: "short"
});
formatterShort(2);
// > "2 sec"

var speedFormatter = Intl.UnitFormat("en", "kilometer-per-hour", {
  form: "short"
});
speedFormatter(100);
// > "100 kph"

var speedFormatterPt = Intl.UnitFormat("pt", "kilometer-per-hour", {
  form: "short"
});
speedFormatterPt(100);
// > "100 km/h"

[rxaviers]

• CLDR UTS #35: Part 2 (General), #6 Unit elements
• Example data: https://github.com/unicode-cldr/cldr-units-full/blob/master/main/en/units.json

[srl295]

@rxaviers I like this format better, but I think it should be speed-kilometer-per-hour not just kilometer-per-hour

[zbraniecki]

@rxaviers - I like your proposal better than mine. Thanks!

One question - did you mean for UnitFormat to return a function or an object with format property? I think I'd prefer an object (would give us resolvedOptions() and be consistent with other Intl objects)

[caridy]

I like what I'm seeing here, but I want to bikeshed on the api a little bit more until we feel confortable with it, here is an alternative proposal where we will have:

• style: long (default), short, and narrow, which represents the style of the output value
• unit: "whatever output format you want", which represents the scale to be used.

var formatter = Intl.UnitFormat(locales[, options]);
formatter.format(value);

// Example
new Intl.UnitFormat("en", { 
   unit: "second"
}).format(2);
// > "2 seconds"

new Intl.UnitFormat("en", {
  style: "short",
  unit: "second"
}).format(2);
// > "2 sec"

new Intl.UnitFormat("en", {
  style: "short",
  unit: "kilometer-per-hour"
}).format(100);
// > "100 kph"

new Intl.UnitFormat("pt", {
  style: "short",
  unit: "kilometer-per-hour"
}).format(100);
// > "100 km/h"

[rxaviers]

One question - did you mean for UnitFormat to return a function or an object with format property? I think I'd prefer an object (would give us resolvedOptions() and be consistent with other Intl objects)

@zbraniecki whoops, I forgot the new operator. But, object yeap.

@caridy, we're definitely making improvements, thanks.

@srl295, sure, we should brainstorm about the unit values. The reasons I like the shorter form (e.g., second) better than the longer one (e.g., duration-second ) are:

• It's simpler. A user will always figure out (without reading documentation) that a unit should be second, but figuring out it's duration-second (or possibly time-second) requires consulting docs.
• The implementation for internally deducing the categories (e.g., speed-*, duration-*, area-*, etc) is no more complex than relying on users to input these. Because, implementation has to deal with it anyway when deducing the per unit compounds [1].
• It allows using custom -per- constructions. For example, say you want to format the bytes throughput (e.g., MB/sec). How would you name the category? throughput (e.g., throughput-megabyte-per-second)? CLDR provides digital-megabyte and duration-second, but no precomputed form for megabyte per second. So, there's no defined category for such custom (yet popular) combination.

1: On CLDR docs (6.1 per Unit patterns), it says Some units already have 'precomputed' forms, but for all other ones, there are rules for deducing them by using compoundUnit and perUnitPattern. The implementation has to figure out that megabyte-per-second should be made from digital-megabyte and duration-second (i.e., it must deduce the unit categories).

[zbraniecki]

@rxaviers in #35 (comment) suggests that things like durations (hh:mm:ss -> 00:23:15) should be part of UnitFormat, let's discuss how to alter the API proposal to achieve that.

For this kind of formatting to work we need to either choose a max-min units, or some reference unit.

For Time, I feel like maxUnit/minUnit would be better, because those can be then used for UIs like video/music players (00:23 in the song, or -01:34 left) or stopwatch/timer (00:00:23.15).

So, my proposal is:

new Intl.UnitFormat("en", {
  minUnit: 'second',
  maxUnit: 'hour',
  type: 'time'
}).format(6 * 60 * 60 * 1000 + 34 * 60 * 1000 + 12 * 1000);
// > "06:34:12"

new Intl.UnitFormat("en", {
  minUnit: 'millisecond',
  maxUnit: 'minute',
  type: 'time'
}).format(34 * 60 * 1000 + 12 * 1000 + 340);
// > "34:12.34"

new Intl.UnitFormat("en", {
  minUnit: 'second',
  maxUnit: 'minute',
  type: 'time'
}).format((2 * 60 * 1000 + 12 * 1000) * -1);
// > "-02:12"

Open questions:

• how to define paddings - 02:12 vs 2:12, or maybe in this format we always want paddings?
• how to define roundings - milliseconds are in 1000, but usually, we'll want to use deciseconds. So 12:34.56 instead of 12:34.559
• how to merge this with the default proposal for 2 sec, 2 seconds, 2 km/s.
• how to incorporate other units, like height for 5'11"

Alternatively, maybe we shouldn't merge DurationFormat with UnitFormat? Maybe it is different enough that it will just not fit?
Because getting 5'11" in height is fairly easy and similar to 35.5 ℃ - you just need a unit (celsius, feet) and formatting for fraction digits like in NumberFormat.
You could try to build my stopwatch example with UnitFormat, but I'm not sure how could we handle two different fractions (say unit is minutes, and we want to allow for seconds and milliseconds).

Ideas? Opinions?

[caridy]

I don't like min/max definitions. Instead, I think we can use a more familiar schema, which is the use of best fit vs specific unit to fit the output format.

[zbraniecki]

I understand why you don't like it. I'm also not super happy with it, but I can't find any other way to squeeze all of the forms in one API. bestFit sounds good for API that is supposed to guess which is the max-unit, and which is the min-unit - so if we somehow establish that 2000 is in seconds, we can use bestFit to turn it into some form of 33 minutes and 20 seconds, but for scenarios where we need specific unit range, like stopwatch formatting, or music player with mm:ss, we need to know which units to show, even if the value for a given unit is zero (so, if we want to show mm:ss for 20 seconds we have to show it as 00:20, but if we're showing it in a stopwatch, we'll show it as hh:mm:ss).

My current thinking goes in the direction of the distinction between formatting that tries to build the best 'written' representation of data ('20 seconds', '5 minutes', '10 hours'), and more UI-like scientific representation (00:20).

I'm thinking a lot about this vs. number format. I feel like it's pretty unfortunate that NumberFormat specified currency and currencyDisplay as explicit arguments because currency (and percent) are just another unit type. The first type seems to be fitting NumberFormat with their useGrouping, and min and max integer|fraction|significant digits.

I still think that bestFit makes no sense unless we know what is the reference point and type of units - 2000 means nothing for bestFit unless we know we talk about time, and it's in milliseconds. Then we know it's 2 seconds.

So, in a perfect world, I imagine the former group go into NumberFormat this way:

var f = Intl.NumberFormat(locale, {
  type: 'time',
  unit: 'second',
  style: 'long'
}).format(2); // '2 seconds'

var f = Intl.NumberFormat(locale, {
  type: 'time',
  unit: 'second',
  style: 'short'
}).format(2); // '2 sec'

var f = Intl.NumberFormat(locale, {
  type: 'time',
  unit: 'second',
  style: 'narrow'
}).format(2); // '2s'

and then if we establish some rule that, for example for type: time reference unit is milliseconds, we could do:

var f = Intl.NumberFormat(locale, {
  type: 'time',
  unit: 'bestFit',
  style: 'long'
}).format(1395000); // '23 minutes and 15 seconds'

The benefit of that is that it would work for type currency, time, distance, weather, percent, digital. It would nicely allow for groupings ( 23 000 hours), fraction digits (23.5 hours) etc.

The problem is that we don't have how to accommodate compound patterns, and of course, that that train has already left.

The other part of the equation is what I so far am calling DurationFormat, but I think it stretches beyond duration. It's the scientific counterpart of that. It's not supposed to be part of the sentence (or whole message), it's part of UI - timer, stopwatch, music player, video editing tool, screen information.

For that, users want to define the upper and lower limit (hour-second, minute-millisecond, feet-inches), we don't need long/short/narrow forms, we don't need compound units, we don't need guessing, we need to fit the data into a pattern.
Instead, we need to know how many units to display, how to round (32:12.256 vs 32:12.26 vs 32:12.3), should we show padding zero in front of the first unit (02:12 vs 2:12).

I feel like the goal should be to squeeze both into one API, but I struggle to find common option list for both models. For height, we could want to show 6 feet or 6" for the former, but stick to regular 6"0' in the latter scenario to make it regular between 6"0', 6"1' etc.

Regarding time, my list of displays I can imagine would be needed are:

• 2 seconds
• 2 sec
• 2s
• 2.5 seconds
• 2.5 sec
• 2.5s
• 2.0s
• 2.0 seconds
• 00:02
• 00:02.00
• 0:02.00
• 02.00
• 02.000
• 2 seconds and 500 milliseconds
• 2 sec and 500 ms
• 2s, 500ms
• 2500 seconds
• 0.5 seconds

Any ideas how to accommodate all those forms in one API? Maybe we just need to functions on the formatter? Like formatter.format and formatter.pattern? Or maybe just a parameter on format? But then how to design the list of options?

Ugh, it's complex :)

[zbraniecki]

So, it seems that ICU is using MeasureFormat (which is similar to UnitFormat) for durations - http://www.icu-project.org/apiref/icu4j/com/ibm/icu/text/MeasureFormat.html - but I can't find an example how they enable translation of an integer into 78:12:02 timer style string.
@srl295 - can you help me here?

[zbraniecki]

I ended up opening #47 to discuss the duration formatting. Without it, I feel like UnitFormat is much simpler with @caridy proposal with style and unit which can be x-per-y.

One item from CLDR that I see missing from the API is unit sequence - http://www.unicode.org/reports/tr35/tr35-general.html#Unit_Sequences

Is there room for that in this API?

[zbraniecki]

Asking because CLDR examples - 5° 30′ and 3 ft 2 in. seem to use space to join the components, but ICU example [0] uses ListFormatter ( #33 ) - 3 feet, 2 inches,

Once we get the API's working, we could construct the ICU example by running two components via UnitFormat and then using ListFormat to join them.
But the CLDR example seems different and I'm not sure if it requires sequence formatter or not.

[0] http://www.icu-project.org/apiref/icu4j/com/ibm/icu/text/MeasureFormat.html#formatMeasures%28com.ibm.icu.util.Measure...%29

[rxaviers]

@zbraniecki

I'm thinking a lot about this vs. number format. I feel like it's pretty unfortunate that NumberFormat specified currency and currencyDisplay as explicit arguments because currency (and percent) are just another unit type.

True but with a few additional rules. About currency, the symbol position appears after the number in some locales, but before in others, which doesn't happen with unit in general (always after); the fraction digits also change depending of the chosen currency (e.g., JPY uses no fraction digits, others use 5 decimal points instead of 2). About percent, the number value gets adjusted on formatting, i.e., 0.1 becomes 10%.

[rxaviers]

@zbraniecki

One item from CLDR that I see missing from the API is unit sequence - http://www.unicode.org/reports/tr35/tr35-general.html#Unit_Sequences

...

Asking because CLDR examples - 5° 30′ and 3 ft 2 in. seem to use space to join the components, but ICU example [0] uses ListFormatter ( #33 ) - 3 feet, 2 inches,

Once we get the API's working, we could construct the ICU example by running two components via UnitFormat and then using ListFormat to join them.
But the CLDR example seems different and I'm not sure if it requires sequence formatter or not.

Good point. Perhaps, keeping UnitFormat simple (single units and compound only) and then using UnitFormat + ListFormat for the "5° 30′" and "3 ft 2 in" ones would be the best approach indeed.

I've noticed from your link that CLDR 6.2 Unit Sequences section points to the listPattern data. So, that seems the way to go.

For clarity, I understood that if we take that approach, that's what we are going to get:

• FormatUnit for "3.3 oz", "10 kph", "100 m/s" ones.
• ListFormat(FormatUnit, FormatUnit, ...) for "5° 30′", "3 ft 2 in" ones.
• DurationFormat for "8:55", "8:55:00" ones.

[zbraniecki]

I'm implementing UnitFormat currently for FxOS and there are a few things I noticed:

• UnitFormat is very, very simple. It literally just does ${group}-${unit}-${style} and pools from CLDR.
• In almost every case it will require a unit selector on top. I'm currently working most on duration and digital, and they will have different roundings (0.9mb makes sense and looks better than 900kb, but 0.9h does not)
• We need something for compound units

To the last point, @rxaviers - I don't think that ListFormat will solve it for us. ListFormat does Anne, John and Mary, not Anne Mary.

We can either add another style to ListFormat (it already has three - duration, regular and numbers) - like unit, or we need to figure out how to make UnitFormat handle compound units.

[rxaviers]

We need something for compound units

Please, could you clarify? What about using the -per- units as discussed above? The implementation is also pretty straightforward. This is how this is implemented in Globalize: https://github.com/jquery/globalize/blob/master/src/unit/format.js#L34-L43

To the last point, @rxaviers - I don't think that ListFormat will solve it for us. ListFormat does Anne, John and Mary, not Anne Mary.

I'm referring to CLDR 6.2 Unit Sequences section and the listPattern data, which should be used to glue the unit sequences, right? Did I understand you well? Using their own example: 5° 30′ for 5 degrees 30 minutes.

[zbraniecki]

Please, could you clarify? What about using the -per- units as discussed above?

Do you want to allow for arbitrary units on both sides? bytes-per-kilometer? Seems like we have no reason not to?

I'm referring to CLDR 6.2 Unit Sequences section and the listPattern data, which should be used to glue the unit sequences, right? Did I understand you well? Using their own example: 5° 30′ for 5 degrees 30 minutes.

That's cool! Didn't see that before.

So, we should just add type: 'unit' to ListFormat.

[rxaviers]

Do you want to allow for arbitrary units on both sides? bytes-per-kilometer? Seems like we have no reason not to?

Yeap, for arbitrary units on both sides. Although some might not make any sense, it's more generic and up to the user to pick. It also goes in line with CLDR 6.1 per Unit patterns, including the precomputed forms.

That's cool! Didn't see that before.

So, we should just add type: 'unit' to ListFormat.

Yeap, possibly :)

[zbraniecki]

Landed UnitFormat in Gaia as mozIntl shim - mozilla-b2g/gaia@ad3661e

As you can see the UnitFormat.format itself is very primitive ATM (I didn't add compound units), but the more interesting is the mozIntl._gaia.getFormattedUnit that lives on top of it and provides unit selection based on category and value.

Let's see how it'll handle in real life environment.

[zbraniecki]

Heads up, this has been advanced to Stage 0 as of today.

[zbraniecki]

We plan to request stage 2 at the next TC39 meeting

[zbraniecki]

This proposal has not received an approval to move to stage 2 yet.

The TC39 committee asked us to

• refine our understanding of scope of categories we want to support
• refine the relationship between UnitFormat and RelativeTimeFormat, DurationFormat and upcoming Duration type proposal.
• explore the possibility of dividing this API into a list of Intl.LengthFormat, Intl.DigitalFormat etc.

@maggiepint may want to look at https://github.com/zbraniecki/proposal-intl-unit-format

My current plan is to focus on ListFormat first, then RelativeTimeFormat, but in the meantime I'd be happy to see more conversation about the scope and shape of UnitFormat proposal.

[jungshik]

Has there been any discussion on 'compound' unit format? '3 meters and 45 cenitmers' or '3 m 45 cm' As with 'Relative Date', UnitFormat can also benefit from that.

[caridy]

@jungshik IIRC, we have discussed briefly in some of the F2F meetings in the pass. Do you think that we need to open a new issue to track just that?

[littledan]

@jungshik Do you think we should include compound units in the first version, or as a follow-on?

[sffc]

See my follow-up in #215. I included an option for compound units.

[ByteEater-pl]

Is support for IEC 80000-13 binary prefixes planned? It would be a great pity to perpetuate the ambiguity about the meaning of k, M, G, etc.

[sffc]

Is support for IEC 80000-13 binary prefixes planned? It would be a great pity to perpetuate the ambiguity about the meaning of k, M, G, etc.

We use the units provided by CLDR. Myself and others have long requested that the binary prefixes be added for digital units, but CLDR has held the position that the goal of CLDR units is to reflect common use, not necessarily correct use. As long as the "old" prefixes are in common use and the "new" prefixes are not, there will continue to be pushback for adding them to CLDR.

Here is one discussion on the topic: https://unicode.org/cldr/trac/ticket/7214

As to gibibyte, you are disregarding previous email on this topic. While a laudable attempt, it is simply not in widescale use.
"...According to a Google search, there are 305 times (30500%) more instances of gigabyte...."

There is a new ticket however where these units appear to be back on the table. I am putting a reference to this thread on that ticket:

https://unicode.org/cldr/trac/ticket/11454

[ByteEater-pl]

On Linux binary prefixes are used quite commonly. Don't make Google the ultimate oracle.

Besides, producers of mass memory usually (understandably, as it gives higher numbers) use the SI prefixes correctly, e.g. a 100 MB hard disk can store 100000000 bytes.

[zbraniecki]

This has been folded into Intl.NumberFormat rev. 2 in #215. Closing. Reopen if needed.