What does "backward-compatible with MF1" really means?

[mihnita]

Backward-compatibility is one of the (very) few things where we had agreement as a group:

Consensus 1 & 2: Identified as prerequisites for maintaining backwards-compatibility with MessageFormat 1 once Consensus 3 & 4 are agreed upon. Reached during the meetings of the #103 task-force, and codified during the October 2020 task-force meeting. Accepted at the November 2020 meeting of the working group.

https://github.com/unicode-org/message-format-wg/blob/main/docs/consensus_decisions.md

We should maybe define what "backwards-compatibility" really means.

I think it might beneficial for both issue #271 (Explain how to implement select "best match" with custom functions) and issue #350 (Allow names to start with a digit).
Maybe for others.

[mihnita]

For me this MF1 compatibility means that:

• MF2 can do all the good things that MF1 can do
• It does not mean syntax compatibility (I think we already agree on that, as the current syntax is not MF1 like)
• It does not mean method signature compatibility. One should not expect that they can swap out an MF1 import for a MF2 one without any other code changes.
• It is absolutely OK (and desired) to drop the things that are bad in MF1. In Why MessageFormat needs a successor (WG approved) we listed a few, and we also acted on some of them in (almost?) unanimity, for example "Select and plurals inside the message"

If we agree on this meaning for "compatibility" (and yes, it is a big IF :-), then "all we need to do" is to debate if something is a good thing (worth keeping) or a bad thing (to drop).

I strongly believe that MF1 implements best match, that we need it for proper functionality, and we can't implement it by "sorting outside runtime"

---

And to avoid some nitpicking: if at runtime I sort then select the first, it is best match. Because the comparator for the sort function looks at 2 items and decides "this one is better than the other" (chooses the best one).

I don't think that a company arguing "we revert-sorted people by age, and laid off the top 10%, but we did not choose the old people, we only took the first in the list, so it is not discrimination" would fly in any court (although you never know :-)

---

And asking one to sorting outside a method means the new method is not backward compatible.
Backward compatible means "same output for the same input"

I can't claim that a method doing this:

int getMinValueFromArray(int [] a) {
    return a[0];
}

is backward compatible with the older version that iterated the array
("but all you have to do is sort the array before passing it to my method" :-)

[eemeli]

Here's my definition of MF1 compatibility:

It should be possible to take the public API of any existing MF1 library, and keep that API stable while rewriting the library's internals to parse any input message into an MF2 data model and use an MF2 runtime with some defined registry of functions for its formatting.

To help ensure that we keep at least to this level of compatibility, I've written and maintain a proof-of-concept MF1-to-MF2 compiler.

[mihnita]

From my understanding of what you describe that is being able to reimplement MF1 on top of the MF2 data model.
That is what the proof-of-concept does, I think.

But MF2 is not only data model, it is also how that model is interpreted / "rendered".
The runtime behavior is part of MF2, it is what you describe in PR #333.

So if to be compatible with MF1 I need to replace part of the MF2 standard (the behavior part), it means the standard is not really compatible.

Unless we say that the behavior is not part of the MF2 spec.
Which I don't think is the position of anyone in this group.

[eemeli]

I don't see how my PoC replaces any part of MF2 runtime behaviour. It:

1. parses MF1 syntax into a corresponding representation in an MF2 data model,
2. defines a set of expression handlers (i.e. a function registry) for MF2, and
3. constructs an MF2 formatter instance using the above.

How do you think that it replaces some part of the MF2 standard?

[stasm]

Replying to @mihnita:

And asking one to sorting outside a method means the new method is not backward compatible. Backward compatible means "same output for the same input"

MF2 has different data model and a different syntax, so it's not "the same input". As such, a migration is necessary.

It's important why we want backwards compatibility. We don't expect MF1 and MF2 to run side-by-side in the same implementation. We want to ensure that a migration path exists from MF1, but we understand that it may involve changes to (a) how translations are stored, and (b) how the formatting API is called.

[aphillips]

@stasm noted:

It's important why we want backwards compatibility. We don't expect MF1 and MF2 to run side-by-side in the same implementation. We want to ensure that a migration path exists from MF1, but we understand that it may involve changes to (a) how translations are stored, and (b) how the formatting API is called.

Actually, we don't know or care how translations are stored (i.e. resource management). MF1 pattern strings are definitely not compatible with MF2 runtimes.

What I think @mihnita is trying to say (please to correct me) is that given an equivalent pattern string to MF1 and the same args (Array or Map), the same functional output can be achieved: one could use MF2 to get the exact same string without having to alter anything in args to get it to work.

@mihnita elsewhere above you said:

It does not mean method signature compatibility. One should not expect that they can swap out an MF1 import for a MF2 one without any other code changes.

We could have calling equivalence, that is, minimal disruption to existing MF1 Java code:

import com.ibm.icu.text.MessageFormat; // or java.text...
//...
MessageFormat mf = new MessageFormat(somePattern, someLocale);
Map args = ImmutableMap.of("arg1", arg1, "arg2", arg2);
String output = mf.format(args);

One can change the import above and have the code still work (same too with the Object[] consuming version, see our recent thread about how to name the args). I know this is true, because I've already done it twice. In my case, the replacement formatter had a superset of features and different recommended interfaces, but developers weren't required to use them.

Method signature compatibility is only really relevant to the Java implementation. That's a pretty narrow (and implementation specific) consideration. With one proviso, we can place this out of our own scope. That proviso would be: we need to ensure that our function registry provides at least:

selectors = "choice" | "plural" | "selectordinal" | "select"
functions = "number" | "date" | "time" | "spellout" | "ordinal" | "duration"
; ripped from MF1's doc, they don't all apply each function on the above function/selector
options   = "short" | "medium" | "long" | "full" | "integer" | "currency" | "percent" | 
          / argStyleText | "::" argSkeletonText

Is this what we mean by compatibility?

[stasm]

Actually, we don't know or care how translations are stored (i.e. resource management).

I only meant "storage" as in "syntax". A migration from MF1 to MF2 will require a translation from one syntax to the other.

[eemeli]

MF1 pattern strings are definitely not compatible with MF2 runtimes.

How so? While an MF2 syntax parser won't be able to parse MF1 as well, the rest of the runtime is at least currently compatible with MF1; that's what my PoC demonstrates.

[aphillips]

MF1 pattern strings are definitely not compatible with MF2 runtimes.
How so? While an MF2 syntax parser won't be able to parse MF1 as well, the rest of the runtime is at least currently compatible with MF1; that's what my PoC demonstrates.

MF1 pattern strings are not compatible with being formatted by MF2 runtimes (without additional processing). And, in fact, they are treated as syntax errors. I did not say or intend to say that they couldn't be transformed into MF2 😉

[mihnita]

MF2 has different data model and a different syntax, so it's not "the same input". As such, a migration is necessary.

By "same input" and "same output / behavior" I meant the same information, not necessarily same exact string (syntax).
We can define that better.

Similar to "if you give me a container you get that back sorted". There is no implication what the "container" is, or the sort algorithm, of the programming language.

[aphillips]

close as stale in the 2023-11-06 telecon