Add markup support

[eemeli]

A core value proposition of MF2 is being able to represent messages that have some internal structure, such as:

{Click {+a href=$url id=x}here{-a} to continue}

{Text can be {+b}bold {+i}and{-b} italic{-i}}

{Sometimes, you need to be {+ssml.sub}subtle{-ssml.sub}.}

In the MF2 spec, the + and - annotation prefixes are intentionally loosely defined to correspond to "start/open" and "end/close" parts of a message, to allow for implementation flexibility. The examples above demonstrate some of that: They may have inputs and options; they do not need to nest cleanly, and their names may include characters like . that aid in subclassing.

To support these in Intl.MessageFormat, the approach proposed here is both rather restrictive and permissive: The resolution of the syntax parts is not opened for customization (as is the case for standalone placeholders like :number), but the corresponding formatted parts are emitted in a form that makes their post-processing as easy as possible. For example:

const src = '{Click {+a href=$url id=x}here{-a} to continue}';
const mf = new Intl.MessageFormat(src, 'en');
mf.formatToParts({ url: 'https://example.com/' });

[
  { type: 'literal', value: 'Click ' },
  { type: 'open', name: 'a', source: '+a', options: { href: 'https://example.com/', id: 'x' } },
  { type: 'literal', value: 'here' },
  { type: 'close', name: 'a', source: '-a' },
  { type: 'literal', value: ' to continue' }
]

Given that, a post-processor could quite easily construct from it a DOM fragment, a sequence of React elements, or an HTML string representation, applying any validation rules specific to the target structure.

Formatting an open/close markup expression to a string always results in the empty string, so continuing with the above:

mf.format({ url: 'https://example.com/' }); // 'Click here to continue'

This post-processing approach is required to support the wide range of use cases and environments that JavaScript has, and it's to some extent forced by the MF2 spec, which does not support internally defining something like a formatter that is capable of consuming a sequence of message parts, rather than just one part.

[dminor]

This looks good to me.

My only suggestion is to incorporate some of the explanation from the pull request into the explainer itself.

[eemeli]

@dminor Thank you for the review! I've added a few phrases in response to your inline comments.

In the MFWG, unicode-org/message-format-wg#470 may end up refining that spec layer's understanding of markup, so I'll hold off on merging this until at least next week, given that MFWG members will be meeting in Seville during W3C TPAC and some progress is expected on this topic.

[eemeli]

@catamorphism Thank you for the review! The terminology was indeed a bit wrong, but I ended up updating it towards the practice introduced in unicode-org/message-format-wg#470, which treats markup as a different thing from expressions.

[eemeli]

This PR will need to be updated once the current MFWG discussions on markup are concluded.

[eemeli]

The changes here have now been updated to match the accepted spec design. I've also included spec.emu changes, still leaving variable resolution as "TODO" given that including it here would effectively be feature creep.

[eemeli]

I ended up dropping the spec.emu changes from here, as I ended up following this with a more thorough spec implementation, where the parts added here needed to be modified further. I'll submit those changes as a separate PR.

[dminor]

A few nits, but overall, this looks really good!

[ryzokuken]

The actual contents of this PR LGTM and the idea to just emit markup in formatToParts to let people post-process it themselves sounds like a fair plan but did I misunderstand at some point or was this always how markup was intended to work? I know that formatToParts is a JS-only thing at the moment but I assumed that handling markup would involve exposing special functions (which would work in both format and formatToParts) but this should work too. If this way of dealing with markup is already implemented in the polyfill, I'd like to try it out with some sort of React app that injects components inside a message to see if it works as intended.

[eemeli]

Yeah, requiring post-processing for markup has been the idea for a while, and it's supported by the polyfill. So with messageformat@next, you can do this:

import { MessageFormat } from 'messageformat';
const mf = new MessageFormat('Have a {#b}great{/b} day!', 'en');
mf.formatToParts()

and get:

[
  { type: 'literal', value: 'Have a ' },
  { type: 'markup', kind: 'open', source: '#b', name: 'b' },
  { type: 'literal', value: 'great' },
  { type: 'markup', kind: 'close', source: '/b', name: 'b' },
  { type: 'literal', value: ' day!' }
]