Editorial: Refactor for normative reference by higher web specs

[littledan]

Other web specs do various kinds of breaking, especially line breaking. Factor the Intl.Segmenter spec text such that there is an abstract algorithm that they can call to get at the break, to cement the fact that Intl.Segmenter uses the same breaking algorithm as higher web specs.

cc @annevk

[annevk]

In particular we want this for whatwg/dom#431 and w3c/selection-api#37.

[annevk]

cc @rniwa

[rniwa]

I don't think we can use it for modifying selection at least in WebKit because whitespace treatment and the way word granularity is computed is very different from the way ICU determine a word boundary in various languages.

Perhaps this could be used to determine line breaking behaviors in CSS though.

[annevk]

Wouldn't you use that implementation for Intl.Segmenter too though? Seems weird to be inconsistent.

[rniwa]

No, we don't, intentionally due to subtle differences in the way whitespaces are treated and how word boundaries are determined. It's a real mess but it's quite important to have a good UX.

[littledan]

OK, we don't have to make a reference there; another place where it might be nice to have consistency is line breaking, cc @eaenet. However, @zbraniecki pointed out to me that not all engines use UAX 14-based line breaking, so I was thinking of weakening the language in this specification to be more similar to how CSS Text Level 3 explains it.

[annevk]

@rniwa but if it's important to get good UX, wouldn't that be important for applications too, that likely end up using Intl.Segmenter to get it?

[rniwa]

@annevk : that's why you should be using getSelection().modify instead of Intl.Segmenter to modify selection. Because of the way we implement things, I don't think we can implement expose it via Intl.Segmenter. In order to decide where the word boundaries exist for selection purposes require us doing hit testing in AppKit. Obviously, we're not gonna do that in a pure JS API. That would be insane.

[gibson042]

It's not clear to me that there's anything to do here, but I'm happy to refactor AdvanceSegmentIterator there's a concrete request (e.g., a signature for the desired algorithm).