Sort channel names the same way as Zulip web

[gnprice]

On the subscriptions / channels page, the sorting of channels by name doesn't fully match the behavior we have in the Zulip web app. In particular, we got a user report that names starting with an emoji are being sorted to the end, while they appear at the beginning in Zulip web.

We should match the ordering of Zulip web, so that people see their channels in the same order when they move between using Zulip on one platform vs. another.

If doing so exactly is impractical or difficult, then we should (a) do a hack that at least handles an emoji at the start of the name, matching Zulip web's ordering for emoji vs. non-emoji and for emoji vs. other emoji; and (b) file a follow-up issue for matching the ordering exactly.

Implementation

The name isn't the only key that the subscriptions / channels list is sorted by; it's also sorted by the channel's status as pinned and/or muted. The role of pinning and muting should remain unchanged.

This issue applies to everywhere a list of channels appears. That means:

• the inbox page
• the subscription-list page (accessed through the "channel" icon in the bottom nav bar)
• the all-channels page, once implemented: Offer a list of all channels, and a way to subscribe #188

[chrisbobbe]

Web's left sidebar has three headered groups: "Pinned", "Active", "Inactive", in that order. Within a group, non-muted channels come before muted ones.

After that, sorting is done on channels' names with this compare function:

// If we can, use a locale-aware sorter.  However, if the browser
// doesn't support the ECMAScript Internationalization API
// Specification, do a dumb string comparison because
// String.localeCompare is really slow.
export function make_strcmp(): (x: string, y: string) => number {
    try {
        const collator = new Intl.Collator();
        return collator.compare.bind(collator);
    } catch {
        // continue regardless of error
    }

    return function util_strcmp(a: string, b: string): number {
        return a < b ? -1 : a > b ? 1 : 0;
    };
}

export const strcmp = make_strcmp();

I assume the Intl.Collator behavior is what most web-app users get these days.

Dart doesn't have an analog of Intl.Collator. Could we get close to that behavior by using ICU (probably libicui18n?) via ffi?

Otherwise, would unorm_dart be useful for this? That library has come up as the preferred library for filtering lists of strings insensitively to diacritics, like #1067. No, turns out this doesn't offer anything for sorting.

[gnprice]

Thanks, that's helpful!

Yeah, I think using an ICU library via FFI is probably the full solution. That could be something we do ourselves once FFI support is more mature, as the Flutter roadmap calls for next year. Or it looks like people are working on building such a thing for everyone (and for the full scope of ICU functionality): package:icu4x, published by "unicode.org". Still a work in progress but has Unicode engineers working with the Dart team to build the things they need to make it all work:

• Ship binaries for Dart unicode-org/icu4x#4689

(And that is indeed using FFI; here's the Dart code implementing Collator.compare.)

In the meantime let's do this:

a hack that at least handles an emoji at the start of the name, matching Zulip web's ordering for emoji vs. non-emoji and for emoji vs. other emoji

I'll file a separate issue for that nearer-term hack, and edit this issue to focus on the full solution. (Revising the plan I said above.)

[gnprice]

matching Zulip web's ordering […] for emoji vs. other emoji

This part will also require essentially the same data, so I'll leave it out of the near-term issue. The emoji portion of the ordering is in CLDR here:
https://unicode.org/cldr/charts/46/collation/root.html
https://github.com/unicode-org/cldr/blob/9bbbc7769d6/common/collation/root.xml#L924-L1364
(The spec path leads through https://tc39.es/ecma402/#sec-intl.collator.prototype.compare to https://unicode.org/reports/tr10/ . I don't think we'll ever want to spend the time to read the latter in full — that'll be ICU4X's job.)

We do have a separate plan to get a version of that emoji ordering data, from the server:

• Order Unicode emoji options the same way as web #1201

So once we've done that, we might revisit the hack to use that information, if this issue for the full solution is still open.

[gnprice]

OK, filed:

• Sort leading emoji first in channel names to match web #1202

And I'll push this back to M7 Future.