Add unicode sequences support

[oliger]

Description

This PR follows up on #4305. It adds support for flag, keycap and tag unicode sequences.

Example

Without this PR:

CleanShot.2021-06-08.at.17.37.13.mp4

With this PR:

CleanShot.2021-06-08.at.17.39.47.mp4

Context

Improves getCharacterDistance so that it handles all Unicode sequences. Stop reversing the string when moving from right to left. Split it by codepoint and iterate from first to last when moving from left to right and iterate from last to first when moving from right to left.

Checks

• The new code matches the existing patterns and styles.
• The tests pass with yarn test.
• The linter passes with yarn lint. (Fix errors with yarn fix.)
• The relevant examples still work. (Run examples with yarn start.)
• You've added a changeset if changing functionality. (Add one with yarn changeset add.)

[changeset-bot]

🦋 Changeset detected

Latest commit: f584aeb

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
slate	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[dylans]

Overall I think this is really amazing. I gave a few specific questions and feedback.

One note, parts of this change are very specific to knowledge about unicode, so more comments and references would be a good idea in my opinion.

I'm happy to review again after you've had time to address my suggestions.

[dylans]

Thanks @oliger, this looks great to me. I'm going to leave it open for a day or two in case anyone else has suggestions before we land it.

[dylans]

@oliger please review the failing test, thanks!

[oliger]

Hello @dylans! Thanks for the review. A made some requested changes and left a comment. Concerning the comments I do agree that a bit more would be better. I am also sure that the code could be edited to be more readable... But unfortunately, I do not have the time to focus on this at the moment.

[oliger]

Looking at why the tests are failing... This is curious 🤔

[oliger]

@dylans tests are back to green. I also fixed a mistake I made while resolving conflict.

[gitcatrat]

@oliger How well do you remember all the unicode logic here?

I think you broke one case: variation selector, because I'm pretty sure that my initial unicode logic supported it.

• https://emojipedia.org/smiling-face/
• https://emojipedia.org/frowning-face/

Try to paste them here: https://www.slatejs.org/examples/richtext

Both emojis end with the following variation selector: https://emojipedia.org/emoji/%EF%B8%8F/

[oliger]

@gitcatrat Hello, it seems that the issue is related some BMP emojis not being white listed in isBMPEmoji. Not sure why it is not in the list... That being said we might want to change the code so that it does not rely on a white list.

I have no idea if it was working before and I cannot test as I have limited access to a computer at the moment. I am not sure if I'll have the time to focus on this anytime soon.