Emojis can get cut off in the terminal

[alexdima]

Testing #34868

emoji-test.txt

• save this file as UTF8 with bom (I think it is already)
• configure "terminal.integrated.scrollback": 5000
• run chcp 65001
• run cat emoji-test.txt

Observe:

cut emojis e.g.

combining sequences that Chromium supports e.g.

[Tyriar]

@alexandrudima what font is this with? For all the monospace fonts I've tested they consistently only cover 2 cells.

Skin tone modifiers are not just a matter of joining the characters in the terminal as the shell doesn't understand that these characters are one. This is as designed for the time being as we're relying on the older tech in shells. See xtermjs/xterm.js#942

[alexdima]

@Tyriar Did you attempt to reproduce this given the steps I have provided ? I did not customize the font, otherwise I would have mentioned it.

[Tyriar]

@alexandrudima I don't have access to a Windows machine, trying to get enough information to fix without access.

[alexdima]

Sure, if it helps it is also reproducible on OSX with default settings with this:

2639 FE0F                                  ; fully-qualified     # ☹️ frowning face

[Tyriar]

Ah I can reproduce as well, seems that not all emojis are being handled correctly:

This may have been introduced in xtermjs/xterm.js@c4da832#diff-59a6c78910b740a7b8381b034eeb73a2. Chars are now drawn and their dimensions measured to see if they consume the following cell. Previously I used this regex to test if it was an emoji:

char.search(/([\uD800-\uDBFF][\uDC00-\uDFFF])/g) >= 0;

[alexdima]

@Tyriar this is how I look for emojis in the editor input:

/**
 * Generated using https://github.com/alexandrudima/unicode-utils/blob/master/generate-emoji-test.js
 */
const CONTAINS_EMOJI = /(?:[\u231A\u231B\u23F0\u23F3\u2600-\u27BF\u2B50\u2B55]|\uD83C[\uDDE6-\uDDFF\uDF00-\uDFFF]|\uD83D[\uDC00-\uDE4F\uDE80-\uDEF8]|\uD83E[\uDD00-\uDDE6])/;

export function containsEmoji(str: string): boolean {
	return CONTAINS_EMOJI.test(str);
}

if (/\uFE0F/.test(potentialEmojiInput) || strings.containsEmoji(potentialEmojiInput)) {
  // it is probably an emoji
}

[Tyriar]

@alexandrudima thanks for this, I'll probably just test this before doing the actual measure (which covers other unicode chars) 👍

[Tyriar]

Turns out this was already fixed upstream, I just wasn't on the latest. Fixed in f9a040f

[Tyriar]

No plan to fix the issue where the emoji is so width is expands beyond 2 cells. I'll keep it in mind though if we tweak the cell ownership model.