Numeric separators allowed in extended Unicode escape sequences?

[anba]

The current proposal allows "\u{10_ffff}". Is this intentional?

[mathiasbynens]

Not intentional, per September 2017 TC39 meeting.

[rwaldron]

@anba @mathiasbynens 39d0bc0

The solution I chose is to explicitly define the MV such that NLS has no impact on the resulting value.

[mathiasbynens]

With the change, e.g. '\u{10_FFFF}' is now explicitly allowed and equivalent to '\u{10FFFF}'.

I’d have preferred to disallow it in escape sequences, following the “no real benefit there” logic in https://github.com/tc39/proposal-numeric-separator#octal-literal, but don’t feel strongly about it. What do you think, @anba?

[rwaldron]

More on this:

• The SV of UnicodeEscapeSequence :: u { CodePoint } is the UTF16Encoding of the MV of CodePoint.

CodePoint ::
HexDigits but only if MV of HexDigits ≤ 0x10FFFF

HexDigits ::
HexDigit
HexDigits NumericLiteralSeparator _opt HexDigit

And the NLS spec adds:

• The NumericLiteralSeparator :: _ has no MV.
• The MV of HexDigits :: HexDigits NumericLiteralSeparator HexDigit is (the MV of HexDigits × 16) plus the MV of HexDigit.

[rwaldron]

To be clear, this was only one of 3 possible solutions that @leobalter and I discussed. We agreed that an explicit Static Semantics could be written as well.

[leobalter]

I don't have any strong opinion if we should disallow NLS in escape sequences or not. My goal is only to fetch the correct spec for what ends up as the chosen one.

One feedback that might be helpful is collecting implementors PoV. It disallowing NLS in escape sequences is easier to implement, I'm all in.

I'd be happy to help writing an explicit error to disallow.

[littledan]

I don't feel really strongly, but my intuition was the same as @mathiasbynens 's--I don't see a strong reason to permit numeric separator in Unicode literals. It wasn't clear to me during the Stage 3 presentation that you were proposing to allow it--I misunderstood that the fix was to disallow it, and allowing numeric separators there was an accident.

[ajklein]

FWIW, my understanding of the presentation matched @littledan's.

[rwaldron]

Reopening for explicit Static Semantics to produce a SyntaxError whenever _ appears in "\u{10_ffff}"

[littledan]

Ping, it would be good to get clarity on this issue before integrating with the BigInt specification (or implementing and shipping).

[leobalter]

@littledan I believe the decision is throwing when a Num. separator appears in escape sequences. We might need to update the specs to match that.

[leobalter]

This seems ugly, but we could get a fix to the escape sequence by changing CodePoint to:

CodePoint ::
  CodePointDigits "but only if MV of CodePointDigits ≤ 0x10FFFF"

CodePointDigits ::
  HexDigit
  CodePointDigits HexDigit

or:

CodePoint ::
  HexDigits "but only if MV of CodePointDigits ≤ 0x10FFFF"

HexIntegerLiteral ::
  0x HexDigits [+NSL]
  0X HexDigits [+NSL]

HexDigits[NSL] ::
  HexDigit
  [~NSL] HexDigits HexDigit
  [+NSL] HexDigits NumericLiteralSeparator_opt HexDigit

I'm not sure if I like adding more grammar production flags.

[rwaldron]

I believe the decision is throwing when a Num. separator appears in escape sequences.

Confirm, these are the desired semantics.

[rwaldron]

@leobalter I've updated the spec to include your first option.

NumericLiteralSeparator is not allowed

Tests to follow.

[littledan]

The fix in 06b17eb LGTM. (I was surprised that CodePoints is under the template literal heading, but seems like that's what string literals, RegExps, etc cross-reference).

Does MV need an update to explain what the value of CodePointDigits is, or is that considered clear from analogy with its other productions?