Normative: bug re Annex B production for AtomEscape

[jmdyck]

The main-body definition of AtomEscape says (in part):

AtomEscape[U, N] :: CharacterEscape[?U]

but the Annex-B definition of AtomEscape says:

AtomEscape[U, N] :: CharacterEscape[~U, ?N]

So if AtomEscape is 'invoked' with +U, main-body uses CharacterEscape[+U] whereas Annex-B uses CharacterEscape[~U].

This is presumably a bug, since Annex B claims that it doesn't change any [+U] syntax.

---

It looks like the bug was introduced in 251dcef (for PR #403), where (using the old notation for grammatical parameters)

AtomEscape[U] ::
  [+U] CharacterEscape[U]
  [+U] CharacterClassEscape
  [~U] CharacterClassEscape
  [~U] CharacterEscape

was changed to:

AtomEscape[U] ::
  CharacterClassEscape
  CharacterEscape

but should have been changed to:

AtomEscape[U] ::
  CharacterClassEscape
  CharacterEscape[?U]

(Compare to the same commit's parallel change to the ClassEscape production, where CharacterEscape does get the [?U].)

@anba, please confirm.

[mathiasbynens]

@jmdyck's reasoning makes sense to me. I'd love to hear from @anba in case I'm missing something.

[bakkot]

I agree with @jmdyck. Having CharacterEscape parsed with ~U here would mean /\z/u would be legal (NB z is not a valid escape), as derived from CharacterEscape[U] :: IdentityEscape[?U] -> IdentityEscape[U] :: [~U] SourceCharacterIdentityEscape. That definitely is not the intent, and indeed all major engines reject /\z/u.

[bakkot]

Fixed by #2235.