Re-consider use of escapes in canonical N-Quads

[gkellogg]

Re-consider the use of UCHAR and ECHAR escapes in N-Triples/N-Quads canonicalization. The 1.1-based recommendation prohibits the use of UCHAR (U+XXXX) and allows ECHAR only for U+0022 (quote \"), U+005C (backslash \\), U+000A (LF \n), and U+000D (CR \r). However, the use of control characters can obfuscate text when presented, creating a potential security concern.

A future version may consider requiring all characters between U+0000 and U+001F (other than U+000A (LF) and U+000D (CR)) and U+007F (_DEL) to be represented using UCHAR. Characters that can be represented using ECHAR MUST use that representation. All other code points MUST NOT be represented by UCHAR.

[gkellogg]

See #2 (comment).

[gkellogg]

Probably also need U+007F _DEL to be UCHAR escaped.

[gkellogg]

From @afs in https://github.com/w3c/rdf-n-quads/pull/17/files#r1108143303:

Unfortunately, the IRI rule allows more than the characters of a legal IRI so if the goal is a canonical form that corresponds to the grammar (I think it should), it has to cope with this situation.

Raw backspace, tabs, null and form-feed anywhere in the canonical form leads to obfuscation attacks. If this canonical form is used for RCH, these will need to be addresses or a positive justification made for them. Otherwise, the security considerations sections will be advising differently to the canonical representation.

It would be good if there were a simple check to see whether a claimed canonical form actually hiding an obfuscation. One way is the presences of any the most dubious codepoints.

[gkellogg]

Unfortunately, the IRI rule allows more than the characters of a legal IRI so if the goal is a canonical form that corresponds to the grammar (I think it should), it has to cope with this situation.

This treads on erratum 29 (which was actually more about terminology) where the grammar for IRI is from a spec that never made it to standard. We may need to define the parts of RFC3987 in RDF Concepts to be able to fall back on it.

Given that, I'd say that saying the the value matched by the IRIREF terminal MUST be a valid IRI based on that ABNF grammar. (We also need to comb through the use of IRI terminology, as @pchampin actually suggested).

[afs]

re: RFC3987.

Yes, good point. It would be good to do something about that.
The simplest would be to adopt the text from the proposed standard (sections 2 & 5). We then don't have any conversion text nearby

Issue created: w3c/rdf-concepts#15, tracked by w3c/sparql-query#13.

[afs]

IRIREF terminal MUST be a valid IRI

We have the liberal grammar to remove implementation burden. Provided a dubious IRI conforms to IRIREF everything should work (and there are often dubious IRIs in some public datasets).

There are parts of RFC 3986 which are "should". One example is "no empty port" (because they weren't banned originally?).

Valid IRIs come in layers - URI schemes define addition syntax constraints which a general processor can't always know about.

Other specs we aren't touching also have the same grammar rule.

I think SHOULD is better.

[gkellogg]

Just a datapoint for RDF Canonicalization. If we change to escape \f and ' it will break expected results for test060. All other tests pass.

input:

<urn:ex:s> <urn:ex:000:empty> "" .
<urn:ex:s> <urn:ex:001:simple> "simple" .
<urn:ex:s> <urn:ex:002:quote> "\"" .
<urn:ex:s> <urn:ex:003:backslash> "\\" .
<urn:ex:s> <urn:ex:004:nl> "\n" .
<urn:ex:s> <urn:ex:005:cr> "\r" .
<urn:ex:s> <urn:ex:006:all> "\"\\\n\r" .
<urn:ex:s> <urn:ex:007:uchar> "\u0022\u005c" .
<urn:ex:s> <urn:ex:008:echar> "\t\b\n\r\f\"\'\\" .
<urn:ex:s> <urn:ex:009> "\\u0039" .
<urn:ex:s> <urn:ex:010> "\\n" .
<urn:ex:s> <urn:ex:011> "\\\\" .
<urn:ex:s> <urn:ex:012> "\"\"" .
<urn:ex:s> <urn:ex:013> "\\\\\\" .
<urn:ex:s> <urn:ex:014> "\"\"\"" .
<urn:ex:s> <urn:ex:015> "\u221e" .
<urn:ex:s> <urn:ex:016> "∞" .

expected results:

<urn:ex:s> <urn:ex:000:empty> "" .
<urn:ex:s> <urn:ex:001:simple> "simple" .
<urn:ex:s> <urn:ex:002:quote> "\"" .
<urn:ex:s> <urn:ex:003:backslash> "\\" .
<urn:ex:s> <urn:ex:004:nl> "\n" .
<urn:ex:s> <urn:ex:005:cr> "\r" .
<urn:ex:s> <urn:ex:006:all> "\"\\\n\r" .
<urn:ex:s> <urn:ex:007:uchar> "\"\\" .
<urn:ex:s> <urn:ex:008:echar> "\n\r
                                   \"'\\" .
<urn:ex:s> <urn:ex:009> "\\u0039" .
<urn:ex:s> <urn:ex:010> "\\n" .
<urn:ex:s> <urn:ex:011> "\\\\" .
<urn:ex:s> <urn:ex:012> "\"\"" .
<urn:ex:s> <urn:ex:013> "\\\\\\" .
<urn:ex:s> <urn:ex:014> "\"\"\"" .
<urn:ex:s> <urn:ex:015> "∞" .
<urn:ex:s> <urn:ex:016> "∞" .

cc/ @dlongley

[philarcher]

Just to note there that the RDF Canonicalization & Hash WG resolved to support the approach to escaping proposed in this issue, see https://www.w3.org/2023/03/15-rch-minutes.html#r01

[yamdan]

Please allow me to confirm this for better understanding.
If we adopt the proposed escape method, I understand that the expected result for test060 will change to the following. Is this correct?

expected results (after the new escaping):

<urn:ex:s> <urn:ex:000:empty> "" .
<urn:ex:s> <urn:ex:001:simple> "simple" .
<urn:ex:s> <urn:ex:002:quote> "\"" .
<urn:ex:s> <urn:ex:003:backslash> "\\" .
<urn:ex:s> <urn:ex:004:nl> "\n" .
<urn:ex:s> <urn:ex:005:cr> "\r" .
<urn:ex:s> <urn:ex:006:all> "\"\\\n\r" .
<urn:ex:s> <urn:ex:007:uchar> "\"\\" .
<urn:ex:s> <urn:ex:008:echar> "\t\b\n\r\f\"\'\\" .
<urn:ex:s> <urn:ex:009> "\\u0039" .
<urn:ex:s> <urn:ex:010> "\\n" .
<urn:ex:s> <urn:ex:011> "\\\\" .
<urn:ex:s> <urn:ex:012> "\"\"" .
<urn:ex:s> <urn:ex:013> "\\\\\\" .
<urn:ex:s> <urn:ex:014> "\"\"\"" .
<urn:ex:s> <urn:ex:015> "∞" .
<urn:ex:s> <urn:ex:016> "∞" .

[gkellogg]

Yes, that's what my implementation produces.