U+005E (^) in path percent-encode set but only sometimes encoded?

[wolfgang42]

I’m filing this issue here because my best guess is that it’s a bug in Ada somewhere, but I’m quite confused and having trouble interpreting the WhatWG URL spec so maybe this is expected behavior?

URL	Ada	Firefox	Chrome
http://foo/bar^baz	http://foo/bar^baz	http://foo/bar%5Ebaz	http://foo/bar%5Ebaz
http://foo/bar^baz>	http://foo/bar%5Ebaz%3E	http://foo/bar%5Ebaz%3E	http://foo/bar%5Ebaz%3E
quux://foo/bar^baz	quux://foo/bar^baz	quux://foo/bar^baz	quux://foo/bar%5Ebaz
quux://foo/bar^baz>	quux://foo/bar%5Ebaz%3E	quux://foo/bar^baz%3E	quux://foo/bar%5Ebaz%3E

My reading of whatwg/url#607 is that ^ is supposed to be percent-encoded in pathname more or less unconditionally (it seems like Firefox is maybe only doing it for special schemes, but that’s a side point), and frankly #908 seems like it ought to have straightforwardly implemented exactly that.

I haven’t tried to reverse-engineer Ada’s exact behavior, but it seems like caret is only getting encoded if there’s also something else in the URL that needs encoding at the same time, which doesn’t seem right. I guess the WPT tests didn’t catch this because they test all of the percent-encodings at once in a single URL, and so bypass this bug.

[wolfgang42]

Hm, this seems incredibly suspicious:

ada/src/checkers.cpp

Lines 85 to 91 in dbd9e11

	ada_really_inline constexpr uint8_t path_signature(
	std::string_view input) noexcept {
	// The path percent-encode set is the query percent-encode set and U+003F (?),
	// U+0060 (`), U+007B ({), and U+007D (}). The query percent-encode set is the
	// C0 control percent-encode set and U+0020 SPACE, U+0022 ("), U+0023 (#),
	// U+003C (<), and U+003E (>). The C0 control percent-encode set are the C0
	// controls and all code points greater than U+007E (~).

It seems like there are maybe optimization paths that aren’t sufficiently covered by any of the tests or fuzzers?

I applied a very small patch which seems to fix my problem, but I’m extremely nervous that there may be other places where the list of magic numbers need to be updated:

diff --git a/src/checkers.cpp b/src/checkers.cpp
index 346c474f..c5ad7fa4 100644
--- a/src/checkers.cpp
+++ b/src/checkers.cpp
@@ -67,7 +67,7 @@ static constexpr std::array<uint8_t, 256> path_signature_table =
       std::array<uint8_t, 256> result{};
       for (size_t i = 0; i < 256; i++) {
         if (i <= 0x20 || i == 0x22 || i == 0x23 || i == 0x3c || i == 0x3e ||
-            i == 0x3f || i == 0x60 || i == 0x7b || i == 0x7d || i > 0x7e) {
+            i == 0x3f || i == 0x5e || i == 0x60 || i == 0x7b || i == 0x7d || i > 0x7e) {
           result[i] = 1;
         } else if (i == 0x25) {
           result[i] = 8;

[anonrig]

How's the behavior in Safari? Chrome's URL parser implementation is not a good baseline.

[wolfgang42]

I don’t have ready access to a Mac to test, unfortunately, but I’m also not sure how relevant that is? Having tracked down the code I link above it now seems unlikely to me that Ada’s current behavior was intended either way.

[CarlosEduR]

How's the behavior in Safari?

Safari appears not to percent-encode it.

URL	Safari
http://foo/bar^baz	http://foo/bar^baz
http://foo/bar^baz>	http://foo/bar^baz%3E
quux://foo/bar^baz	quux://foo/bar^baz
quux://foo/bar^baz>	quux://foo/bar^baz%3E

[anonrig]

@annevk is this a bug with WebKit or is this intended? Would you mind clarifying the relevant spec change?

[annevk]

WebKit is aligned with the specification as far as I know. You have to use Safari Technology Preview 216 or newer or Safari on iOS or macOS 26.