Should we use less strict uri parsing?

[daurnimator]

At the moment url parsing is strict: it doesn't allow forbidden characters like {.

Should things like request.new_from_uri normalise to the encoded form?

References

• https://stackoverflow.com/questions/40568/are-square-brackets-permitted-in-urls
• https://bugzilla.mozilla.org/show_bug.cgi?id=407172
• https://stackoverflow.com/questions/6741476/httpclient-problem-with-urls-which-include-curly-braces (confused user)
• https://perishablepress.com/stop-using-unsafe-characters-in-urls/ (sort of a counter-point)

[RyanSquared]

Perhaps add either a strict boolean argument or add an escape_uri(uri) function, would be what I suggest. (Is there a URI module? If so, you could put escape_uri in there).

[daurnimator]

Perhaps add either a strict boolean argument

To what? There are many functions that take uris, and it will be a pain to add an extra parameter to all of them...

add an escape_uri(uri) function, would be what I suggest.

There are many ways to escape a URI depending on which piece of the URI you're talking about.

Is there a URI module? If so, you could put escape_uri in there

lua-http uses lpeg_patterns.uri for uri parsing.
The http.util module has a few various functions.

---

I'm actually tempted to close this issue. I think being strict about URIs is probably the better path here. Various encodings and pieces of software rely on invalid uri characters to work well. e.g.

• curl uses square brackets for globbing unless you pass -g
• in markdown, you surround links with parenthesis (and hence links can't contain a close parenthesis)
• URI Templates

The biggest exception seems to be browsers, where anything starting with a scheme seems to get thrown through a uri escaping process. That hasn't been great for everyone either (see https://bugzilla.mozilla.org/show_bug.cgi?id=473822 ).

I originally created this issue when I was working with postgrest's foreign entitys syntax and wanted to copy/paste from docs to lua-http; but now I don't think it's worth it.

[daurnimator]

More related links:

• https://daniel.haxx.se/blog/2016/05/11/my-url-isnt-your-url/
• It's not immediately clear that "URL syntax" and "URL parser" conflict whatwg/url#118

[daurnimator]

Another link: https://tools.ietf.org/html/draft-ruby-url-problem-01