HTTP.jl version 1.0.0

[fredrikekre]

I think it would be nice to make a push to version 1.0 of HTTP.jl. There are enough breaking PRs in the pipeline that would be good to get in:

• Remove AWS auth layers (Removed AWS4AuthRequest Layer | Deprecate HTTP.Post without headers #486)
• Separate HTTP.request entrypoint from internal Layers.request function (Separate HTTP.request from Layers.request #469)
• Stop calling close on response_stream (Do not call close() on the response_stream #775)
• Remove support for http pipelining ([BREAKING]: Remove client support for pipelined requests #783)
• Decide what to do with duplicate query param keys (add support for duplicate query param keys #433) (Edit: this code has been moved to URIs.jl)
• Change internal dispatch stack to be value-based instead of type-based ([BREAKING]: Remove client support for pipelined requests #783 (comment))

Some other things I think should be done before 1.0:

• Drop support for Julia 1.0 (and remove legacy code necessary to support it, e.g. the @v1_3 macro)
• Remove (or at least comment out) code related to IODebug (this results in a lot of invalidations, see Method invalidations #712)
• Rewrite debug logging to use Julia's logging infrastructure?
• Thorough websockets code review/fix open issues (currently 17 open issues as of 5/27)
• Overhaul package docs; cleanup inline docstrings, proper manual with guides, quickstart, examples, reference, etc.
• Careful review of core Server code; simplify by removing things like ratelimiting that can be done in middleware and try to make the core server logic as performant and robust as possible; I plan on doing a thorough comparison with the golang core server code as a reference

Anything else?

[c42f]

Great list 👍

Custom layers

The HTTP stack changes will make https://github.com/JuliaComputing/HTTPDownloads.jl less of a hack. As part of that, it'd be nice to have a good API to modify the default stack. The current HTTP.insert_default! / HTTP.remove_default! is pretty weird.

There's also some other oddities which you encounter if you try to implement custom layers. For example, the following hack to check whether the body was streamed:

https://github.com/JuliaComputing/HTTPDownloads.jl/blob/a5fd409c91a3a669c2d8e0b5fc078ad49e63f46a/src/HTTPDownloads.jl#L112

Error reporting

Another big wishlist item I have would be some consistency in error reporting, but after I started writing that I wasn't sure it was 1.0 material, so I moved that to a new issue: #792

[quinnj]

I'd like to add 2 items to the 1.0 list:

• Overhaul of HTTP.Router; it's using an eval-unsafe hack to do route dispatch, which, while clever, currently hurts performance (use of @eval inhibits performance #747), and requires statically known routes. I've had sketches around for a while around how to redesign, just need to get around to doing it
• Thorough review of WebSockets code; there are too many open issues that need to be reviewed so we can get a feel for how confident we can be in that code

Now, while I'm personally ok tackling both these issues, I'm also personally aware of my available time/efforts and so this means waiting a bit on a proper 1.0 release. I'd like to counter-propose that we finish the value-based internal layers PR (which closes the 3 remaining tasks in the top list here) and make a 0.10 release, since we've had some important multithreaded fixes just sitting on master for a while now.

[clarkevans]

Will 1.0 support (or at least enable the option to have) streams (e.g. chunked)? This is needed for large file support. Pardon if I'm a bit behind the times, but this is marked in my head as being a current shortcoming of existing HTTP.jl implementation.

[quinnj]

Will 1.0 support (or at least enable the option to have) streams (e.g. chunked)? This is needed for large file support. Pardon if I'm a bit behind the times, but this is marked in my head as being a current shortcoming of existing HTTP.jl implementation.

This has been supported for quite a while; you can see the docs for this under the "Streaming Options" in the docs.

[quinnj]

I've updated @fredrikekre's original post with 3 additional tasks: websockets cleanup, docs overhaul, and server code review. I plan on tackling these over the next few weeks. I've marked a few other select issues as 1.0 milestone that I think would be good to address before 1.0. If anyone has other bigger issues/things for 1.0, speak up! I think we're getting close-ish now.