rustdoc-json: Postcard output

[aDotInTheVoid]

r? @ghost

What

rustdoc --output-format=postcard is like rustdoc-json, but using https://postcard.rs/ / https://docs.rs/postcard/1.1.1/ instead of JSON.

Why

JSON Size and speed isn't great. People want more speed, and smaller docs. There are proposals to make the JSON smaller (and therefor faster) by making field-names shorter, and omitting them when the value is the default. But

How good is it?

In a very unscientific benchmark for aws-sdk-ec2, it's ~3.6x smaller (255MiB vs 69 MiB) and ~1.8x faster to deserialize (1.6273 s vs 914.05 ms)

What's the metaformat

• 22 bytes of magic numbers
• varint(u32) format version
• Crate as usual

This way, users can look at the magic number to check it's a rustdoc-json-postcard file, then read the version number to know if they can decode it. Only then can they deserialize the Crate itself. I plan to write a library that does this, so it's easy to do well.

Why is this a draft

• Needs more tests of CLI flag interactions
• Currently both HtmlRenderer and JsonRenderer are configures from the same options, we should change this
• I want to make it more principled how rustdoc before the format inspects .is_json() instead of the current hacks
• Compiletest changes should be spun into their own thing
• Docs

[aDotInTheVoid]

CC @obi1kenobi @nnethercote @LukeMathWalker

[aDotInTheVoid]

A friend points out https://hackers.town/@zwol/114155807716413069, with advice on how to design a magic number.

[nnethercote]

Having 'Json' in there seems perverse :)