Automate docs publishing

[kennykerr]

Every time I publish a new release of the Windows crate, I need to manually republish the docs. This is a tedious and error-prone process that should be automated. The docs publishing also weeds out issues in the Windows crate as it builds all of the APIs. This should ideally be used to validate CI builds (with cargo check) to avoid surprises after release.

This was unavoidable before as the Rust compiler was way too slow and used too much memory to build the docs in a VM. But a lot has improved since then and hopefully it can now be done.

[kennykerr]

I don't think I'm going to get to this any time soon. The build is also still pretty slow so I doubt the CI build VMs will support it either.

[tim-weis]

Is this something I could working on? If only to identify current issues that would need to be resolved, and collecting prerequisites for fully automating this task, to determine when it's time to do so.

Could you quickly summarize what's required for this, and comment on whether this can be done outside of Microsoft/in a forked repository? I mean, I'd love to free you up some time to invest into more relevant areas, establishing quality gates along the way as a bonus.

[kennykerr]

Sure I'd love the help. The challenge is just to come up with a more sustainable story for API docs. I set up this repo:

https://github.com/microsoft/windows-docs-rs

Out of which I publish documentation here:

https://microsoft.github.io/windows-docs-rs/

This uses GitHub Pages for hosting.

This is just an experiment but some developers have expressed that this is a very valuable tool. So part of the challenge is to figure out whether this is the right approach for Windows API documentation in Rust and if so improving the quality of the experience, from automating the publishing/validation all the way to profiling end to end to figure out how we can make it work more efficiently/reliably. There are performance bottlenecks in rustc, cargo-doc, and in the Windows crate itself. Then there's #931 and whether we can find a way to weave in actual documentation, not just API signatures.

[kennykerr]

Oh, and there's #432 that may impact this as well.

[tim-weis]

I hadn't realized how complex this process was. Thanks for having gone through this for every release in the past.

Now personally, I find the generated documentation to be extremely helpful. It makes it easy to discover which module any given API or type gets generated into. The information is presented using a familiar color theme and layout, and allows for quick navigation. ILSpy solves the discoverability aspect, though it is very CLR-centric and fails on all the other points. Plus, it necessarily cannot know the final signature of the Rust projection.

With all that, I don't see the need for this type of documentation going away anytime soon. I will invest time into finding ways to automate the process as much as possible, even if it cannot be run as part of a CI workflow for the foreseeable future. Profiling is another area that would certainly benefit from being easy to do. I'm not sure how to go about that yet, without getting the respective Rust teams involved (e.g. in providing public symbol information for their Windows binaries).

#931 is likely to open up much more impactful ways to streamline everyday development tasks. Having access to API documentation right from within any given IDE would be a massive quality-of-life improvement. Though, as I understand, this would require code generator involvement, adding rust-doc comments as needed (correct me if I'm wrong, or overlooking alternatives). If I can I will try to see if that can be made to work, and gather estimates on the impact this might have on code generation. Maybe this will have to wait until there is some more progress on #432.

I will leave comments here whenever I make progress, and ask questions in the meantime, if you don't mind.

[jyn514]

There are performance bottlenecks in rustc, cargo-doc, and in the Windows crate itself.

@kennykerr do you happen to have on hand exactly what's slow? I'm still interested in improving rustdoc's performance :)

[kennykerr]

Were you able to build the bindings crate on windows-docs-rs? That does a great job demonstrating the performance woes. 😉

[jyn514]

@kennykerr no, I responded on microsoft/windows-docs-rs#6

[yoshuawuyts]

Now that we're making progress on publishing a pre-built version of windows-rs to crates.io (see #1225, #1232, #1233), do we still want to track doc publishing as a separate item?

[jyn514]

@yoshuawuyts are you sure docs.rs will be able to compile it? There's a 15 minute time limit on all builds.

Also, could you please set the metadata so the docs only get built for one target? That will reduce the time needed for docs.rs quite a lot. https://docs.rs/about/metadata

[kennykerr]

@jyn514 do you mean something like this: #1237

The time to compile is getting pretty close. 😉

C:\git\windows-rs>cargo doc --no-deps --all-features --target-dir c:\git\windows-docs-rs\docs
   Compiling proc-macro2 v1.0.30
   Compiling unicode-xid v0.2.2
   Compiling syn v1.0.80
   Compiling windows_x86_64_msvc v0.22.0 (C:\git\windows-rs\crates\targets\x86_64_msvc)
   Compiling windows_quote v0.22.0 (C:\git\windows-rs\crates\deps\quote)
   Compiling windows_reader v0.22.0 (C:\git\windows-rs\crates\deps\reader)
   Compiling quote v1.0.10
    Checking windows_gen v0.22.0 (C:\git\windows-rs\crates\deps\gen)
   Compiling windows_macros v0.22.0 (C:\git\windows-rs\crates\deps\macros)
 Documenting windows v0.22.0 (C:\git\windows-rs)
    Finished dev [unoptimized + debuginfo] target(s) in 10m 55s

[kennykerr]

rustdoc does however use upwards of 10GB of RAM and produce 2.5GB of documentation so will probably trip another limit.

[jyn514]

If you open an issue in https://github.com/rust-lang/docs.rs/issues I can raise the memory limit (it's 3 GB by default, but we have existing crates with an 8 GB limit and I don't think 10 is unreasonable).

[kennykerr]

Unfortunately, it looks likes its closer to 18 GB...

[jyn514]

Yeah, that wouldn't be feasible, our build machine isn't large enough ... We plan to upgrade it sometime in the medium future, I'll keep you posted about that.

[kennykerr]

No worries, I'm experimenting with injecting #[cfg(not(doc))] or #[doc(hidden)] into the generated code to see if it helps Rust not consume so much memory.

[kennykerr]

Now that the pre-packaged APIs are available I really need to find a way to document which features to enable for each API.

Here's an example: https://docs.rs/web-sys/0.3.55/web_sys/struct.AbortSignal.html

[jyn514]

@kennykerr you can replicate that "requires the following crate features to be activated" text with doc(cfg(feature = "AbortSignal")). If you have cfg(...) already, rustdoc will autogenerate the doc(cfg(...))) automatically as long as you have feature(doc_cfg) in the crate root.

[kennykerr]

Thanks!

[kennykerr]

The docs have now been updated to indicate which features a particular API requires:

https://microsoft.github.io/windows-docs-rs/doc/windows/