RFC: everything in rust docs

[sam0x17]

This came up in our FRAME weekly meeting, some internal docs discussions, and some conversations with community members on discord and elsewhere, and across the board there has been wide support for it so re-posting here for a more public discussion.

Everything in Rust Docs

Tutorials, documentation explaining what various blockchain and FRAME concepts are...everything... can and should live in rust-docs in our (soon-to-be-a-thing) monorepo because this is the lowest friction way for people to learn about these things (inline, at the exact moment when their confusion occurs while coding).

The way this could work is with a substrate-concepts / frame-concepts or something like that sub-crate (probably dev dependency of frame crate or root substrate crate) that is docs-only and has a bunch of empty modules (or ZSTs) that correspond with various concepts, tutorials, etc. The rust docs for these modules would be the content of tutorials or explanations of concepts, etc., and can use a docify-like approach to embed markdown files for the longer ones especially tutorials that are easier to maintain as a .md file. Images we can easily embed, videos would become links which I think is fine.

Then for example anywhere you mention something related to "staking", you could directly doc-link to Staking, and since these are dev-only dependencies, we can link to them from anywhere in substrate without causing dependency-related strain.

The exact setup I'm happy to iterate on with a working example that maximizes what we can docify-embed, but this is the general idea.

Advantages

• If all of these things are version-controlled under the same repo, we solve the extremely difficult problem of out-dated/semver-tagged tutorial/concept/etc links.
• If everything is in the same repo, we can use tools like docify to directly embed code examples from elsewhere in substrate within the tutorials that reference them such that they will automatically be updated when they are changed in the code.
• Once we start publishing the monorepo to crates.io, all of our docs will get massive SEO and google visibility from docs.rs, lib.rs and all the docs.rs clone sites, much better than we could ever do with our own docs domains because of how pagerank works. This also means we don't need devops/etc to maintain these sites, we can just lean on crates.io to publish everything.
• Search within our rust docs becomes like a superpower that will tell you anything you want to know about our ecosystem / blockchain technology / etc.
• Common blockchain and substrate/FRAME concepts become significantly more discoverable/learnable when browsing our rust docs.
• Developers have direct line-of-sight into all of our docs without going to a different repo and can update them as part of any PR that makes changes to substrate.
• Doc changes are published in lock-step with code changes that those docs relate to.

Would love some thoughts on this and further discussion. General reception so far has been highly positive so would love to hear some dissenting views.

cc @kianenigma @nukemandan @liamaharon @gupnik @franciscoaguirre @KiChjang @ggwpez @juangirini @aaronbassett

[juangirini]

I really like the idea of having a single source of truth, I think it simplifies maintenance a lot.
They might not be dealbreakers but it's worth mentioning that a downside I see is that we could break backlinks when updating our code/structure. And that we would be limited by what it is possible within Rust Docs.

[liamaharon]

I'm all for this for the described reasons.

I really like the idea of having a single source of truth, I think it simplifies maintenance a lot. They might not be dealbreakers but it's worth mentioning that a downside I see is that we could break backlinks when updating our code/structure. And that we would be limited by what it is possible within Rust Docs.

I believe there're linters we can run as part of CI to prevent broken links.

[juangirini]

I believe there're linters we can run as part of CI to prevent broken links.

There's definitively tools to find broken backlinks, but we won't be able to fix them as controlling URLs will become very difficult. The consequences of this will be confused visitors that will land in a 404. SEO wise won't be a big problem as such broken links would only negatively impact docs.rs as a domain

[sam0x17]

we could break backlinks

This is also a feature though -- CI will fail and you'll have to fix them

As long as all the links are rust doc links like

[`Something`]

And not URL links

Plus we could have some sort of convention around deprecating instead of removing old doc modules so we can link to the new location there

[liamaharon]

There're tools to help catch broken URL links too https://github.com/deadlinks/cargo-deadlinks#usage

[juangirini]

When I said backlinks I meant a organic linking from website xyz.com pointing to docs.rs/whatever, not internal linking. When we update whatever to whatever2, the backlink will be broken. If we controlled docs.rs we could add a redirection link, as we don't control docs.rs we can't do that.
In any case, I don't think is a deal breaker, plus we will be able to solve that with proper semver as docs.rs supports versioning.
Aside from that minor issue, I like this proposal a lot.

[sam0x17]

When I said backlinks I meant a organic linking from website xyz.com pointing to docs.rs/whatever, not internal linking. When we update whatever to whatever2, the backlink will be broken. If we controlled docs.rs we could add a redirection link, as we don't control docs.rs we can't do that.

In any case, I don't think is a deal breaker, plus we will be able to solve that with proper semver as docs.rs supports versioning.

Aside from that minor issue, I like this proposal a lot.

So the nice thing is with docs.rs you get versioned links so we do have some options there. For anything we link to externally we can choose to link to a specific version, in which case it will never break and when new versions are released they will see the red "a newer version is available" at the top left of the page, or we can link to latest in which case we run the risk of broken links, like you said.

But it's just about the most flexibility we could have, and again if we have a careful deprecation policy on naming of items in the docs/tutorials crate we could basically ensure this never becomes an issue by leaving the old name with a link to the new one, as a policy.

[gilescope]

When we go to our new doc portal, (wherever the new version of https://paritytech.github.io/cumulus/ goes to) I'd like it if private documentation is turned on - that's not available in docs.rs and is helpful. E.g. documentation around nft pallet features is all non-public at the moment.

[KiChjang]

The one thing that I don't quite completely like about this approach is the purported "docs situated right next to code" aspect -- this is not quite true. Sure, we can have references and links to actual types, but the reality is that the docs are hosted in a completely different doc module altogether, so it's at least a layer removed from the actual code.

That does change a couple of things about the advantages listed out here:

• We could still get an out-of-date documentation, simply because we forgot to look in the doc module and update the text there;
• Doc changes may not necessarily be published in lock-step with code changes;
• Devs don't really have a direct line-of-sight, unless we purposefully introduce links back to the doc module from the concrete types/structs/functions

[liamaharon]

I'm writing new Migration and Try State tutorial/docs (which could find their way into the frame master tutorial when that is ready), and was initially planning to experiment writing this tutorial/guide format in Rustdocs.

However, after working on it I feel like mdbook is better suited for stepped-tutorials.

• Prettier, easier read
• Better navigation UX
• The ToC on the left hand side of mdbook is easier for the user to navigate through sections, and easier for the developer to write & organise sections than trying to create a new crate for every 'section' in rustdocs.
• It supports testing code examples and dependencies

Everything is still just markdown, so it won't be difficult to migrate to rustdocs in the future if we decide.

I'm still in support of experimenting with all standalone pieces of docs that don't belong in a stepped tutorial being in rustdocs.

[sam0x17]

• Devs don't really have a direct line-of-sight, unless we purposefully introduce links back to the doc module from the concrete types/structs/functions

Yeah, doubling-down on doing this everywhere would be key

[sam0x17]

However, after working on it I feel like mdbook is better suited for stepped-tutorials.

Yeah, this makes me wonder if there might be some way to export this > mdbook or mdbook > this in some automated way where we can get the best of both worlds

[chevdor]

All rustdoc

While I like the idea, I am not convinced using exclusively rustdoc would work for all authors (especially non dev ones). There will always be markdown written and not always embedded in some rustdoc.

As far as code is concerned and direct documentation of the code, afaic there is no doubt this should be rustdoc in the code or linked markdown.

The topic rustdoc vs markdown book will depend of the reader.
Someone coding will prefer rustdoc where all the documentation of a given piece of code is available, whereas for a general reader, a "book" will be better to extract relevant content (and skip less relevant one).

mdbook / static site generator

As for the more generic documentation, I would leave it to the doc team to pick the the solution they can be maintained best.

I see a big benefit in bring all the doc we have together in one browsable/searchable site. That will also contribute keeping a good quality of content.

Everything is still just markdown, so it won't be difficult to migrate to rustdocs in the future if we decide.

I agree as long as links are valid and it may be problematic in "flying" markdown that may not always be in sync with some pieces of code it refers to.

no dup

It is obvious and critical to have no duplication in the doc source, we should not have content in both code (.rs) and markdown and no markdown duplicated for the sake of generating rustdoc and some markdown book.

That being said, I don't think it would be an issue to have part of the doc used in both some rustdoc and some markdownbook/statuic doc site.

URL validity

There are related issues:

• [CI] Add link checker #993
• Fix all broken links #134

The later is the first to be taken care of, then we can enable a fix for #993 to ensure URLs remain valid. That will work for external urls.

Documentation decay

One reality is that some part of the doc become stale over time. This can be due to maintainers of a piece of code to move of to other topics, not being around anymore, or the lack of updates due to oversights.

rustdoc has the edge here since it will help keep the doc in sync with the code whereas some "flying" markdown could get out of sync.

Our CODEOWNERS config was no longer valid in the monorepo, I prepared a fix: #1415

I was also tinkering with the idea of adding a "freshness/anti-decay check" to the markdown at least.

So for instance, a .md file not updated for more than N days (we probably would start with some value in months...) would become invalid and raise a warning in CI.

If is still valid, a simple noop change could reset the counter and mark the validation that the doc was checked and is "fresh" enough.

It is never fun to take over doc written by others but this will/is become necessary to ensure the overall quality.

[chevdor]

see also https://paritytech.github.io/polkadot-sdk/

[sam0x17]

The topic rustdoc vs markdown book will depend of the reader.

The nice thing about using docify here is you can have both -- we can have a folder of markdown files that becomes nested modules of rustdoc and gets published somewhere as an md book

[kianenigma]

This is a great idea, I will adopt it as such:

In the new doc-vision manifesto, there are 3 classes of documents that we produce:

1. A master tutorial
2. Native rust-docs for low level items, functions and Rust items.
3. Reference docs for whatever remains in between that it too broad to be documented in rust-doc, and too specific to be in the master tutorial.

(See here for a more detailed description).

The intention of the docs effort should be to do as much of the above as possible in rust-docs. As noted, these rust-docs could use docify, be processed into .md files, which will itself become a md-book of some sort.

But the main point is, the source of truth is always .rs files.

Some open challenges, questions:

• One downside of this would be that we are adding more to the monotony of the mono-repo: Now, there is a bunch of tutorial/doc related code in the mono-repo as well, which you need to update on your PRs. I think this is acceptable. What we have to remember that not ALL tutorials and ref docs should live in the mono-repo. Only the fundamental ones.
• The tutorial, as I see it for now, is somewhat hard to craft without making every step of the tutorial be a fully compile-able artifact. This artifact is usually a pallet, and sometimes a runtime. This will be a lot of code duplicate, especially if we have different runtimes per step of the tutorial. You can already see this in https://github.com/paritytech/polkadot-sdk-docs/tree/main/tutorial.