Versioned documentation

[rossabaker]

It would be nice to have a turnkey solution for projects that wish to support versioned documentation.

Requirements

Table stakes

• Each maintained branch gets its own set of docs
• Can publish from maintenance branches without breaking new things
• Can publish from main without breaking old things

Nice to have

• A new version updates the version selector in old versions
• EOL versions can easily point to a maintained version
• Version selector jumps to corresponding page, or a fallback if not relevant. (Laika does this.)
• If a PR will break the site generator, it fails. (A risk if mdoc compilation is separate from site generation.)

Extra credit

• /stable/* redirects to the latest stable version, like Read The Docs. This was possible on Netlify, which is an administrative headache. This is impossible on gh-pages.
• Works across repos for polyrepo projects (subdomains are a viable alternative)
• EOL versions are not indexed by Google. (Maybe. This would improve SEO within the site, but could hurt it to the site.)

Prior art

http4s

The /docs directory on live branches publishes to a subdirectory of the gh-pages branch. The /website directory is published off main and contains the project information that pertains to all branches.

Because each branch publishes independently, there is no way to update the theme across the board, nor a way to automatically update the version selector on old branches when release statuses change. Care must be taken to not touch anything outside that subdirectory.

cats-effect

I don't know how it works. Submodules and stuff. Generates crufty PRs, but the end user experience is best-in-Typelevel.

Monix

Has also been doing it for years in an artisanal fashion.

Other Scala projects that do a nice job and are worth studying

• Akka
• Play

Other tooling

Laika has multiple version support, and sbt-typelevel already integrates Laika

Other ideas

All of these are based on the idea that the site generator runs once, instead of independent sites per version and for the common area. This raises a basic tradeoff:

• If the site generator runs in the CI of a branch, we have a race condition
• If the site generator is triggered by a workflow after a branch is merged, that branch can break the site

Publish mdocs per branch, separate job generates the site

Requires careful git exclusions. Is the most economical with CI time.

mdoc all the branches and generate the site in one job

Awfully slow for large projects.

Parallelize building all branches, upload artifacts, generate in separate job

It's a heavy build, but it avoids git exclusions

[eed3si9n]

/stable/* redirects to the latest stable version

This is impossible on gh-pages.

GitHub Pages seems to forbid symlinks. My workaround for sbt website was to write this script (https://github.com/sbt/sbt.github.com/blob/master/script/redirect.scala) that generates html files.

https://www.scala-sbt.org/release/docs/Using-Sonatype.html for example would redirect to https://www.scala-sbt.org/1.x/docs/Using-Sonatype.html.

[eed3si9n]

Can publish from maintenance branches without breaking new things

This can be done by tweaking GhPagesKeys.synchLocal - https://github.com/sbt/website/blob/c1b34210ac22f8fedc929612835e41c20f291bea/project/Docs.scala#L162-L178

[eed3si9n]

EOL versions can easily point to a maintained version

For sbt, there's a one-liner js file that lists available documentation sites (https://github.com/sbt/sbt.github.com/blob/master/assets/versions.js):

var availableDocumentationVersions = ['1.x', '0.13', '0.12.4', '0.7.7']

and warnOldDocs.js, which we borrowed from Akka (https://github.com/sbt/sbt.github.com/blob/master/assets/warnOldDocs.js), which displays "This version of sbt (sbt 0.13) is outdated and not supported! Click here to go to the same page on the 1.x version of the docs."
https://www.scala-sbt.org/0.13/docs/Using-Sonatype.html

[armanbilge]

Thanks for chiming in Eugene :)

Politely pinging @jenshalm; if you have a chance, I would appreciate your insight on how we might implement some of these ideas with Laika. Thanks!

[eed3si9n]

EOL versions are not indexed by Google

This might be related to release cadence. For sbt, we have a documentation for "1.x", so we keep appending new features we add in the minor version 1.1.x, 1.2.x, etc onto the same site, and disallow indexing of old documentations or version-specific Scaladoc API - https://github.com/sbt/sbt.github.com/blob/master/robots.txt
Since 1.0.0 came out in 2017, Google has had 5 years to use the pages.
In addition, sitemap is generated https://github.com/sbt/website/blob/develop/project/SiteMap.scala, which hopefully helps SEO.

[jenshalm]

@armanbilge As you know, many of the listed versioning features come out of the box with Laika, maybe apart from linking between versions which is not straightforward (would only work in a non-validated fashion, as the version indexing only scans and saves the files that exist per version, not the targets within that file). So unless you have more concrete questions, I can only think of a few high-level recommendations at the moment:

• The setup that you have in http4s with two documentation projects is something I would not recommend as the base case for simple requirements. Since a single Laika project can mix unversioned and versioned documents via configuration, a split into multiple projects is only needed when you want to modularize the site for multiple sub-modules with their own versioning.

• Maintenance branches can set the renderUnversioned flag to false to ensure that the main site is only published from the main branch. This project could probably automate the setting of that value. (You don't use that flag in http4s because you have two separate documentation projects).

• When it comes to integrating Laika in another sbt plugin in general, I'd recommend not to do it like the sbt-site plugin. They re-bind keys in a way that some Laika keys are shadowed causing a loss of functionality (e.g. you cannot freely compose inputs). I'd always recommend against using that integration as users lose more functionality than they gain (apart from the fact they integrate an ancient version).

• One thing you could add is convenient defaults for integration with the ghpages plugin, which Laika currently does not offer (I did not want to introduce a hard dependency for the minor benefit of users saving a single line of plumbing). But this project is a big collection of plugins anyway, so providing the bindings out of the box would be sensible.

[armanbilge]

@jenshalm thanks for your advice, and all the work you did on http4s :)

I guess mostly I'm confused about the versionInfo.json:

1. How does it interact with the Helium.defaults.site.versions(versions) config? Do all branches need to keep this config up-to-date?
2. How is versionInfo.json maintained/updated? And on which branch(es)?
3. If each branch is published independently, what's the best way to make sure they don't overwrite each other? I suppose this is what renderUnversioned := false is for?

Btw, I did create a Laika integration in sbt-typelevel for unversioned sites, and a few Typelevel projects have already adopted it. Would you mind having a look in case I did it wrong?

sbt-typelevel/site/src/main/scala/org/typelevel/sbt/TypelevelSitePlugin.scala

Line 32 in c7e1988

object TypelevelSitePlugin extends AutoPlugin {

http://typelevel.org/sbt-typelevel/
http://typelevel.org/case-insensitive/
http://typelevel.org/cats-collections
http://typelevel.org/cats-time/
http://typelevel.org/frameless/
http://typelevel.org/log4cats/
http://typelevel.org/monoids/
http://typelevel.org/mouse/
http://typelevel.org/spire/
http://typelevel.org/vault/

[jenshalm]

@armanbilge Happy to have a look at the plugin over the weekend. Regarding your questions:

1. This only needs to be complete and accurate on the main branch, as only that branch should publish the version config that all branches read from. In maintenance branches, this only needs to contain the correct configuration for the one version that particular branch represents. The DX could probably be improved, as it's not really clear from the config API.

2. In case of using the renderUnversioned flag in the intended way, versionInfo.json will also only ever be published by the main branch (as it is an unversioned document itself). In maintenance branches you may choose to update this file, if you want more accurate local testing, but it is not necessary.

3. Yes, as you suspected, the use of the renderUnversioned flag should prevent any conflicts. I believe this is not a widely adopted functionality yet, so there might be unexpected edge cases, but nothing that would not be easy to fix in Laika.

[armanbilge]

@jenshalm thank you very much, that explanation was very helpful and I think I understand how it works now :)

Going through Ross's wishlist it seems straightforward to support:

• Each maintained branch gets its own set of docs
• Can publish from maintenance branches without breaking new things
• Can publish from main without breaking old things
• A new version updates the version selector in old versions
• EOL versions can easily point to a maintained version
• Version selector jumps to corresponding page, or a fallback if not relevant. (Laika does this.)
• If a PR will break the site generator, it fails. (A risk if mdoc compilation is separate from site generation.)
• /stable/* redirects to the latest stable version, like Read The Docs. This was possible on Netlify, which is an administrative headache. This is impossible on gh-pages.
• Works across repos for polyrepo projects (subdomains are a viable alternative)
• EOL versions are not indexed by Google. (Maybe. This would improve SEO within the site, but could hurt it to the site.)