fix publish workflow #1078

i10416 · 2021-10-18T13:25:41Z

Add workflow to publish latest document microsite because https://typelevel.org/spire/ does not show latest document.

After this PR

the microsite for latest spire version will be published every time PR containing docs change are merge to main branch.
link to API documentation will be replaced with the link to latest javadoc page https://www.javadoc.io/doc/org.typelevel/spire_2.13/latest/spire/index.html

This is a preview hosted on my gh-page.
https://i10416.github.io/spire/

.github/workflows/publish_microsite.yml

armanbilge · 2021-10-18T17:28:15Z

Thanks btw! We have some big changes in progress in #1067, and there's also #1069 to fix. This will go nicely with them.

i10416 · 2021-10-19T03:21:46Z

@armanbilge I will take a look at #1069 and try to send another PR.

armanbilge · 2021-10-19T03:32:54Z

.github/workflows/publish_microsite.yml

+          echo "$HOME/vendor/bundle/bin" >> $GITHUB_PATH
+      - name: Build docs
+        run: |
+          sbt docs/makeMicrosite


Where does sbt come from? I think you usually need to install it with an action.

Instead of as a hand-written workflow, Do you think we can include this in the sbt build with sbt-gh-actions? Then most of the setup and compiling is done automatically. See e.g. https://github.com/typelevel/fs2/blob/main/build.sbt#L349

Thanks!

Where does sbt come from?

ubuntu-latest has sbt installed.

@armanbilge

~~I choose a hand-written workflow because I have one concern. Is it possible for sbt-gh-actions to define multiple workflow~~

~~It is faster to publish docs when we use multiple workflows because workflow runs in parallel but steps do not run in parallel.~~

Sorry, I missed the e.g. link.

@armanbilge

Do you think we can include this in the sbt build

Answer is "yes, we can" but do we really need that?

I think hand-written workflow is better when it comes to managing spire document because

current publish task has little to do with Scala(only mdoc task), it is done by jekyll and actions-gh-pages. Wrapping them in sbt task does not seem straightforward.

I took a look sbt-gh-actions and sbt-gh-actions focus mainly on making it easier to test, build and publish Scala sources over cross java/scala versions and cross target platforms(jvm/native/js) (though it is possible to add publish docs jobs).

In addition, if we include publish task in jobs, we have to introduce another action to detect file changes, which can be done by github workflow's path filter.

some libraries in typelevel repo like cats-effect uses hand-written workflow for publishing docs. https://github.com/typelevel/cats-effect/blob/docs/.github/workflows/microsite.yml

publish docs workflow has a lifecycle different from CI workflow. Ci should run every PR or commit in order to validate changes, but publish docs workflow should run when microsite actually changes. They are different concern.

separating document settings from build settings will keep build settings simple(and makes it easier to migrate to another document tools in the future.)

You say "setup and compiling is done automatically" but these features make little difference for publishing docs. We do not have to setup Java or Scala for publishing docs. We need setup jekyll, envs and paths even if we use sbt-gh-actions.

If we would publish docs for each spire versions over each Scala versions and each platforms it would be worth managing it in build.sbt, but current microsite is independent of Scala versions or platforms, just displaying information of the latest version, so the publish doc workflow is simple enough and will not be likely to swell further.

@i10416 thanks for the detailed research and write-up, this is great.

We do not have to setup Java or Scala for publishing docs.

Yes, this was my faulty assumption. Sorry about that.

publish docs workflow has a lifecycle different from CI workflow. Ci should run every PR or commit in order to validate changes, but publish docs workflow should run when microsite actually changes. They are different concern.

That's true, but we should build the docs as part of CI for PRs to make sure the PR didn't break the documentation somehow. E.g. maybe an example in the docs is using some code that changed, etc.

In addition, if we include publish task in jobs, we have to introduce another action to detect file changes, which can be done by github workflow's path filter.

IIUC publishing the docs also publishes the API docs: http://typelevel.org/spire/api/spire/index.html
If it doesn't, then it should. This can change even when there haven't been changes in the docs folder. So we should actually be careful to only publish the site when we release a new, stable version, I think, so that the docs don't get mismatched?

Ah, you mean you want to programatically garantee docs are successfully built using 'if ....' or 'needs build' ? If so, it would be better to include the doc job in build.sbt.

Sorry, I assumed that docs are supposed to be successfully built everytime PR is merged to main (or version tag is set) because current workflow contains build docs step and you can be sure docs are vaild.

So we should actually be careful to only publish the site when we release a new, stable version

Ok, workflow becomes simpler and there's no need to check doc file changes as publishing documents happens only when stable release, not when doc files changes.

@armanbilge

Yes, just for my own clarification the docs should be built as part of every PR, to avoid breaking something. And as you point out, they already are. But of course, they should not be published from the PR :)

Ok, workflow becomes simpler and there's no need to check doc file changes as publishing documents happens only when stable release, not when doc files changes.

The downside to this is that docs contributions/updates are not published until the next release ... but, it prevents the API docs from getting out of sync with the published artifact, which could be very confusing. I'm open to ideas to improve this.

One way would be to publish the API docs under a version directory (e.g., 0.17, 0.18, SNAPSHOT). So this way, even though updating the site would also update the API docs, it would publish under SNAPSHOT instead of overwriting the API docs for the current.

For clarification, I'm distinguishing between the site/main docs (which should ideally be updated as frequently as possible) and the API docs aka scaladocs, which should match the published artifact.

Thoughts? Thanks again for your help with this, much appreciated!

i10416 · 2021-10-19T03:34:52Z

made Issue #1080 about #1067

i10416 · 2021-10-20T08:00:03Z

One way would be to publish the API docs under a version directory (e.g., 0.17, 0.18, SNAPSHOT).

Yes, it is good idea but unfortunately sbt microsite supports multi version publishing only with light theme. I think supporting multi-versons while keeping the site apperance needs a lot to do.

I think migrating to laika (as http4s project is doing) may help, but it still needs amounts of work too.

i10416 · 2021-10-20T08:00:18Z

@armanbilge

i10416 · 2021-10-20T08:06:04Z

So, for now I will move workflow in sbt task and try to support multi versioned docsite in another PR later.

armanbilge · 2021-10-20T08:07:16Z

Oh, I had a better idea for this. Let's keep your strategy of publishing the site every time the docs folder changes, and instead of hosting the API docs ourselves we can just point to https://www.javadoc.io/doc/org.typelevel/spire_2.13/latest/spire/index.html

That way, it is a stable link to API docs matching the latest release, and users can even change the version there. What do you think?

i10416 · 2021-10-20T08:54:29Z

That is quite simpler!
So, we keep separated workflows and just change API link? or even if we point to javadoc.io, would it be better to integrate workflow into sbt task?

armanbilge · 2021-10-20T08:55:36Z

I'm happy to keep your workflow as-is here, you made a good case for keeping it simple. So we just need to update the link :)

build.sbt

i10416 · 2021-10-22T14:21:07Z

@armanbilge ping
Fix what you pointed out.

armanbilge

Thanks for persevering on this! LGTM, will be a big improvement.

I'm going to ask @cquiroz to have a look as well.

armanbilge · 2021-10-22T14:50:20Z

build.sbt

-    mdocIn := (Compile / sourceDirectory).value / "mdoc"
+    mdocIn := (Compile / sourceDirectory).value / "mdoc",
+    mdocVariables := Map(
+      "VERSION" -> version.value


👍 this is great!

cquiroz

Looks good to me, thanks for making this work

cquiroz · 2021-10-22T14:55:24Z

docs/src/main/mdoc/index.md

@@ -65,7 +65,7 @@ JVM and JS platforms.
 To get started with SBT, simply add the following to your `build.sbt` file:

 ```
-libraryDependencies += "org.typelevel" %% "spire" % "0.17.0"
+libraryDependencies += "org.typelevel" %% "spire" % "@VERSION@"


armanbilge · 2021-10-22T15:28:06Z

Darn, the publish job failed:
https://github.com/typelevel/spire/runs/3977435250

It's not your fault here, it seems the docs are broken. I thought we are checking them in CI for each PR, but I guess we're not.

#1092

fix publish workflow

ba24e2e

armanbilge requested changes Oct 18, 2021

View reviewed changes

.github/workflows/publish_microsite.yml Outdated Show resolved Hide resolved

armanbilge added this to the v0.18.0 milestone Oct 18, 2021

fix condition

b849bf6

armanbilge reviewed Oct 19, 2021

View reviewed changes

move api documentation url

ffa5234

armanbilge reviewed Oct 20, 2021

View reviewed changes

build.sbt Outdated Show resolved Hide resolved

i10416 added 5 commits October 20, 2021 21:06

small fix

3e14896

use mdoc variable

ec48161

replace travis badge with github actions one

bd4abe9

resolve conflict

7b9e502

small fix

0ad028b

armanbilge approved these changes Oct 22, 2021

View reviewed changes

armanbilge requested a review from cquiroz October 22, 2021 14:52

cquiroz approved these changes Oct 22, 2021

View reviewed changes

armanbilge merged commit 1d691c4 into typelevel:main Oct 22, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix publish workflow #1078

fix publish workflow #1078

i10416 commented Oct 18, 2021 •

edited

Loading

armanbilge commented Oct 18, 2021 •

edited

Loading

i10416 commented Oct 19, 2021

armanbilge Oct 19, 2021

i10416 Oct 19, 2021

i10416 Oct 19, 2021 •

edited

Loading

i10416 Oct 19, 2021 •

edited

Loading

i10416 Oct 19, 2021 •

edited

Loading

armanbilge Oct 19, 2021 •

edited

Loading

i10416 Oct 20, 2021

i10416 Oct 20, 2021

armanbilge Oct 20, 2021

i10416 commented Oct 19, 2021

i10416 commented Oct 20, 2021

i10416 commented Oct 20, 2021

i10416 commented Oct 20, 2021

armanbilge commented Oct 20, 2021

i10416 commented Oct 20, 2021

armanbilge commented Oct 20, 2021

i10416 commented Oct 22, 2021

armanbilge left a comment

armanbilge Oct 22, 2021

cquiroz left a comment

cquiroz Oct 22, 2021

armanbilge commented Oct 22, 2021 •

edited

Loading

fix publish workflow #1078

fix publish workflow #1078

Conversation

i10416 commented Oct 18, 2021 • edited Loading

armanbilge commented Oct 18, 2021 • edited Loading

i10416 commented Oct 19, 2021

armanbilge Oct 19, 2021

Choose a reason for hiding this comment

i10416 Oct 19, 2021

Choose a reason for hiding this comment

i10416 Oct 19, 2021 • edited Loading

Choose a reason for hiding this comment

i10416 Oct 19, 2021 • edited Loading

Choose a reason for hiding this comment

i10416 Oct 19, 2021 • edited Loading

Choose a reason for hiding this comment

armanbilge Oct 19, 2021 • edited Loading

Choose a reason for hiding this comment

i10416 Oct 20, 2021

Choose a reason for hiding this comment

i10416 Oct 20, 2021

Choose a reason for hiding this comment

armanbilge Oct 20, 2021

Choose a reason for hiding this comment

i10416 commented Oct 19, 2021

i10416 commented Oct 20, 2021

i10416 commented Oct 20, 2021

i10416 commented Oct 20, 2021

armanbilge commented Oct 20, 2021

i10416 commented Oct 20, 2021

armanbilge commented Oct 20, 2021

i10416 commented Oct 22, 2021

armanbilge left a comment

Choose a reason for hiding this comment

armanbilge Oct 22, 2021

Choose a reason for hiding this comment

cquiroz left a comment

Choose a reason for hiding this comment

cquiroz Oct 22, 2021

Choose a reason for hiding this comment

armanbilge commented Oct 22, 2021 • edited Loading

i10416 commented Oct 18, 2021 •

edited

Loading

armanbilge commented Oct 18, 2021 •

edited

Loading

i10416 Oct 19, 2021 •

edited

Loading

i10416 Oct 19, 2021 •

edited

Loading

i10416 Oct 19, 2021 •

edited

Loading

armanbilge Oct 19, 2021 •

edited

Loading

armanbilge commented Oct 22, 2021 •

edited

Loading