Add GHA that will check links + Fix broken links

[spier]

📑 Summary

Adds a GitHub Action (GHA) that check links in documentation files using lychee.
The markdown files that are checked so far are: packages/mermaid/src/docs/**/*.md README.md README.zh-CN.md

This is based on a conversation from a previous PR:
#3760 (comment)

📏 Design Decisions

• I am suggesting to run this link check once per week only, rather than for every PR. The link check can be a bit brittle, so running them for each PR can create too many false positives (I made this experience on another project).
• There were some relative links like ../config/configuration that did not contain the .md extension. Therefore lychee reported these links as broken. I have added the .md extension for these cases
• The section about "Contributing to Documentation" has various issues, as it is still pointing to the wrong docs folder. As that is unrelated to checking broken links, I have mostly ignored these for now.

Open Questions

• Various documentation files link to /packages/mermaid/src/docs/config/setup/README, which does not exist any more. Where was that content moved? => answer: auto-generated via pnpm docs:code
• The file https://github.com/mermaid-js/mermaid/edit/develop/src/docs/_sidebar.md does not exist any more. Should we link to packages/mermaid/src/docs/.vitepress/config.ts instead?

📋 Tasks

Make sure you

• 📖 have read the contribution guidelines
• 💻 have added unit/e2e tests (if appropriate)
• 🔖 targeted develop branch

[spier]

@aloisklink I created this based on our conversation as a Draft PR, so that we can work out how to best get this ready for merging.

I will leave some more comments inline to explain why i did what.

I already fixed various links, so the new link checker only reports a few broken links now.
See https://github.com/mermaid-js/mermaid/actions/runs/3406168811

[aloisklink]

So far, progress looks good. Great work!

From a quick look, it seems like lychee has already caught quite a few broken links :)

---

I am suggesting to run this link check once per week only, rather than for every PR. The link check can be a bit brittle, so running them for each PR can create too many false positives (I made this experience on another project).

My gut feeling is to leave it running on each push/PR, and if does start causing false positives, we could then disable it and just keep it running on a schedule.

But let's see what other reviewers say once this is ready to review.

---

Various documentation files link to /packages/mermaid/src/docs/config/setup/README, which does not exist any more. Where was that content moved?

https://github.com/mermaid-js/mermaid/blob/develop/packages/mermaid/src/docs/intro/index.md
is probably the new closest thing to it.

(a processed version appears in https://github.com/mermaid-js/mermaid/blob/develop/docs/intro/index.md)

---

The file https://github.com/mermaid-js/mermaid/edit/develop/src/docs/_sidebar.md does not exist any more. Should we link to packages/mermaid/src/docs/.vitepress/config.ts instead?

Sounds good!

---

The section about "Contributing to Documentation" has various issues, as it is still pointing to the wrong docs folder. As that is unrelated to checking broken links, I have mostly ignored these for now.

Before we merge this, it's important that the CI passes (otherwise the link checking CI failing might hide other more important issues). It should be fine if you don't have time to update the "Contributing to Documentation" section, but in that case, we may want to either:

a) ignore that file in CI
b) leave this PR open until a maintainer has time to fix that bit

[spier]

Figured out why the link checker could not find /packages/mermaid/src/docs/config/setup/README.
The content of the setup folder is generated from code, see packages/mermaid/package.json:

"docs:code": "typedoc src/defaultConfig.ts src/config.ts src/mermaidAPI.ts && prettier --write ./src/docs/config/setup",

When running that it outputs:

Documentation generated at ./src/docs/config/setup
src/docs/config/setup/modules/config.md 75ms
src/docs/config/setup/modules/defaultConfig.md 92ms
src/docs/config/setup/modules/mermaidAPI.md 333ms
src/docs/config/setup/README.md 4ms

The folder src/docs/config/setup is ignored by version control. See packages/mermaid/.gitignore.

That means that the link checker cannot check these links locally, as they don't exist as static files in the file system but rather are generated during the build.

So an interim solution for the link checker would be to ignore links below /packages/mermaid/src/docs/config/setup.

However there is still another mystery:

Even after generating these extra documentation files using npm docs:code, the documentation files I see locally are not the same as what is deployed at https://mermaid-js.github.io/mermaid/#/./Setup. I have no idea where the content that I see online comes from. Also when you try the "Edit this Page" this leads to a 404. Very strange :)

[spier]

Hi @aloisklink, I cannot figure this out, so maybe you can help me:

I understood that I am not supposed to change files in the /docs folder.
However I keep keep getting this error in the lint (18.x) GHA:

Changed files were transformed in src/docs but do not match the files in ../../docs. Please run 'pnpm --filter mermaid run docs:build' after making changes to src/docs to update the ../../docs directory with the transformed files.

I tried that command but it didn't resolve the issue.

I went back and forth and tried various thinks but now I am officially out of ideas :)
Help would be much appreciated.

[aloisklink]

So an interim solution for the link checker would be to ignore links below /packages/mermaid/src/docs/config/setup.

Sounds good! We could also do a pnpm --filter mermaid run docs:build first in GitHub Actions, but it might be simpler to leave it as something for a future PR.

Even after generating these extra documentation files using npm docs:code, the documentation files I see locally are not the same as what is deployed at https://mermaid-js.github.io/mermaid/#/./Setup. I have no idea where the content that I see online comes from. Also when you try the "Edit this Page" this leads to a 404. Very strange :)

The old content was auto-generated from code comments by documentationjs, but that's unfortunately has some bugs (e.g. documentationjs/documentation#1553), and the package is in low maintenance so they might not be fixed.

The new auto-generation code should be better in the long-term, since it supports TypeScript types, but unfortunately, a bunch of the old comments aren't compatible with the new generator, so those docs are currently pretty empty until we update those comments.

I understood that I am not supposed to change files in the /docs folder.
However I keep keep getting this error in the lint (18.x) GHA:

Changed files were transformed in src/docs but do not match the files in ../../docs. Please run 'pnpm --filter mermaid run docs:build' after making changes to src/docs to update the ../../docs directory with the transformed files.

I tried that command but it didn't resolve the issue.

I just pulled down your branch and ran pnpm --filter mermaid run docs:build and this is what git diff 66c543cb8fd8bb52451ff96231976bf2949f7f22 looks like:

Maybe the pnpm --filter mermaid run docs:build script is somehow viewing the contents of your git repo to see whether the default origin is? You might have found a bug! That script was only added in #3678 (6 days ago), so thank you for being our beta tester!

As a temporary measure, you could try manually replacing all the https://github.com/spier/mermaid with https://github.com/mermaid-js/mermaid and see if that works. Otherwise, I'd be happy to just manually push to your branch once you think the PR is ready to go.

[spier]

@aloisklink thanks for the quick response.

I think I took this as fast as I could, as some of the remaining internals of how this all works are still unclear to me.
e.g. I don't understand how the full documentation gets published to https://mermaid-js.github.io/mermaid as it looks different from the documentation that I can generate for the project locally.

I'd be happy to just manually push to your branch once you think the PR is ready to go.

Yes, it would be great if you could do that. Thank you!

I have marked this PR as "ready for review" 🥳

[aloisklink]

Thanks for making this PR! Lots of broken links have been fixed, so you've done a great job, and it definitely feels like Lychee is a valuable addition!

I think I took this as fast as I could, as some of the remaining internals of how this all works are still unclear to me.
e.g. I don't understand how the full documentation gets published to https://mermaid-js.github.io/mermaid as it looks different from the documentation that I can generate for the project locally.

I believe that only the docs on the master branch are pushed to https://mermaid-js.github.io/mermaid

The docs were recently reworked in #3678, so I'm guessing they'll only be published with Mermaid v9.3.0

I'd be happy to just manually push to your branch once you think the PR is ready to go.

Yes, it would be great if you could do that. Thank you!

I have marked this PR as "ready for review" 🥳

Done! I think the Mermaid team may need to investigate whether we can make pnpm run docs:verify point links to mermaid-js/mermaid even from a GitHub fork.

[sidharthv96]

Done! I think the Mermaid team may need to investigate whether we can make pnpm run docs:verify point links to mermaid-js/mermaid even from a GitHub fork.

It should work now. I've fixed the typedoc config.

[sidharthv96]

LGTM!
Thank you @spier :)

Just a small query.