add checklink support

[coke]

When building the site to run locally, we need some support for running a link checker. Can discuss if it needs to be a blocker or just informational, but historically there have been many issues on the old site build from raku/doc.

[finanalyst]

@coke BIG SMILE!!!! You said that the error report should be removed from production. Since I broke it (see below), I removed it altogether. There are some considerations for discussion.

The function of my link-error-test plugin. It used to check to see whether (a) external links worked, (b) internal links worked (c) local links work.

• An external link is one to a schema such as https://.

  • The big problem with this is that testing of external links has a very high time cost. There are some 250 (I think) external links, and it can take tens of seconds to get a HTML response from a link.

• A local link is to a document in the Collection eg /type/Cool#say where Type/Cool.pod6 is a source file.

  • Since the Collection build process gathers all the information for all the possible anchors (headers or X<> marked entries) in all of the rendered sources, it becomes possible to check whether a specific link eg L<Say| type/Cool#say> in one file does have an anchor in another file.
  • This can be done quite fast in the build process.
  • Then a report can be generated.
  • My idea was for the community to be able to see these errors and so the community could fix them. Since the errors could occur due to a typo error in the POD6 file, it wouldn't be picked up until the whole build was complete.

• An internal link is one to a place inside the same document. These should not occur by definition. And even if they do, it is very difficult to find them as the browser will do a soft fail on them.

Status of link-error-test. I broke the plugin when I was fixing the link rendering. I could fix it now, if necessary.

[coke]

The raku/doc repo had (until very recently) util/test-links.sh which was a shell script using wget to run a similar check.

[finanalyst]

I'll work on the link-error-test plugin again.
We can decide what to do with the generated html page, whether to have a visible link to it, or for the page to be present for developers/maintainers who want to see it

[finanalyst]

I have fixed the link-error-test plugin.
It is picking up many content errors.
The plugin as committed disables remote link testing, viz. links with schemas http/https
Remote testing takes 5 min. This is almost certainly because I have a sub-optimal remote link checking routine. I'm positive tweaking it would decrease the test time.
The routine is sub test-remote in Website/plugins/link-error-test/let-callable.raku. It uses LibCurl::HTTP and I think there should be some form of http-header.

The error-report can be seen at More->Anomalous Links which invites community involvement to solve problems.

[coke]

While it's nice to have a report on these somewhere, I wonder if it would be better to move it earlier in the process and check them on the raku/doc side, so we can verify it upstream. (I realize this would involve different tooling probably)

[finanalyst]

@coke

I have gone through the external 404's and raised PRs on them in Raku/doc.

• eventually I think maybe we could pin an issue in Raku/doc with a link to error-report.html in the website for people to check.
• The error-report would be built every day, but not on each push (the reason being that it takes 2 s to get a response back on a bad link. So the whole remote test takes 5 min.
• the error-report would not be linked to from the web page.

[coke]

While it's nice to have a report on these somewhere, I wonder if it would be better to move it earlier in the process and check them on the raku/doc side, so we can verify it upstream. (I realize this would involve different tooling probably)

To this end, just added xt/link-checker.rakutest in the raku/doc repository and found several broken links that I was able to fix. This should help us keep better track of the local links going forward. (I can only check primaries and known secondaries (link top level page /language), but it did find several issues.

[finanalyst]

Hopefully, xt/link-checker will remove links broken by typos, leaving systematic errors / corner cases that can be resolved upon generation.