MDX loader: linkify should process the md AST instead of the md string

[slorber]

Have you read the Contributing Guidelines on issues?

• I have read the Contributing Guidelines on issues.

Motivation

Our current MDX loader code looks like this:

export default function markdownLoader(
  this: LoaderContext<DocsMarkdownOption>,
  source: string,
): void {
  const fileString = source;
  const callback = this.async();
  const options = this.getOptions();
  return callback(null, linkify(fileString, this.resourcePath, options));
}

The linkify process that turns [link](./xyz.md) to [link](/baseUrl/xyz-pathname) is based on string manipulations and regexes.

As a result, linkify has many edge case bugs related to the unability to properly resolve markdown relative file path links such as [](./relative-path.md)

We should instead do this with a Remark plugin after MDX has parsed the string.

See also:

• fix(utils): make Markdown link replacement much more rigorous #8927 (comment)
• Plugin is unable process resource links (Docusaurus process them anyway) #9041 (comment)
• fix(utils): better handling of code blocks in link replacement #9046
• Docusaurus v3 markdown links don't resolve correctly from index pages during build step when trailingSlash is set to false #9720 (comment)

Self-service

• I'd be willing to do some initial work on this proposal myself.

[Josh-Cena]

See previous attempt: #6261

[slorber]

Another case it would fix is a bug when using relative md links on markdown images:

[text](./installation.mdx)

[![alt](/img/slash-introducing.svg)](./installation.mdx)

Only the first link resolves.

But reported here: #9720 (comment), that should also be solved by this proposed solution.

[slorber]

Another case reported where markdown links resolution fails due to usage of inline code blocks with triple backticks (TIL this exists and is supported by MDX 😅). #9876

```something```

[test](test.md)

[slorber]

Another use-case reported is when the link is multi-line:

<!-- This works -->
A [link to another page](other-page.md)

<!-- This doesn't work -->
A [link to
another page](other-page.md)

#9636
#10067

[slorber]

Another use-case reported is when the link contains certain combinations of symbols:

[a](./migration/migrate-6.md#hooks) ([b](./migration/migrate-6.md#hooks))

[`Type1`](some_doc.md#type1)\<[`Type2`](some_doc.md#type2)\>

#9518
#9553
#10072