Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: upgrade to MDX v2 #8288

Merged
merged 176 commits into from
Apr 21, 2023
Merged

feat: upgrade to MDX v2 #8288

merged 176 commits into from
Apr 21, 2023

Conversation

slorber
Copy link
Collaborator

@slorber slorber commented Nov 3, 2022

Motivation

Docusaurus v3 will upgrade MDX to v2

It is much better to work with than v1 for many reasons explained here: https://mdxjs.com/blog/v2/

MDX v2 also exposes a better infrastructure for Docusaurus to build new useful features, and optimize performances.

Fix #4029

Also makes it possible to support a CommonMark mode (opt-in), requested in #3018, but by default we still use MDX (even on .md files, see also #9097)

Need support to upgrade to MDX v2? See MDX v2 Upgrade Support

Other notable changes:


Breaking changes

Breaking changes coming from MDX v2 itself, notably:

  • Need to update your custom Remark/rehype plugins for v2
  • Lowercase JSX tags are not substituted by MDXProvider components anymore: tags like <pre> or <code> will render as is
  • Html comments are not supported (➡️ compat option provided)
  • Syntax for explicit heading ids is invalid (➡️ compat option provided)
  • Syntax admonition title is different (➡️ compat option provided)
  • Everything else documented by MDX v2 itself (v2 migration guide)
  • Need to upgrade Remark-math + Rehype-katex?

Breaking changes coming from Docusaurus public APIs:

  • .md and .mdx can be handled differently if you opt-in for CommonMark (
  • blog truncate marker should now use {/* truncate */} for .mdx files (➡️ compat option provided)
  • docs.admonitions.tag option is removed, we can now only use ::: (markdown directive syntax)
  • docs.admonitions.extendDefaults option is now true by default (merge default+user-provided admonition keywords instead of replacing them)
  • the mdx-code-block workaround for Crowdin is implemented in a very different way. It still works in most cases but we recommend to not use this anymore and put complex React components outside of MDX files.

Notable private API changes:


Compatibility options

Site config

We have a few options to keep Docusaurus v3 more retro-compatible with v2 features, that are for now turned on by default.

The goal is to give you the opportunity to progressively adopt a stricter MDX syntax.

This is important to aim for MDX compatibility, so that other tools understand your files: GitHub, VSCode, Prettier, ESLint, Crowdin... You will have to progressively turn these flags off to do so.

const config = {
  markdown: {
    // By default we compile all files as MDX (allow JSX), but you can opt-in for CommonMark if you want
    // See https://github.com/facebook/docusaurus/pull/9097
    format: 'mdx', 

    mdx1Compat: {
      // Allow html comments in MDX files like blog `<!-- truncate -->` markers
      comments: true,
      // Allow former admonition title syntax :::note Title instead of new syntax :::note[Title]
      admonitions: true,
      // Allow usage of unescaped {#headingId} syntax instead of \{#headingId}
      headingIds: true
    },

    // Gives you opportunity to apply your own string processing before the MDX compilation
    // Usage is NOT recommended, use this as a last resort escape hatch or temporarily)
    preprocessor: ({ filePath, fileContent }) => {
      return fileContent;
    }
  }
};

MDX vs CommonMark

By default, Docusaurus v3 will still compile all files as MDX (like Docusaurus v2 did), but it is possible to opt-in for CommonMark parser (does not allow usage of JSX).

You can do so:

  • with site settings: siteConfig.markdown.format: 'detect' (will use CommonMark for .md files and MDX for .mdx files)
  • with front matter: format: 'md'

Note: some Docusaurus features currently don't work yet in CommonMark, make sure to test this issue before adopting this new mode: #9092


Follow-up

Things to handle later, after this PR is merged:


Tests

We have many unit tests to ensure the behavior is similar to what it used to be.

We also started using visual regressions tests, comparing visually each page from our sitemap to detect potential regressions. The setup is quite hacky (using an external repo for now) but you can take a look here:

Some review links:


Other Relevant links:

@slorber slorber added the pr: new feature This PR adds a new API or behavior. label Nov 3, 2022
@facebook-github-bot facebook-github-bot added the CLA Signed Signed Facebook CLA label Nov 3, 2022
@netlify
Copy link

netlify bot commented Nov 3, 2022

[V2]

Name Link
🔨 Latest commit 5ab8e86
🔍 Latest deploy log https://app.netlify.com/sites/docusaurus-2/deploys/6442c74c8c23ef0008b98b54
😎 Deploy Preview https://deploy-preview-8288--docusaurus-2.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@github-actions
Copy link

github-actions bot commented Nov 3, 2022

⚡️ Lighthouse report for the deploy preview of this PR

URL Performance Accessibility Best Practices SEO PWA Report
/ 🟠 82 🟢 97 🟢 92 🟢 100 🟢 90 Report
/docs/installation 🟠 76 🟢 100 🟢 92 🟢 100 🟢 90 Report

@github-actions
Copy link

github-actions bot commented Nov 3, 2022

Size Change: +148 B (0%)

Total Size: 1.01 MB

Filename Size Change
website/build/assets/css/styles.********.css 113 kB +36 B (0%)
website/build/assets/js/main.********.js 752 kB +112 B (0%)
ℹ️ View Unchanged
Filename Size
website/.docusaurus/globalData.json 101 kB
website/build/index.html 41.3 kB

compressed-size-action

@slorber
Copy link
Collaborator Author

slorber commented Nov 4, 2022

Note the docs says:

Unescaped left angle bracket / less than (<) and left curly brace ({) have to be escaped: < or { (or use expressions: {'<'}, {'{'})

https://mdxjs.com/docs/what-is-mdx/

Problem - Heading anchor syntax {#anchor-id}

MDX 2 interprets { as the start of an expression and it should be escaped to \{ in our case.

https://mdxjs.com/docs/troubleshooting-mdx/#could-not-parse-importexports-with-acorn-error

Solution: escape the md files with regexp for retrocompatibility and convenience (83064b2)

It's not possible to implement this as a remark plugin (too late in the process)

We probably don't want to ask users to do such escaping, nor change our existing syntax.

Note the React beta site have the same feature with a different syntax:
https://github.com/reactjs/reactjs.org/blob/main/beta/src/content/index.md

## What is this site? {/*what-is-this-site*/}

Problem - <!-- HTML comments

The < lead to a similar acorn error and need to be escaped.

Note: it only happens with format: "mdx", not 'md', and we may allow users to use the md format (through frontmatter) for improved compatibility and easy migration to Docusaurus.

Solution? A quick prepressing that escapes all HTML comments


Problem - // js comments in JS

This code in MDX doesn't work anymore due to the JS comment:

import LiteYouTubeEmbed from 'react-lite-youtube-embed';

<div className="video-container">
  <LiteYouTubeEmbed
    // cSpell:ignore Yhyx Sksg
    id="Yhyx7otSksg"
    params="autoplay=1&autohide=1&showinfo=0&rel=0"
    title="Docusaurus: Documentation Made Easy"
    poster="maxresdefault"
    webp
  />
</div>

Unexpected character / (U+002F) after self-closing slash, expected > to end the tag (note: JS comments in JSX tags are not supported in MDX

Solution? Remove the comment?


Problem - Math equations

$$
I = \int_0^{2\pi} \sin(x)\,dx
$$

Could not parse expression with acorn: Expecting Unicode escape sequence \uXXXX

Solution: again, escape \{ ?

$$
I = \int_0^\{2\pi} \sin(x)\,dx
$$

@slorber
Copy link
Collaborator Author

slorber commented Jun 9, 2023

Struggling to upgrade your site to MDX v2?

Here's a discussion to ask for support: #9053

@johnnyreilly
Copy link
Contributor

johnnyreilly commented Jun 21, 2023

Just linking in a related issue here: #9081 (comment)

I'd love to see a guide on how to migrate remark / rehype plugins to Docusaurus v3 / MDX v2

I documented this a bit better already for v3, but due to Crowdin issues our website refuse to deploy for a long time... 😓

It sounds like the docs may exist - but not yet be visible!

@fflaten fflaten mentioned this pull request Jul 13, 2023
This was referenced Oct 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed Signed Facebook CLA pr: breaking change Existing sites may not build successfully in the new version. Description contains more details. pr: new feature This PR adds a new API or behavior.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Migrate to MDX 2.0
7 participants