Improve migration cli

[slorber]

🚀 Feature

The new migration cli works fine most of the case but there are still some things we can easily improve.

We have e2e tests that seems to produce a buildable site with the command:

yarn docusaurus-migrate migrate ./website-1.x ./test-migrated

But it fails if we use this command:

yarn docusaurus-migrate migrate ./website-1.x ./test-migrated --mdx --page

Some pages and markdown files produce errors. I'd like to get this end2end test working at the end of this issue, including the pages. This won't guarantee that it will work on all d1 website but if it works for our d1 website at least it's a good thing.

Here are raw suggestions, and things I tried locally that could quite easily be automated.

---

Preserve git history

This has already been reported in the migration PR here: #3015

We should aim to preserve the docs git history. This is a real problem, that has notably be reported about the ReactNative website migration.

Edit: for RN website, we just kept the master docs in ./docs, so that they keep their history more easily. Maybe we should just recommend users to do the same? Preserving history of versioned docs seem less important and much harder to solve.

Pages

Page entry

I think we should transform the main page export from:

module.export = class Index {
  render() {
    // old code
  }
}

to

class Index {
  render() {
    // old code
  }
}

export default function MyPage(props) {
  const {siteConfig} = useDocusaurusContext();
  return (
    <Layout
      title="title for <filename>"
      description="description for <filname>">
      <Index {...props} config={siteConfig} />
    </Layout>
  );
}

Page custom fields

As we have a list of known customfields in the config, we can probably handle this in pages too:

siteConfig.users => siteConfig.customFields.users
siteConfig.unknownName => siteConfig.customFields.unknownName
siteConfig.renamedAttribute => siteConfig.newAttributeName

Page component libraries

We currently emulate the comp library with:

const CompLibrary = {
  Container: props => <div {...props}></div>,
  GridBlock: props => <div {...props}></div>,
  MarkdownBlock: props => <div {...props}></div>
};

This implementation is basic and will not work fine most of the time, particularly for GridBlock which accepts a contents array prop.

The old component library only contains 3 components. We could copy those components in ./src/components/docusaurusV1/, tweak them a bit (apply some existing Infima styles and inline styles) and they'd be usable in pages.

Page translate

const translate = props => <div {...props}></div>;

This won't work, we need all comps being uppercase to work, + I think translations should use spans (not sure, to check what's best)

const Translate = props => <span {...props}></span>;

Page placeholder components

const Showcase = require(`${process.cwd()}/core/Showcase.js`);

gets transformed as:

const Showcase = (props) => <div {...props}></div>;

That's nice but user may not notice that some comp is missing in the page, particularly if children is an object or undefined. I'd suggest to have some kind of visible placeholder component that we put in the same folder.

You could do something like:

const Showcase = (props) => <CompPlaceholder originalCompName="Showcase" {...props}></div>;

export default function CompPlaceholder({ originalCompName, ...props }) {
  if (
    React.isValidElement(props.children) ||
    props.children instanceof String
  ) {
    return (
      <div>
        <div>{originalCompName}</div>
        <div>{props.children}</div>
      </div>
    );
  } else {
    return (
      <div>
        <div>{originalCompName}</div>
        <div>{JSON.stringify(props, null, 2)}</div>
      </div>
    );
  }
}

Page js transforms

There are existing jscodeshift transforms that we can run on the pages to modernize the js syntax to modern the standarts of a modern React codebase. For example transforming classes with only render fn to function components etc.

Page expected result

By applying just those simple transforms (that are probably doable with just string replace and regexps), I was able to get the following v1 site homepage migrated.

https://gist.github.com/slorber/96b4b2886f88e4a9d54aba20e658f5a7

It's not pretty (I'm sure you can do better :p), but at least it renders the existing components, and users can iterate on this to fix the remaining css issues:

---

MDX issues

It seems that the "markdown features" doc fails to convert properly. This is required to make the v1 site work in the CI.

There's a script in it that seems to make it fail: <script src="https://gist.github.com/yangshun/d36d04f383c40beb3f31dd2a16666f6c.js" />

Also found some very suspicious diff. For those reasons I'm not very confortable recommending to the users migrating the mdx files automatically for now 😅

---

Formatting

We should run prettier on the generated files (config, sidebars, json, pages, docs...) to ensure decent formatting of the output.

There are formatting issues in the produced files, for example:

Some MDX formatting, like the list item indentation, can be fixed by Prettier.

---

Recommendations

How should the user run this migration process? What's the recommended/safest workflow?

What I did:

• create a new git branch
• run: yarn docusaurus-migrate migrate ./website-1.x ./test-migrated
• get the site partially running (with empty homepage)
• commit
• run: yarn docusaurus-migrate migrate ./website-1.x ./test-migrated --mdx --page
• some migrated pages and mdx docs might not work: inspect the migration diff and try to fix it

This process permitted me to see the diff on the docs, and to see that the mdx migration actually didn't work so well. We don't see it at first without inspecting the diff, but many code blocks of v1 have been removed in this step.

---

This is just improvement ideas.

Do whatever you can in this list, but if we can get the v1 site pages migrated automatically, and wire that to the CI, that would be cool.

As it's for a migration, I think if we find quick and dirty solutions that work most of the time, it's probably good enough.

It's probably not worth investing months of work in this cli, so if you do some migrations with regexps and string replaces instead of a more robust jscodeshift infra, that's fine ;)

[slorber]

Note, it's worth checking review comments on the initial PR for additional things to improve: #3015

[slorber]

This is a migration cli for v1=>v2

We are now close to v3 and nothing in this issue is now worth implementing