Nextra maintainability over time and Next.js

[ovflowd]

Nextra is an awesome Framework on top of Next.js. It has fantastic features and is an awesome way for teams to quick-start documentation and blog websites.

Where is Next.js right now?

Having that said, over the passing months, Next.js itself is becoming more and more powerful out-of-the-box.

• Its Rust-based MDX implementation is incredible and would speed up the transpilation of MDX into JavaScript. It also uses way less memory and it is way faster.
• Dynamic Pages and Dynamic Static Pages are versatile
• TurboPack is a blazingly-fast bundler and would increase our build speeds incredibly and use way less memory.
• Next.js App Router has many new features and for our use-case would work the same way.

All these reasons and more, incline me that Next.js itself (out of the box) has enough features that overcome the need for Nextra.

What's the work being done for easing the transition?

There's currently a Pull Request open that experiments with ISR (Incremental Static Rendering) of missing translated pages from localised directories.

This is the first step to ensure that this idea works.

Currently, all the static pages live within Next.js's pages directory, making themselves individual pages. Since Next.js does not allow actual routes (file system) to coexist with matching dynamic pages, here's the gist:

• The current "legacy" static pages, the MDX-only pages, would live under another folder, say for example, ".pages"
  • Since these are actual pages, with Metadata (Frontmatter) and Layouts, and are MDX, this means that they're not just the "content" of the pages, but the "pages" themselves.
    • For now and until we transition to the pages from the nodejs.dev website, we could keep using the current way we define "layouts".
    • In the future, the new Layouts (based by route) would use the Route Layout feature
• The Base Layout would use the new Root Layouts Feature from Next.js
• The content within the app router folder, would only consist of the dynamic page routes (ISR/ICR and SCR)

Example Structure of the Node.js Pages & App Router

The snippet below showcases how the structure of the website with Next.js Route-based Layouts would work.

Note.: Route-based layouts would only be adopted on the new nodejs.dev migrated pages, as there are fewer layouts, and the remaining components will be part of the page itself (MDX) instead of having a dedicated layout with extra components.

.
├── .app/
│   ├── en/
│   │   ├── about/
│   │   │   ├── governance.md
│   │   │   └── index.md
│   │   ├── download/
│   │   │   ├── index.md
│   │   │   └── current.md
│   │   ├── ...
│   │   └── index.md
│   ├── es/
│   │   ├── about/
│   │   │   ├── governance.md
│   │   │   └── index.md
│   │   ├── ...
│   │   └── index.md
│   └── ...
└── app/
    ├── layout.tsx
    ├── 404.tsx
    └── [locale]/
        ├── [...pathname].tsx // default dynamic route for all pages
        ├── [blog]/
        │   ├── layout.tsx // layout for `blog-index` and categories
        │   ├── [category].tsx // dynamic route for all blog categories
        │   └── [category]/
        │       ├── layout.tsx // layout for all blog posts
        │       └── [...slug].tsx // dynamic route for all blog pages
        ├── [about]/
        │   └── layout.tsx
        └── [docs]/
            └── layout.tsx

Why's that, why moving away from Nextra? And why we adopted Nextra to begin with?

When we decided back in late 2022 to move the Node.js Website into Next.js, we had to make a transitionary step that would allow us to with the smallest amount of changes have the largest amount of feature-set and shared logic from the legacy (previous) Node.js Website.

Nextra was an almost out-of-the-box solution with the way how it worked. In the beginning we also heavily used Nextra's passing of getStaticProps (injection) to every Markdown page as a benefit to allow us to inject the Node release data and i18n data directly on each page's context. This was a stopgap measure done as we needed a quick and easy solution.

Recent PRs authored by myself and @HinataKah0 and the contribution of many others, allowed us to remove these "injected" contexts for i18n and Node Release data with a much better approach.

Improvements can still be done, on the aspect of loading Node.js Release information, but we believe that Client-Side rendering of this information for now is good enough. To be fair, we're still studying ways on simplifying this. One approach, given the size of the Node.js Release data, is to make it fully static and a global context available to the App-itself by a React Provider

We got it, but what are the issues of Nextra right now and maybe a little more context?

When we decided to move to Next.js, as I've mentioned, Nextra fulfilled all the checkboxes of an easy and quick transition. Back in the time, Vercel's VP of Engineering and Nextra's maintainers promised that Nextra was fully maintained. Sadly, with the passing of months and with the confirmation of the maintainer of Nextra themselves that Nextra is not a priority anymore, it worries me more and more as Nextra already has bugs and incompatibilities with the newest Next.js features.

It is also important to mention that we do not use Nextra's Flexsearch or any of the other built-in functionalities. And right now Nextra also has some cache bugs with Next.js and does not support the latest features, such as the App directory. We also want to adopt OramaSearch (cc @ShogunPanda) and getting rid of Nextra here will be a good idea.

How will this migration work and what are the caveats?

Adopting the app directory and moving the pages to .pages is actually simpler than you might imagine. Since we're keeping the current layouts and the way how they're loaded still on the Frontmatter until we start migrating nodejs.dev pages, not much needs to be changed.

The work done on this PR introduces next.dynamic.mjs and a set of helpers that are responsible for deterministically loading the MDX-content and providing the pages is the foundational step.

All the remaining work can be summarised on one PR:

1. Moves the current pages to .pages
2. Creates the new Dynamic Routes as explained above (In this case way fewer dynamic routes are needed)
3. A few modifications need to be done on the next.dynamic.mjs logic
4. The current theme.tsx and theme.dynamic.tsx will become one
5. All Nextra's dependencies will be removed

Some important Caveats

• Full static generation will still be respected and work as expected
• For Vercel and non-static builds all the pages except 404.mdx are fully dynamic and will be generated as needed. Meaning that the builds will be incredibly smaller
• Builds done on GitHub CI will be switched for the full-static ones, as this allows us to detect errors, as initial builds will not build all pages.
• With the incremental transition to the new Route Layouts and Route Templates, the build time will decrease even more
• MDXRemote which is the core of transpiling MDX on the fly with Next.js still uses the regular mdxjs and not Next.js's Rust-based MDX, and we will in the future eventual research how to use Next.js's built-in MDX transpiler.
• Simply by not using Nextra we will already be able to use Turbopack. Note that Turbopack for now is only available on Next.js's Development server.
  • We might be able to adopt ESBuild or something similar here to speedup production builds (cc @Harkunwar)

Timeframe of this development

In the coming weeks, I'm going to start working on this once the initial next.dynamic.mjs PR got merged. Then I'm going to work on the PR that should hand off the usage of Nextra.

[ovflowd]

Note that this discussion is an idea on how we can future-proof the Node.js Website and take advantage of latest Next.js features. And remove a Framework (Nextra) that was super useful for us on the beginning of our transition, but that now can be safely removed.

It is also important to mention that I'm under the assumption that removing Nextra and doing the changes I'm talking about will work out of the box. No guarantees at this point.

[bmuenzenmeyer]

Read through this and am in full alignment. I was intially very impressed with the level of lift Nextra could provide during the transition. I've been observing development here and helping in small ways here and on Nextra. It's a great project - but fits a static or OotB niche that a site like nodejs.org may have simply outgrown. I don't yet know the extent to which that is true, and appreciate the approach of Next, nextra, and the threading of needles that needs to happen.

It is also important to mention that we do not use Nextra's Flexsearch or any of the other built-in functionalities

The more I learn about this website and its needs, the more things like this standout. Custom theme is another one, albeit perhaps smaller. My employer is starting to explore nextra, and its out of the box capabilities are a step-change over what we have. We need hundreds of well-created defaults serving static content. Nextra Nodejs.org needs one stellar experience that is far more dynamic. The ease at which you can break out to Nextjs from within Nextra perhaps made this entire discussion inevitable.

Again, I love the approach we made here to change as little as possible, evolve, and now re-evaluate.

When we decided to move to Next.js, as I've mentioned, Nextra fulfilled all the checkboxes of an easy and quick transition. Back in the time, Vercel's VP of Engineering and Nextra's maintainers promised that Nextra was fully maintained. Sadly, with the passing of months and with the confirmation of the maintainer of Nextra themselves that Nextra is not a priority anymore, it worries me more and more as Nextra already has bugs and incompatibilities with the newest Next.js features.

Do you have a source for this? I'd love to learn more.

[ovflowd]

Hey @bmuenzenmeyer some of the discussion was in-Slack and Twitter. Nothing is formally defined anywhere.

But yes, going out from Nextra to plain-Next.js will be extremely simple.

[ovflowd]

Hey, y'all I made a draft PR which already solves the aforementioned discussion. It's a draft, but code-wise it's complete.

I'm going through the code to spot minor typos, and issues and update documentation.

Please feel free to add comments/reviews here: https://github.com/ovflowd/nodejs.org/pull/2.

Note that only when #5426 gets merged the other PR can become viable for being merged.

[ovflowd]

I'd like to mark this discussion as closed as we've migrated away from Nextra.