[feature] Configurable Pages Directory with Multiple Support

[danielkov]

Describe the feature you'd like to request

At my company, we'd like the following feature to be added to next:

Add configurable pages directory option, with multi-directory support.

Our use-case is the following:

• A monorepo supports multiple projects with a shared codebase
• There is a shared directory where the default pages are: (src/pages)
• Each site lives within sites directory
• Each site can have a pages directory, where additional pages can be added and shared pages can be overridden
• When starting next for development, an environment variable has to be defined for the current site, which is then accessed in next.config.js where it's added the a new config pagesPaths array.
• The lookup strategy for each page is as such:
  1. Check first item in pagesPaths array
  2. Keep iterating on pagesPaths until all pages are discovered

Describe the solution you'd like

My current solution is the following:

1. Added the following option to next.config.js:

module.exports = {
  pagesPaths: ["sites/example/pages", "src/pages"],
};

2. Updated my fork of next to be able to handle this config and read from multiple directories.

The working reference implementation can be found here

I'd love for this to be considered for merging into next. I understand there's been similar issues in the past and they'd been shot down because of the "convention over configuration" argument, but I have a working reference implementation that does not compromise current workings of next, only extends it with negligible cost.

Describe alternatives you've considered

1. Create a script that merges the required directories together when starting the next process - while this works for building, because there are filesystem changes required to make this work, it is not feasible in development mode.
2. Use TypeScript path aliases to override components - this only works for overriding files, but not for adding new ones.
3. Use my fork of next.js - this requires continuous active maintenance and other people looking for this feature won't be able to benefit.
4. Use patch-package to add the delta of the changes after install - this has all the disadvantages of the previous solution, plus there's an extra step of having to build, copy and apply patch after each update.

[kachkaev]

Interesting! Have you tried using symlinks in the pages directory instead? I haven’t, but I wonder if Next.js would pick those up or not. AFAIU, the framework generally supports monorepos, so it might be OK with the trick.

[danielkov]

Well symlinks cannot merge directories together. If I understand correctly, you're proposing running next from the individual repositories and ln -sing each file in the core directory during start. This has the exact same effect and therefore drawbacks as Alternatives/1.. For example, when creating a new file, it would have to be instantly linked too and this way there would be no way to override files, as a symlink with the same name as an already existing file cannot co-exist.

[danielkov]

I've published my forked version on NPM here: https://www.npmjs.com/package/@docler/next. If you're interested, feel free to try it out and let me know if there's any issues to fix.

[ghost]

I have a similar need where I'd also like to provide a base directory of pages, then provide site-specific pages that can override or add new pages to the app. What you're proposing would be a great addition and would also open the door for simply changing the page path based on the env variables.

There were also some ideas for loading pages via a plugin system: #9133.

[danielkov]

I read through that briefly and I cannot find the part of that RFC that would solve this issue. Could you point me in the right direction?

[bordeo]

Thank you @danielkov for you work.
I have just published a single commit that includes your edits with a minor changes to make it compatible with next@v11.0.1
Here you can find a patch applicable with patch-package for next@v11.0.1 that include those edits.

I'm using pagePaths to add pages from different package in order to have a fallback page system.

pagesPaths: ['src/pages', 'node_modules/@mycompany/ui/src/pages']

[bordeo]

⚠️ I have found a bug. When navigating pages through router js (client side), ssr pages won't load data. I discovered it because clicking on a next/link don't fetch __next/data/development/page-name.json
I still not have a solution for this problem.

[yosbelms]

I'm needing this feature. I will prefer this way to structure apps instead of module federation. However I propose a different config structure:

routes: {
    '/': 'src/pages',
    '/site1': 'src/sites/site1/pages',
    '/site2': 'src/sites/site2'/pages,
}

In that way navigating to /site1/example should render src/sites/site1/pages/example.js

[miloofcroton]

Here's what always annoyed me about pages: what if I want to deploy just a backend? I still have to write something at /pages/api/someEndpoint.ts. This is just nonsensical. I would rather it be called routes, which is an isomorphic concept. A "page" is only really a front end concept. Do you want Nextjs's API route functionality to only be used as a backend-for-frontend in the same app that you have your front end, or do you want people to look at it as potentially a general purpose tool for writing backends quickly and painlessly?

I propose /routes and /src/routes in addition to or in replacement to /pages/ and /src/pages. I would be fine with this as a hardcoded value, but it also seems reasonable to have literally a single config value that can be defined for where your pages dir is.

Also note that people use the 'router' abstraction on the front end and back end, so it is a universal concept. There's react router, and there's express router.

@timneutkens I know you have been adamantly against a configuration option for the pages dir in the past (example), but would you agree that /pages/api/someEndpoint is not very good naming, especially for solely a back end that has no front end and thus no pages? How about /routes/api/someEndpoint?

If you'd like, I could make a separate thread for this.

[nora-soderlund]

"I haven't heard a good argument to allow it" is a naive statement.

You haven't heard a good argument according to yourself. There's hundreds of reactions disagreeing with your decision about this. That should be enough in an open-source project to warrant lifting your foot off this.

I don't understand why Vercel is so against this, if we go back to the days of self-hosting with IIS and Apache, there was not a single soul even trying to propose a limitation of e.g. the IIS physical path or the Apache filesystem locations, yet they pretty much ran the internet.

Here's a "good argument" if you wish, by telling developers that you're putting your foot down on not implementing this, you're indirectly telling them:

• Either follow my personal desires ("I wouldn't use src/pages personally") or do it yourself in a fork and stop receiving security updates.

This is certainly not the way to go about this, in my opinion...

---

You as a company definitely do not need to provide support for contributions like these that you do not agree with, you can make that exceptionally clear, but at the very least - allow contributors to implement what's asked for, even if it has to be put behind some kind of obscure "unsupported" flag to use.