After the initial render, any direct url visits to pages result in 404s

[vjpr]

Bug report

Describe the bug

in require.js#getPagePath, the pages-manifest.json file is read using:

const pagesManifest = require(join(serverBuildPath, PAGES_MANIFEST))

But every time a new on-demand entry is added to the build, the pages-manifest.json file changes.

Using require though returns a cached version.

This causes requirePage to say module could not be found, which causes a 404.

Fix

Strangely, deleting the require.cache entry didn't resolve it.

delete require.cache[join(serverBuildPath, PAGES_MANIFEST)]

But using the following resolved my issue:

  const pagesManifest = JSON.parse(require('fs').readFileSync(join(serverBuildPath, PAGES_MANIFEST), 'utf8'))

System information

• OS: [e.g. macOS, Windows] macOS
• Browser (if applies) [e.g. chrome, safari] Chrome
• Version of Next.js: [e.g. 6.0.2] v7.0.2-canary.20 / b46d7e8

[timneutkens]

I wonder what's happening here, as it's definitely working fine, all tests are passing. Are you using a custom server / custom webpack config? We already flush the require.cache in the nextjs-require-cache-hot-reloader.js webpack plugin, and that has been working pretty well so far 🤔

[vjpr]

Are you using a custom server / custom webpack config?

Yeh, its quite custom.

We already flush the require.cache in the nextjs-require-cache-hot-reloader.js webpack plugin, and that has been working pretty well so far

Ah, this is probably the source of the problem. I will dig into it.

[vjpr]

Don't know if it is a cause but nextjs-require-cache-hot-reloader.js exports class called ChunkNamesPlugin. A typo I suspect. There is also another plugin called ChunkNamesPlugin.

[timneutkens]

Don't know if it is a cause but nextjs-require-cache-hot-reloader.js exports class called ChunkNamesPlugin. A typo I suspect. There is also another plugin called ChunkNamesPlugin.

Changed the name, good catch! it won't affect this issue though

[vjpr]

delete require.cache[X] is not working for me inside getPagePath.

So seems like a require hook or something is preventing this - probabably something on my end.

[vjpr]

Found the problem.

The issue was to do with my output dir being inside a symlinked dir. Node caches realpaths, but NextJsRequireCacheHotReloader doesn't resolve realpaths before clearing from cache.

join(serverBuildPath, PAGES_MANIFEST)

/xxx/packages/xxx-symlink/pages-manifest.json

require.resolve(join(serverBuildPath, PAGES_MANIFEST))

/xxx/packages/xxx/pages-manifest.json

Fix

NextJsRequireCacheHotReloader must fs.realpath before clearing cache.

function deleteCache (path: string) {
  path = require('fs').existsSync(path) ? require('fs').realpathSync(path) : path
  delete require.cache[path] 
}

realPath on non-existent file will throw.

I wonder how this will effect performance though.

We should cache realpaths perhaps.

[vjpr]

@timneutkens Would you accept a PR for the fix I suggest above?

[timneutkens]

Sounds like the correct fix, however you don't have to use the sync version, the tap is async.