feat: default esm SSR build, simplified externalization #8348

patak-dev · 2022-05-26T15:49:50Z

Close #8150
Fix #2152
Close #2157
Close #6812
Should close #6023

Description

For context, check:

Use ESM for SSR builds #8150

Initial work towards defaulting to ESM SSR by default. Still WIP. I'm hoping to get some help from folks that are more deep into the weeds here (ping @benmccann, @brillout, @antfu, @bluwy, just to name a few, any feedback welcome)

Since we move to ESM builds, SSR tests have been moved to "type": "module" now. See related work for non SSR tests started by Anthony here:

test: migrate tests with type: module #8252

You can check the changesets independently:

There is a new ssr.target option called node-cjs (and by default the node target will generate a ESM build).

The ssr-react playground is still using a CJS SSR build because it is failing when rollup leaves the CJS only packages as external without interop. Someone could check what is missing?

If the SSR target is node-cjs, the externalize logic has been left untouched. It still uses the scanner to find known imports, and collect all externals eagerly. Since we would like to push people to use the ESM version, I don't think we should refactor the way it works now. I renamed shouldExternalizeForSSR to cjsShouldExternalizeForSSR, and keep the new functions clean.

For ESM build, the externalization logic is now done lazily (there was a comment in the code already asking why we weren't doing so). Right now the logic may be a bit simplistic (check if it is a package entry and in that case if it should be externalized according to ssr.external, ssr.noExternal, with external being the default).

What is the purpose of this pull request?

Bug fix
New Feature
Documentation update
Other

patak-dev · 2022-05-26T15:51:31Z

Tests are failing because of VitePress, I imagine should now set the format to node-cjs 🤔

patak-dev · 2022-05-26T15:54:01Z

docs/config/ssr-options.md

@@ -20,7 +20,7 @@ Prevent listed dependencies from being externalized for SSR. If `true`, no depen

 ## ssr.target

- **Type:** `'node' | 'webworker'`
+- **Type:** `'node' | 'webworker' | 'node-cjs'`


We could also use build.rollupOptions.output.format === 'cjs', but we need this target during dev for the new externalization logic. We could add a new option to switch externalization logic if not. ssr.lazy: true? And then people can choose to go back to the old eager collection with ssr.lazy: false?

Could the externalization logic also look at build.rollupOptions.output.format? Having two options seems a bit dangerous because someone could change build.rollupOptions.output.format, but not ssr.target

Maybe we could, since we want to promote the ESM build and the new externalization logic, we don't really have a new Vite option here... it is more like a escape hatch that we're leaving for people that can yet use the ESM build. If we do that, we should remove the new 'node-cjs' option in ssr.target and let users control the format and eager externalization using build.rollupOptions.output.format: 'cjs'.

Same opinion as Ben: a single source of truth would be nice and build.rollupOptions.output.format seems like a natural fit.

I don't think you can use build.rollupOptions.output specifically, since it may be an array?

Yes, and there are other issues with it, like needing to conditionally change it based on the ssr flag. But this flag isn't currently passed to the ConfigEnv. We could add it so we could configure using ({ mode, command, ssr }) => { ... }), but forcing a conditional as the only way to configure this doesn't seem very appealing. For other options, we discussed previously with @aleclarson and tried to avoid having SSR only equivalents, but this is a big global switch. Given that we already have ssr.target as an enum, I think that adding a node-cjs or cjs is ok. Although another option like ssr.format: 'esm' | 'cjs' could be easier to deprecate in the future... we could add it as @experimental, and give us room to remove it later on.

Changed to ssr.format: 'cjs', we'll later review all these new options together (there is also ssr.scan and ssr.disabled: 'build' for example)
See #8348 (comment)

patak-dev · 2022-05-26T15:55:42Z

packages/vite/src/node/ssr/ssrExternal.ts

+  return (id: string) => {
+    if (processedIds.has(id)) {
+      return processedIds.get(id)
+    }
+    const external =
+      isBuiltin(id) || (isPackageEntry(id) && isConfiguredAsExternal(id))
+    processedIds.set(id, external)
+    return external
+  }


This is the new lazy externalization logic.

packages/vite/src/node/ssr/ssrExternal.ts

Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>

patak-dev · 2022-05-26T22:10:26Z

Fixed VitePress by using ssr.target: 'node-cjs' for it:

fix: use CJS SSR build for VitePress

WIP for ssr-react (some tests should fail, but pushing in case someone else is checking out the PR)

fix: externalize in resolveId to add resolved extension for deep imports

It looks like the issue is that in CJS, we can require deep imports without the extension. But with ESM, we need to import these using the extension. This is the error when running the generated ssr-react entry-server.js

Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/Users/patak/vite-ssr/playground-temp/ssr-react/node_modules/react-dom/server' imported from /Users/patak/vite-ssr/playground-temp/ssr-react/dist/server/entry-server.js
Did you mean to import react-dom@17.0.2_react@17.0.2/node_modules/react-dom/server.js?

Instead of giving rollup an externals function, we can define that an id is external in resolvedId of resolve.ts, where we have the logic to know if the import is a deep import and check if the extension needs to be added to the id. This seems to work for ssr-react, the build is working. There are still some tests failing (and a new one in ssr-vue), so still WIP.

The commit also removes the ssrRequireHookPlugin for the ESM build.

packages/vite/src/node/ssr/ssrExternal.ts

brillout · 2022-05-28T10:16:18Z

The only reason I can think of for having that node_modules check is for linking and then developing dependencies, but I believe we have more reasons for externalizing linked dependencies. As we have seen some Node.js libraries cause problems and, in general, we are moving towards always externalizing dependencies by default. The same arguments that make us move towards externalizing also holds for linked dependencies. (We cannot and should not assume that Vite is able to process all Node.js libraries out there, e.g. libraries with dynamic require/import that Vite cannot statically determine.) For those who want to use Vite to develop linked dependencies they can use vite.config.js#ssr.noExternal. CC @aleclarson seemed opinionated about this topic. CC @benmccann

…

On Fri 27. May 2022 at 23:12, patak ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In packages/vite/src/node/ssr/ssrExternal.ts <#8348 (comment)>: > + if ( + tryNodeResolve( + id, + undefined, + resolveOptions, + ssr?.target === 'webworker', + undefined, + true + ) + ) { + return true + } + try { + // no main entry, but deep imports may be allowed + const pkgPath = resolveFrom(`${id}/package.json`, root) + if (pkgPath.includes('node_modules')) { Yes, I copied this code from the previous logic, but now it is a good time to change things if we see fit. We can try to simplify this even more and then add back logic if there are issues. — Reply to this email directly, view it on GitHub <#8348 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHVQRTXGF3E6RWH77UI4TLVME3CTANCNFSM5XBSWQOQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

poyoho · 2022-05-28T12:53:53Z

I think now new URL(url, import.meta.url) syntax is allow in SSR. So can we remove

vite/packages/vite/src/node/plugins/assetImportMetaUrl.ts

Line 24 in 2923000

!options?.ssr &&

and

vite/packages/vite/src/node/plugins/workerImportMetaUrl.ts

Lines 93 to 98 in 2923000

    
           if (options?.ssr) { 
        
             this.error( 
        
               `\`new URL(url, import.meta.url)\` is not supported in SSR.`, 
        
               urlIndex 
        
             ) 
        
           }

when target is not node-cjs 🤔

userquin · 2022-05-28T16:30:11Z

ok, pnpm run test now ok on Windows 10 PRO 🚀 : with core.symliks=true (GIT), process.exit(0) building Vite (scripts/patchCJS.ts) and 0b054fc applied (playground/vitestSetup.ts)

aleclarson · 2022-05-28T16:56:23Z

@brillout I agree with your assessment

FWIW, this is what Saus uses to determine if an SSR module should be compiled by Vite:

const isCompiledModule = (id: string) =>
  !id.includes('/node_modules/') && id.startsWith(root + '/')

So technically, if a linked package exists in the project root (but not within node_modules), it will be compiled by Vite. Not exactly sure how perfect this heuristic works when using a monorepo root as viteConfig.root whilst also having pre-compiled packages in the same monorepo (but it's probably no biggie).

patak-dev · 2022-05-28T17:58:49Z

@brillout @aleclarson thanks for reviewing that logic. In the current CJS externalization heuristic, we're currently doing:

      try {
        // no main entry, but deep imports may be allowed
        const pkgPath = resolveFrom(`${id}/package.json`, root)
        if (pkgPath.includes('node_modules')) {
          ssrExternals.add(id)
        } else {
          depsToTrace.add(path.dirname(pkgPath))
        }
        continue
      } catch {}

So the dep isn't added to ssrExternals if it isn't in node_modules, but the it still added to depsToTrace... and I think it ends up traced later on. @dominikg is also reporting issues with linked dependencies that may be related. I think that we an opportunity to try @brillout's proposal here. Let's remove the node_modules check, and we can add it back when we have a bug report to justify it. Done in fix: externalize packages out of node_modules

patak-dev · 2022-05-28T18:00:37Z

I think now new URL(url, import.meta.url) syntax is allow in SSR. So can we remove

This is great @poyoho, let's merge first this PR, and you can later send a new one so we also add docs and a test case. Good catch, the ESM SSR build will clean up a lot of these restrictions.

patak-dev · 2022-05-28T19:52:54Z

Updated the PR to use ssr.format: 'cjs' instead of ssr.target: 'node-cjs'. See rationale here #8348 (comment)

I think we should merge this PR, and we can later discuss the naming (or the need for the option to go back to CJS in a team meeting together with other experimental options we've been adding).

The SSR build is still using plugin common-js, I'll try to remove that in another PR.

antfu · 2022-05-29T03:49:52Z

packages/vite/src/node/build.ts

@@ -431,10 +421,12 @@ async function doBuild(

  try {
    const buildOutputOptions = (output: OutputOptions = {}): OutputOptions => {


Even if the input is .mjs we still emits .js. I am not sure if we should always emit .mjs on ESM build, or respect the input (might need to also convert .mts)

Maybe we should do the same we are doing in v3 for lib mode?

fix(lib): use proper extension #6827

I'll merge this one and work on this logic in another PR.

antfu

Other than my previous comment, it LGTM 🔥

brillout · 2022-05-29T14:36:41Z

LGTM. I'm thrilled we are now always externalizing by default.

patak-dev · 2022-05-29T14:40:59Z

LGTM. I'm thrilled we are now always externalizing by default.

Thanks for pushing for externalizing by default all this time, we needed a bit of time for all the pieces to fall into place 🙂

patak-dev added 3 commits May 26, 2022 17:23

feat: default esm SSR build, simplified externalization

3b08d34

feat: move SSR tests to type module

7aa0b7a

chore: format

92cfb81

patak-dev commented May 26, 2022

View reviewed changes

benmccann reviewed May 26, 2022

View reviewed changes

packages/vite/src/node/ssr/ssrExternal.ts Outdated Show resolved Hide resolved

patak-dev and others added 4 commits May 26, 2022 19:05

perf: avoid some cycles if the id is noExternal

8dc7d98

Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>

chore: merge main

212f81a

fix: use CJS SSR build for VitePress

ac494ac

fix: externalize in resolveId to add resolved extension for deep imports

ef6e8ea

patak-dev marked this pull request as draft May 26, 2022 22:12

fix: externalize only in ssr

2923000

patak-dev marked this pull request as ready for review May 27, 2022 09:46

patak-dev mentioned this pull request May 27, 2022

feat: migrate to ESM (#645) vuejs/vitepress#659

Merged

brillout reviewed May 27, 2022

View reviewed changes

packages/vite/src/node/ssr/ssrExternal.ts Outdated Show resolved Hide resolved

fix: windows

d8adf62

This comment was marked as resolved.

Sign in to view

This comment was marked as off-topic.

Sign in to view

userquin mentioned this pull request May 28, 2022

test-serve(main): not-preserve-symlinks test on Windows fails #8366

Closed

7 tasks

fix: externalize packages out of node_modules

ad9f17e

chore: ssr.format

773f234

antfu reviewed May 29, 2022

View reviewed changes

antfu previously approved these changes May 29, 2022

View reviewed changes

sapphi-red added enhancement New feature or request feat: ssr p3-significant High priority enhancement (priority) labels May 29, 2022

sapphi-red added this to the 3.0 milestone May 29, 2022

patak-dev mentioned this pull request May 29, 2022

feat: add ssr.format to force esm output for ssr #6812

Merged

9 tasks

chore: merge main

b71e1ea

patak-dev dismissed antfu’s stale review via b71e1ea May 29, 2022 12:19

chore: merge main to fix linter in tests

5ce5087

patak-dev merged commit f8c92d1 into main May 29, 2022

patak-dev deleted the feat/esm-ssr-build branch May 29, 2022 13:09

patak-dev added the breaking change label May 29, 2022

patak-dev mentioned this pull request May 29, 2022

feat: derive proper js extension from package type #8382

Merged

4 tasks

patak-dev mentioned this pull request May 30, 2022

feat: ssr build using optimized deps #8403

Merged

4 tasks

hiro-lapis mentioned this pull request Jun 18, 2022

feat: default esm SSR build, simplified externalization vitejs/docs-ja#501

Merged

userquin mentioned this pull request Jul 23, 2022

fix: build command fails in Windows #9329

Merged

9 tasks

patak-dev mentioned this pull request Apr 26, 2023

fix(resolve): deep import resolvedId error #13010

Merged

9 tasks

bluwy mentioned this pull request Apr 24, 2024

fix: special handling for ssr.noExternal in mergeConfig #4766

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: default esm SSR build, simplified externalization #8348

feat: default esm SSR build, simplified externalization #8348

patak-dev commented May 26, 2022

patak-dev commented May 26, 2022

patak-dev May 26, 2022

benmccann May 26, 2022 •

edited

Loading

patak-dev May 26, 2022

brillout May 27, 2022

aleclarson May 28, 2022 •

edited

Loading

patak-dev May 28, 2022

patak-dev May 28, 2022

patak-dev May 26, 2022

patak-dev commented May 26, 2022

brillout commented May 28, 2022 via email

poyoho commented May 28, 2022

This comment was marked as resolved.

This comment was marked as off-topic.

userquin commented May 28, 2022

aleclarson commented May 28, 2022

patak-dev commented May 28, 2022

patak-dev commented May 28, 2022

patak-dev commented May 28, 2022

antfu May 29, 2022

patak-dev May 29, 2022 •

edited

Loading

patak-dev May 29, 2022

antfu left a comment

brillout commented May 29, 2022

patak-dev commented May 29, 2022

		@@ -431,10 +421,12 @@ async function doBuild(

		try {
		const buildOutputOptions = (output: OutputOptions = {}): OutputOptions => {

feat: default esm SSR build, simplified externalization #8348

feat: default esm SSR build, simplified externalization #8348

Conversation

patak-dev commented May 26, 2022

Description

What is the purpose of this pull request?

patak-dev commented May 26, 2022

Choose a reason for hiding this comment

benmccann May 26, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aleclarson May 28, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

patak-dev commented May 26, 2022

brillout commented May 28, 2022 via email

poyoho commented May 28, 2022

This comment was marked as resolved.

This comment was marked as off-topic.

userquin commented May 28, 2022

aleclarson commented May 28, 2022

patak-dev commented May 28, 2022

patak-dev commented May 28, 2022

patak-dev commented May 28, 2022

Choose a reason for hiding this comment

patak-dev May 29, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antfu left a comment

Choose a reason for hiding this comment

brillout commented May 29, 2022

patak-dev commented May 29, 2022

benmccann May 26, 2022 •

edited

Loading

aleclarson May 28, 2022 •

edited

Loading

patak-dev May 29, 2022 •

edited

Loading