Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(browser): fix browser testing url for https #4855

Merged
merged 24 commits into from
Jan 16, 2024

Conversation

hi-ogawa
Copy link
Contributor

@hi-ogawa hi-ogawa commented Jan 3, 2024

Description

Closes #4844

I used ViteDevServer.resolvedUrls to grab the url for browser testing. I think this should be always available and If this is not available, then it's probably a Vite bug, so I didn't put any fallback or validation here.

On a related note, currently Vite's base config is not supported yet (as reported in #4686), but ViteDevServer.resolvedUrls will be also generated according to base config, so I think treating resolvedUrls as a single source of truth is appropriate for browser testing.
(Actually base doesn't change browser testing url since it's always root /, but I guess it would still make sense to use resolvedUrls as a way to get origin.)

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

  • It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
  • Ideally, include a test that fails without this PR but passes with it.
  • Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.

Tests

  • Run the tests with pnpm test:ci.

Documentation

  • If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

  • Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.

Copy link

netlify bot commented Jan 3, 2024

Deploy Preview for fastidious-cascaron-4ded94 canceled.

Name Link
🔨 Latest commit ba594a4
🔍 Latest deploy log https://app.netlify.com/sites/fastidious-cascaron-4ded94/deploys/65a6284a38df8d0008e2fc98

@hi-ogawa hi-ogawa marked this pull request as ready for review January 3, 2024 10:15
@@ -143,6 +143,7 @@ jobs:
strategy:
matrix:
browser: [[chrome, chromium], [firefox, firefox], [edge, webkit]]
fail-fast: false
Copy link
Contributor Author

@hi-ogawa hi-ogawa Jan 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the above test job, I thought it's better to run all matrix without fail-fast so that it's easier to identify potential browser-specific quirks.

test/browser/specs/server-url.test.mjs Outdated Show resolved Hide resolved
@hi-ogawa hi-ogawa changed the title fix(vitest): fix browser testing url for https fix(browser): fix browser testing url for https Jan 5, 2024
@hi-ogawa hi-ogawa marked this pull request as draft January 5, 2024 09:34
Comment on lines +23 to +25
// separate cacheDir from test/browser/vite.config.ts
// to prevent pre-bundling related flakiness on Webkit
cacheDir: path.join(path.dirname(fileURLToPath(import.meta.url)), "node_modules/.vite")
Copy link
Contributor Author

@hi-ogawa hi-ogawa Jan 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As commented in #4879 (comment), I found that webkit flakiness is related to Vite pre-bundling somehow. For now, separating vite cacheDir can workaround the issue, so I added this here.

To verify this, I repeated pnpm run test:browser:playwright 20 times consecutively on CI and it didn't fail https://github.com/vitest-dev/vitest/actions/runs/7454247103/job/20281233631#step:11:1

I'm still not sure the same issue could be reproduced outside of Vitest since this issue might be related to Vitest being a linked dep for this in-repository tests.

@hi-ogawa hi-ogawa marked this pull request as ready for review January 8, 2024 23:21
@@ -38,7 +38,7 @@ export function createBrowserPool(ctx: Vitest): ProcessPool {
const provider = project.browserProvider!
providers.add(provider)

const origin = `http://${ctx.config.browser.api?.host || 'localhost'}:${project.browser!.config.server.port}`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a possibility that this might be null if server is restarted (due to a config change): vitejs/vite#15450

Does it affect us?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the reference. There could be something sketchy scenario. I'll look into it.

Also another thing, actually it might not be preferred to use resolveUrls since it looks like Vite has some inconsistency when users set vite config like server.host: "local.example.com".

Let me think about it.

Copy link
Contributor Author

@hi-ogawa hi-ogawa Jan 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just remembered this issue about Vite's --open default behavior:

Essentially Vite does a following when ViteDevServer.openBrowser:

new URL(options.open, resolvedUrls.local[0] ?? resolvedUrls.network[0])

https://github.com/vitejs/vite/blob/2687dbbd4e19c86f9888ee784c9b51598e8b79ca/packages/vite/src/node/server/index.ts#L484-L492

This also means that provider: "none" is already forcing url origin to be resolvedUrls.local[0] ?? resolvedUrls.network[0], so aligning this behavior for all providers should not be that bad.
(Ah, actually that's not the case. new URL(x, y) doesn't change origin to y if x already includes origin.)

Copy link
Contributor Author

@hi-ogawa hi-ogawa Jan 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding Vite server's restart behavior, I don't think that's relevant for Vitest browser mode since browser mode uses a dedicated Vite dev server, which doesn't watch files on its own and is completely re-spawned when Vitest's main server restarts:

// watch is handled by Vitest
server: {
hmr: false,
watch: {
ignored: ['**/**'],
},
},

async initBrowserServer(configFile: string | undefined) {
if (!this.isBrowserEnabled())
return
await this.browser?.close()
this.browser = await createBrowserServer(this, configFile)
}

So, relying on resolvedUrls should be safe before landing vitejs/vite#15450 in Vite v5.1.

Currently the log looks like this for playwright provider:

Restarting due to config changes...

 DEV  v1.2.0 /home/hiroshi/code/others/vitest/test/browser/fixtures/server-url
      Browser runner started at http://localhost:5174/

But webdriverio actually doesn't support restart at all since it kills a whole process when Vitest main server tries to restart and close all existing pools:

this.pool?.close?.()

// TODO: right now process can only exit with timeout, if we use browser
// needs investigating
process.exit()

@hi-ogawa hi-ogawa marked this pull request as draft January 10, 2024 00:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

vitest Browser Mode doesn't cooperate with local server configured for HTTPS
2 participants