Skip to content

test: Run E2E tests in isolated tmp directory #16783

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 32 commits into
base: develop
Choose a base branch
from

Conversation

mydea
Copy link
Member

@mydea mydea commented Jul 1, 2025

This PR updates our E2E test runner to run the apps from an isolated tmp directory, instead of running them inside the monorepo.

Depends on:

The reason to do this is that running them inside the monorepo leads to slightly different behavior, as dependencies can be looked up from parent node_modules folders. This leads to behavior that differs from actual standalone apps.

Now, the whole app is moved into a folder in the system tmp directory. The package.json is adjusted to make it work there (e.g. rewriting volta extends file paths etc), then normally run from there.

Some things had to be changed/fixed to make tests work here properly:

  • Ensure all dependencies are actually defined. E.g. we sometimes used @sentry/core in tests but did not have it as dependency.
  • Ensure every test app has a volta config to ensure consistent versions.
  • Update wrangler in cloudflare apps as v3 had some issues
  • align playwright version used to ensure browsers are always installed
  • removed some unnecessary usage of @sentry/core in tests
  • nuxt & solidstart tests do not need to copy iitm around anymore, this just works now. I also got to remove almost all the overrides of the nft package etc.

@mydea mydea self-assigned this Jul 1, 2025
Copy link
Contributor

github-actions bot commented Jul 2, 2025

size-limit report 📦

Path Size % Change Change
@sentry/browser 23.99 kB - -
@sentry/browser - with treeshaking flags 23.76 kB - -
@sentry/browser (incl. Tracing) 39.59 kB - -
@sentry/browser (incl. Tracing, Replay) 77.69 kB - -
@sentry/browser (incl. Tracing, Replay) - with treeshaking flags 70.78 kB - -
@sentry/browser (incl. Tracing, Replay with Canvas) 82.45 kB - -
@sentry/browser (incl. Tracing, Replay, Feedback) 94.57 kB - -
@sentry/browser (incl. Feedback) 40.75 kB - -
@sentry/browser (incl. sendFeedback) 28.7 kB - -
@sentry/browser (incl. FeedbackAsync) 33.59 kB - -
@sentry/react 25.76 kB - -
@sentry/react (incl. Tracing) 41.58 kB - -
@sentry/vue 28.37 kB - -
@sentry/vue (incl. Tracing) 41.37 kB -0.08% -33 B 🔽
@sentry/svelte 24.01 kB - -
CDN Bundle 25.5 kB - -
CDN Bundle (incl. Tracing) 39.6 kB - -
CDN Bundle (incl. Tracing, Replay) 75.5 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) 80.96 kB - -
CDN Bundle - uncompressed 74.5 kB - -
CDN Bundle (incl. Tracing) - uncompressed 117.63 kB - -
CDN Bundle (incl. Tracing, Replay) - uncompressed 231.68 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) - uncompressed 244.5 kB - -
@sentry/nextjs (client) 43.22 kB - -
@sentry/sveltekit (client) 40.04 kB - -
@sentry/node 162.02 kB -0.01% -1 B 🔽
@sentry/node - without tracing 98.63 kB -0.01% -3 B 🔽
@sentry/aws-serverless 124.4 kB -0.01% -1 B 🔽

View base workflow run

@mydea mydea force-pushed the fn/e2e-test-isolated branch from 12cebf0 to a6a6c3a Compare July 2, 2025 12:34
mydea added a commit that referenced this pull request Jul 2, 2025
)

Not sure why we even have this, but this somehow breaks cloudflare-pages
E2E tests in some scenarios. It also seems simply unnecessary - or is
there a reason we have this? 🤔

Extracted this out from
#16783
@mydea mydea force-pushed the fn/e2e-test-isolated branch from a6a6c3a to 5cbdd25 Compare July 2, 2025 13:14
@mydea mydea changed the title test: Run E2E tests in isolated tmp directory (WIP) test: Run E2E tests in isolated tmp directory Jul 2, 2025
@mydea mydea marked this pull request as ready for review July 2, 2025 13:56
cursor[bot]

This comment was marked as outdated.

Copy link
Member

@s1gr1d s1gr1d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing!!

@@ -17,14 +17,23 @@
},
"dependencies": {
"@sentry/nuxt": "latest || *",
"nuxt": "3.7.0"
"@sentry/core": "latest || *",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only installed because semantic attributes are imported from @sentry/core in the tests -> could it be that those are also exported from @sentry/nuxt?

Copy link
Member

@andreiborza andreiborza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super nice!

@@ -63,7 +63,7 @@
"devDependencies": {
"@cloudflare/workers-types": "4.20250620.0",
"@types/node": "^18.19.1",
"wrangler": "^3.67.1"
"wrangler": "4.22.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we ^ here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hard pinning is fine, we'll revisit soon anyway

mydea added a commit that referenced this pull request Jul 3, 2025
Noticed #16783 in the nuxt-3-min test, that the plugins are not consistently run apparently. In that test, the client integrations plugin was run before the client config plugin, leading to the client not existing yet, and thus not adding the browser tracing integration.

This PR changes this to ensure we have a consistent order of these plugins.
cursor[bot]

This comment was marked as outdated.

mydea added a commit that referenced this pull request Jul 3, 2025
We used to rely on a somewhat complex heuristic to determine if a router change is a pageload or not. This somehow did not work anymore here: #16783 in nuxt-3-min. Likely some vue router difference... However, I think this can be simplified anyhow, by just checking if we have an active pageload span. That seems to work reliably enough.
mydea added a commit that referenced this pull request Jul 3, 2025
We used to rely on a somewhat complex heuristic to determine if a router change is a pageload or not. This somehow did not work anymore here: #16783 in nuxt-3-min. Likely some vue router difference... However, I think this can be simplified anyhow, by just checking if we have an active pageload span. That seems to work reliably enough.
mydea added a commit that referenced this pull request Jul 3, 2025
Noticed #16783 in the nuxt-3-min test, that the plugins are not consistently run apparently. In that test, the client integrations plugin was run before the client config plugin, leading to the client not existing yet, and thus not adding the browser tracing integration.

This PR changes this to ensure we have a consistent order of these plugins.
@mydea mydea force-pushed the fn/e2e-test-isolated branch from c059725 to efd9b27 Compare July 3, 2025 12:17
mydea added a commit that referenced this pull request Jul 3, 2025
Noticed #16783 in the
nuxt-3-min test, that the plugins are not consistently run apparently.
In that test, the client integrations plugin was run before the client
config plugin, leading to the client not existing yet, and thus not
adding the browser tracing integration.

This PR changes this to ensure we have a consistent order of these
plugins.
@mydea mydea force-pushed the fn/e2e-test-isolated branch from efd9b27 to 65a6c9f Compare July 3, 2025 12:41
@mydea
Copy link
Member Author

mydea commented Jul 3, 2025

Also fixes #16796

@mydea mydea linked an issue Jul 3, 2025 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

nuxt-3 (canary) Test Failed
4 participants