Migrate from jest to vitest and stop using jsdom based tests

[AbhiPrasad]

After v8 gets merged in, let's look at migrating from jest to vitest. vitest is way faster, and doesn't fall into the same traps jest does in terms of esm compatibility. Example attempt with Vue SDK: #11071

In addition, we should stop using jsdom based tests, and instead move tests that rely on the browser to use playwright instead. Simulating jsdom in an node environment always has it's faults, easier to not attempt to do that.

First we should align on a common vitest testing standard. Right now we use set globals via vitest/globals, but this is not recommended by vitest themselves. We should instead use direct imports like import { describe, test, expect } from 'vitest';.

sentry-javascript/tsconfig.dev.json

Line 8 in 780875f

"types": ["node", "jest", "vitest/globals"]

There's probably some other discussion that needs to be done here too. We can validate our decisions by changing our vitest usage in the Astro and SvelteKit SDKs.

### Update existing vitest usage
- [ ] https://github.com/getsentry/sentry-javascript/pull/13093
- [ ] https://github.com/getsentry/sentry-javascript/pull/13214
- [ ] Nuxt SDK
- [ ] https://github.com/getsentry/sentry-javascript/pull/13028
- [ ] SolidStart SDK
- [ ] https://github.com/getsentry/sentry-javascript/pull/13026
- [ ] https://github.com/getsentry/sentry-javascript/pull/12960
- [ ] https://github.com/getsentry/sentry-javascript/pull/13027

We can then tackle the following list in whatever order we want!

### Convert to using vitest
- [x] Angular https://github.com/getsentry/sentry-javascript/pull/11091
- [ ] https://github.com/getsentry/sentry-javascript/pull/13092
- [ ] https://github.com/getsentry/sentry-javascript/issues/13434
- [x] Browser Utils
- [ ] https://github.com/getsentry/sentry-javascript/pull/15497
- [ ] https://github.com/getsentry/sentry-javascript/pull/15494
- [ ] https://github.com/getsentry/sentry-javascript/issues/13605
- [ ] https://github.com/getsentry/sentry-javascript/pull/15492
- [ ] https://github.com/getsentry/sentry-javascript/pull/12964
- [ ] https://github.com/getsentry/sentry-javascript/pull/11899
- [ ] https://github.com/getsentry/sentry-javascript/pull/12961
- [ ] https://github.com/getsentry/sentry-javascript/pull/12958
- [ ] https://github.com/getsentry/sentry-javascript/pull/12955
- [x] Wasm
- [ ] https://github.com/getsentry/sentry-javascript/pull/15499
- [ ] https://github.com/getsentry/sentry-javascript/pull/15546

We also have some packages that use mocha. We should try to drop that dependency and make them use vitest

### Remove mocha
- [ ] https://github.com/getsentry/sentry-javascript/pull/11296
- [ ] https://github.com/getsentry/sentry-javascript/pull/11412
- [ ] https://github.com/getsentry/sentry-javascript/pull/11436
- [ ] https://github.com/getsentry/sentry-javascript/pull/11449
- [ ] https://github.com/getsentry/sentry-javascript/pull/11666
- [ ] https://github.com/getsentry/sentry-javascript/pull/11688
- [ ] https://github.com/getsentry/sentry-javascript/pull/11733
- [ ] https://github.com/getsentry/sentry-javascript/pull/11758

[AbhiPrasad]

ref #6040

[s1gr1d]

vistest was already added to Angular here: #11091
I will check it on the list 👍🏻

[billyvg]

@AbhiPrasad Replay was migrated here: #11899

edit though we are using globals and not importing everything from vitest

[timfish]

Looks like the task list needs updating because Node v14 doesn't support vitest?

[AbhiPrasad]

I updated it - looks like we are stuck with jest longer than I thought.

[timfish]

Almost everything is now migrated to Vitest.

There are a couple of packages where I've created branches to migrate them but got stuck on some blockers:

@sentry/remix

Branch - https://github.com/getsentry/sentry-javascript/tree/timfish/test/vitest-remix

• upload-sourcemaps.test.ts tests the the files in ./scripts/** which are all CJS and Vitest cannot not mock require so they can't be tested directly with Vitest

Maybe these files should be in TypeScript and transpiled to the output directory anyway?

@sentry/google-cloud-serverless

Branch - https://github.com/getsentry/sentry-javascript/tree/timfish/test/vitest-google

• google-cloud-grpc.test.ts test fails with UNAUTHENTICATED: Request had invalid authentication credentials
• Looks like dns isn't getting mocked because it hits Google and gets an auth error?

@sentry/nextjs

Branch - https://github.com/getsentry/sentry-javascript/tree/timfish/test/vitest-nextjs

• constructWebpackConfig.test.ts failures because mocks don't appear to be working and other strange behaviour

[AbhiPrasad]

upload-sourcemaps.test.ts tests the the files in ./scripts/** which are all CJS and Vitest cannot not mock require so they can't be tested directly with Vitest

maybe you can try the hack I added for gatsby that patches Module.load to do CJS patching:

sentry-javascript/packages/gatsby/test/gatsby-node.test.ts

Lines 3 to 28 in ba4586f

	vi.hoisted(
	() =>
	void mock('@sentry/webpack-plugin', {
	sentryWebpackPlugin: vi.fn().mockReturnValue({}),
	}),
	);
	// Need to override mock because `gatsby-node.js` loads `@sentry/webpack-plugin` as a CJS file.
	async function mock(mockedUri: string, stub: any) {
	const { Module } = await import('module');
	// @ts-expect-error test
	Module._load_original = Module._load;
	// @ts-expect-error test
	Module._load = (uri, parent) => {
	if (uri === mockedUri) return stub;
	// @ts-expect-error test
	return Module._load_original(uri, parent);
	};
	}
	import { onCreateWebpackConfig } from '../gatsby-node';
	describe('onCreateWebpackConfig', () => {
	// eslint-disable-next-line @typescript-eslint/no-var-requires
	const { sentryWebpackPlugin } = require('@sentry/webpack-plugin');

Looks like dns isn't getting mocked because it hits Google and gets an auth error?

These tests are so hard to debug 😬

I'd be open to try to just completely rewrite them.

constructWebpackConfig.test.ts failures because mocks don't appear to be working and other strange behaviour

let's open a draft PR for this one, we can ask Luca and Charly to help take a look given they have more experience with these tests.

[AbhiPrasad]

I took another look at @sentry/google-cloud-serverless and I think we have two options

1. test using the gcp simulators, and switch to using node integration tests + docker images
2. mock out the PubSub object entirely:

I asked Claude to give me a mock and here's what it provided:

// Mock the entire PubSub module
vi.mock('@google-cloud/pubsub', () => {
  return {
    PubSub: vi.fn().mockImplementation(() => ({
      topic: vi.fn().mockImplementation(topicName => ({
        publish: vi.fn().mockImplementation(() => {
          return Promise.resolve('1637084156623860');
        }),
        publishMessage: vi.fn().mockImplementation(() => {
          return Promise.resolve('1637084156623860');
        }),
        name: `projects/project-id/topics/${topicName}`,
      })),
      close: vi.fn().mockResolvedValue(undefined),
    })),
  };
});

[timfish]

Our instrumentation hooks the google-gax module. If you mock out the PubSub object, it'll never hit any of the google-gax code and the instrumentation does not get tested.

I'm OOO in an hour so I've opened a PR with everything migrated apart from this test which is still failing:

• test(google-serverless): Migrate to Vitest #15567

[AbhiPrasad]

We should write a engineering blog post about this https://sentry.engineering/