Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove typeCache #1023

Merged
merged 1 commit into from
Mar 18, 2022
Merged

Remove typeCache #1023

merged 1 commit into from
Mar 18, 2022

Conversation

tmcw
Copy link
Contributor

@tmcw tmcw commented Mar 17, 2022

So, zod has a typeCache internally that looks at every single value it passes and stores it in a map with its derived type. My assertion is that this method is mostly overhead and in the majority of cases makes things slower, not faster.

  • In real-world data, you'll see a lot of unique values. For example, in my application, I'm running zod on data with coordinate values. So every single floating point coordinate gets pushed into this Map, it gets bigger, and becomes more of an issue.
  • In tiny datasets, you never benefit from the cache, hence the benchmarks below.
  • The work saved by this cache is often tiny: you're making a bet that running Map#get and Map#set is faster than running typeof. I think in most JavaScript engines, typeof is actually faster than a Map lookup & set.

I would like to try and make a better 'real world' benchmark because the current benchmarks are so 'micro' and use the same data - I think that the bench improvement here is demonstrating how typeCache makes zod slower on first pass, but there's more to be said about how these features interact when you're dealing with an array of objects of strings, in which case you have a lot of ._parse calls.

Okay, so before & after:

% y benchmark
yarn run v1.22.4
$ ts-node src/benchmarks/index.ts
z.string: empty string x 2,497,402 ops/sec ±0.46% (95 runs sampled)
z.string: short string x 2,485,095 ops/sec ±0.63% (93 runs sampled)
z.string: long string x 2,489,445 ops/sec ±0.60% (93 runs sampled)
z.string: invalid: null x 89,193 ops/sec ±0.86% (95 runs sampled)
z.string: manual parser: long x 863,177,981 ops/sec ±1.72% (84 runs sampled)
z.object: empty: valid x 1,637,222 ops/sec ±6.03% (89 runs sampled)
z.object: empty: valid: extra keys x 1,496,710 ops/sec ±0.54% (92 runs sampled)
z.object: empty: invalid: null x 90,253 ops/sec ±0.65% (96 runs sampled)
z.object: short: valid x 802,009 ops/sec ±0.47% (93 runs sampled)
z.object: short: valid: extra keys x 763,453 ops/sec ±6.04% (91 runs sampled)
z.object: short: invalid: null x 89,108 ops/sec ±1.44% (90 runs sampled)
z.object: long: valid x 412,287 ops/sec ±1.97% (91 runs sampled)
z.object: long: valid: extra keys x 400,272 ops/sec ±5.71% (89 runs sampled)
z.object: long: invalid: null x 89,423 ops/sec ±0.47% (95 runs sampled)
z.union: double: valid: a x 270,721 ops/sec ±0.42% (93 runs sampled)
z.union: double: valid: b x 266,943 ops/sec ±0.85% (96 runs sampled)
z.union: double: invalid: null x 28,386 ops/sec ±6.73% (79 runs sampled)
z.union: double: invalid: wrong shape x 25,265 ops/sec ±5.17% (84 runs sampled)
z.union: many: valid: a x 131,675 ops/sec ±0.42% (92 runs sampled)
z.union: many: valid: c x 130,832 ops/sec ±0.58% (94 runs sampled)
z.union: many: invalid: null x 16,373 ops/sec ±7.35% (83 runs sampled)
z.union: many: invalid: wrong shape x 15,525 ops/sec ±5.16% (86 runs sampled)
z.discriminatedUnion: double: valid: a x 708,001 ops/sec ±0.63% (96 runs sampled)
z.discriminatedUnion: double: valid: b x 699,173 ops/sec ±0.59% (92 runs sampled)
z.discriminatedUnion: double: invalid: null x 94,834 ops/sec ±0.85% (89 runs sampled)
z.discriminatedUnion: double: invalid: wrong shape x 89,306 ops/sec ±1.06% (90 runs sampled)
z.discriminatedUnion: many: valid: a x 695,504 ops/sec ±1.11% (92 runs sampled)
z.discriminatedUnion: many: valid: c x 703,268 ops/sec ±0.64% (95 runs sampled)
z.discriminatedUnion: many: invalid: null x 95,817 ops/sec ±0.58% (91 runs sampled)
z.discriminatedUnion: many: invalid: wrong shape x 86,274 ops/sec ±0.45% (91 runs sampled)
✨  Done in 165.58s.

After:

yarn run v1.22.4
$ ts-node src/benchmarks/index.ts
z.string: empty string x 3,260,177 ops/sec ±0.46% (96 runs sampled)
z.string: short string x 3,270,906 ops/sec ±0.57% (97 runs sampled)
z.string: long string x 3,284,191 ops/sec ±0.74% (95 runs sampled)
z.string: invalid: null x 96,424 ops/sec ±0.54% (95 runs sampled)
z.string: manual parser: long x 865,293,767 ops/sec ±1.04% (83 runs sampled)
z.object: empty: valid x 2,339,908 ops/sec ±0.50% (93 runs sampled)
z.object: empty: valid: extra keys x 2,084,512 ops/sec ±0.48% (92 runs sampled)
z.object: empty: invalid: null x 96,100 ops/sec ±0.67% (96 runs sampled)
z.object: short: valid x 1,041,623 ops/sec ±0.89% (93 runs sampled)
z.object: short: valid: extra keys x 1,024,063 ops/sec ±0.69% (93 runs sampled)
z.object: short: invalid: null x 93,605 ops/sec ±3.06% (92 runs sampled)
z.object: long: valid x 435,189 ops/sec ±0.64% (93 runs sampled)
z.object: long: valid: extra keys x 431,583 ops/sec ±0.60% (91 runs sampled)
z.object: long: invalid: null x 95,660 ops/sec ±3.05% (89 runs sampled)
z.union: double: valid: a x 286,175 ops/sec ±0.51% (96 runs sampled)
z.union: double: valid: b x 282,846 ops/sec ±0.51% (96 runs sampled)
z.union: double: invalid: null x 34,700 ops/sec ±0.67% (95 runs sampled)
z.union: double: invalid: wrong shape x 26,512 ops/sec ±5.37% (80 runs sampled)
z.union: many: valid: a x 141,332 ops/sec ±0.57% (95 runs sampled)
z.union: many: valid: c x 137,993 ops/sec ±1.11% (92 runs sampled)
z.union: many: invalid: null x 16,728 ops/sec ±5.38% (87 runs sampled)
z.union: many: invalid: wrong shape x 15,319 ops/sec ±5.55% (88 runs sampled)
z.discriminatedUnion: double: valid: a x 760,234 ops/sec ±0.42% (97 runs sampled)
z.discriminatedUnion: double: valid: b x 765,007 ops/sec ±0.36% (97 runs sampled)
z.discriminatedUnion: double: invalid: null x 92,345 ops/sec ±0.92% (93 runs sampled)
z.discriminatedUnion: double: invalid: wrong shape x 87,265 ops/sec ±1.59% (91 runs sampled)
z.discriminatedUnion: many: valid: a x 715,942 ops/sec ±0.56% (95 runs sampled)
z.discriminatedUnion: many: valid: c x 714,360 ops/sec ±0.74% (95 runs sampled)
z.discriminatedUnion: many: invalid: null x 94,706 ops/sec ±1.36% (93 runs sampled)
z.discriminatedUnion: many: invalid: wrong shape x 85,427 ops/sec ±1.42% (92 runs sampled)
✨  Done in 164.53s.

@colinhacks
Copy link
Owner

So my cache made everything slower. Well done self.

I'm convinced, PR looks great, merging.

@colinhacks colinhacks merged commit acba037 into colinhacks:master Mar 18, 2022
@tmcw tmcw mentioned this pull request Mar 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants