TS compilation perf: steal faster extendShape type from tRPC

[jussisaurio]

I have some empirical evidence in a decently sized closed-source project that liberal use of ZodObject.extend() seems to degrade TS compilation and intellisense performance quite rapidly. I remembered seeing an equivalent implementation of extendShape in tRPC (in that project, called Overwrite) that looked simpler. I did some light profiling on it using said closed-source project and here are the results:

// Zod@jussisaurio

% npx tsc --noEmit --extendedDiagnostics --incremental false

Files:                        1248
Lines of Library:            39145
Lines of Definitions:       126023
Lines of TypeScript:         40594
Lines of JavaScript:             0
Lines of JSON:                   0
Lines of Other:                  0
Identifiers:                183660
Symbols:                    290202
Types:                      147133
Instantiations:            2361427
Memory used:               345043K
Assignability cache size:    34109
Identity cache size:          1470
Subtype cache size:            708
Strict subtype cache size:   22394
I/O Read time:               0.04s
Parse time:                  0.38s
ResolveModule time:          0.09s
ResolveTypeReference time:   0.01s
ResolveLibrary time:         0.01s
Program time:                0.60s
Bind time:                   0.17s
Check time:                  1.73s
printTime time:              0.00s
Emit time:                   0.00s
Total time:                  2.49s

// Zod@3.21.1

% npx tsc --noEmit --extendedDiagnostics --incremental false

Files:                        1248
Lines of Library:            39145
Lines of Definitions:       126004
Lines of TypeScript:         40594
Lines of JavaScript:             0
Lines of JSON:                   0
Lines of Other:                  0
Identifiers:                183556
Symbols:                    286851
Types:                      152232
Instantiations:            7692583
Memory used:               356978K
Assignability cache size:    33682
Identity cache size:          1464
Subtype cache size:            708
Strict subtype cache size:   22394
I/O Read time:               0.04s
Parse time:                  0.39s
ResolveModule time:          0.10s
ResolveTypeReference time:   0.01s
ResolveLibrary time:         0.01s
Program time:                0.60s
Bind time:                   0.17s
Check time:                  2.80s
printTime time:              0.00s
Emit time:                   0.00s
Total time:                  3.57s

// Zod@jussisaurio, run #2

% npx tsc --noEmit --extendedDiagnostics --incremental false

Files:                        1248
Lines of Library:            39145
Lines of Definitions:       126023
Lines of TypeScript:         40594
Lines of JavaScript:             0
Lines of JSON:                   0
Lines of Other:                  0
Identifiers:                183660
Symbols:                    290202
Types:                      147133
Instantiations:            2361427
Memory used:               345221K
Assignability cache size:    34109
Identity cache size:          1470
Subtype cache size:            708
Strict subtype cache size:   22394
I/O Read time:               0.04s
Parse time:                  0.40s
ResolveModule time:          0.10s
ResolveTypeReference time:   0.01s
ResolveLibrary time:         0.01s
Program time:                0.63s
Bind time:                   0.17s
Check time:                  1.80s
printTime time:              0.00s
Emit time:                   0.00s
Total time:                  2.60s

// Zod@3.21.1, run #2

% npx tsc --noEmit --extendedDiagnostics --incremental false

Files:                        1248
Lines of Library:            39145
Lines of Definitions:       126004
Lines of TypeScript:         40594
Lines of JavaScript:             0
Lines of JSON:                   0
Lines of Other:                  0
Identifiers:                183556
Symbols:                    286851
Types:                      152232
Instantiations:            7692583
Memory used:               356570K
Assignability cache size:    33682
Identity cache size:          1464
Subtype cache size:            708
Strict subtype cache size:   22394
I/O Read time:               0.04s
Parse time:                  0.40s
ResolveModule time:          0.10s
ResolveTypeReference time:   0.01s
ResolveLibrary time:         0.01s
Program time:                0.63s
Bind time:                   0.16s
Check time:                  2.81s
printTime time:              0.00s
Emit time:                   0.00s
Total time:                  3.61s

Main thing (apart from faster runtime) is the dramatically lower number of type instantiations.

[netlify]

✅ Deploy Preview for guileless-rolypoly-866f8a ready!

Built without sensitive environment variables

Name	Link
🔨 Latest commit	5b2ad4c
🔍 Latest deploy log	https://app.netlify.com/sites/guileless-rolypoly-866f8a/deploys/651d9277c512c00008157d30
😎 Deploy Preview	https://deploy-preview-2839--guileless-rolypoly-866f8a.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[colinhacks]

I feel like there are edge cases where dropping flatten breaks the inferred type signature...but I can't find them.

I'll do a little more testing then merge. Thanks!

[jussisaurio]

FWIW I can try to run some perf tests on how much keeping flatten has an impact. I'm guessing though that replacing identity with something like & {} in the definition of flatten would also make it perform better due to less instantiations

[jussisaurio]

Used this: https://github.com/jussisaurio/zod-ts-perftest (there are already pregenerated zod schemas in the repo)

Running npm run build-bench, three runs each:

master: 4.51s, 4.56s, 4.51s
PR #2839 (this one): 3.69s, 3.61s, 3.72s
PR #2839 + flatten added back: 3.77s, 3.77s, 3.72s
PR #2845: 2.29s, 2.20s, 2.21s
Both PRs: 2.14s, 2.14s, 2.16s

Not too much of a difference at all between keeping or removing flatten, but for some reason this benchmark is really sensitive to the changes in #2845 😲 I'm sure this particular benchmark is not very well rounded (zod-ts-perftest only generates objects, strings, numbers, booleans, plus a crapton of extends and omits).

[psychedelicious]

We use extend quite a bit. This PR is a marked improvement.

zod 3.22.4:

npx tsc --noEmit --extendedDiagnostics --incremental false

Files:                         3394
Lines of Library:             40241
Lines of Definitions:        310600
Lines of TypeScript:          73937
Lines of JavaScript:              0
Lines of JSON:                    0
Lines of Other:                   0
Identifiers:                 592152
Symbols:                     950503
Types:                       311682
Instantiations:             8492589
Memory used:               1151591K
Assignability cache size:    273407
Identity cache size:          23254
Subtype cache size:            5426
Strict subtype cache size:     7230
I/O Read time:                0.04s
Parse time:                   0.65s
ResolveModule time:           0.20s
ResolveTypeReference time:    0.00s
ResolveLibrary time:          0.01s
Program time:                 1.02s
Bind time:                    0.34s
Check time:                   7.75s
printTime time:               0.00s
Emit time:                    0.00s
Total time:                   9.12s

zod 3.22.4 with this patch:

npx tsc --noEmit --extendedDiagnostics --incremental false

Files:                         3394
Lines of Library:             40241
Lines of Definitions:        310610
Lines of TypeScript:          73937
Lines of JavaScript:              0
Lines of JSON:                    0
Lines of Other:                   0
Identifiers:                 592160
Symbols:                     950391
Types:                       310870
Instantiations:             6238717
Memory used:               1058367K
Assignability cache size:    273343
Identity cache size:          23254
Subtype cache size:            5426
Strict subtype cache size:     7230
I/O Read time:                0.04s
Parse time:                   0.66s
ResolveModule time:           0.21s
ResolveTypeReference time:    0.00s
ResolveLibrary time:          0.01s
Program time:                 1.03s
Bind time:                    0.35s
Check time:                   7.32s
printTime time:               0.00s
Emit time:                    0.00s
Total time:                   8.70s

The time to check the whole app isn't a big deal, but VSCode's TS intellisense is a pain. It slowed down substantially after I rewrote a bunch of schemas using extend. We have a number of nested extend schemas in big unions.

[colinhacks]

Merged a variant in #2845

Thanks!