Excessive memory/time usage (12GB)

[marcj]

Search terms

• Reached heap limit Allocation failed - JavaScript heap out of memory
• memory usage

Expected Behavior

Uses less memory

Actual Behavior

Uses too much memory

Steps to reproduce the bug

We have a project that has ~110k TypeScript LoC across 700 files and 46 packages, which takes excessive amount of memory to build typedoc docs.

• git clone git@github.com:deepkit/deepkit-framework.git
• cd deepkit-framework
• npm install
• ./node_modules/.bin/lerna bootstrap --ignore @deepkit/benchmark
• npm run link
• ./node_modules/.bin/tsc --build tsconfig.json
• npm install typedoc
• time node --max-old-space-size=12096 ./node_modules/.bin/typedoc packages/*/index.ts

To get it to work you have to increase old-space-size to something like 12GB, otherwise you get JavaScript heap out of memory.

After a while (to be precise 4:30 on M1 Max) and a substantial amount (~12GB) of memory used it generates the docs.

Is there anything wrong with our code base where we can improve typedoc generation time?

Environment

• Typedoc version: 0.23.2
• TypeScript version: 4.6.3
• Node.js version:v16.13.0
• OS: macOS 12.4

[Gerrit0]

Looking at your config, you've given TypeDoc a setup which will result in it creating 33 ts.Programs in order to run. Unfortunately, each of these is very expensive to create, and with TypeDoc's current architecture, it needs all of them in order to start running (See also: #1606)

Building TypeDoc's docs (a single ts.Program) takes ~330mb, so your memory usage sounds about right.

[Gerrit0]

If your monorepo can be compiled with a tsconfig.json that doesn't use project references to refer to a bunch of other tsconfig.json files, that should save a lot on time but isn't feasible for many projects...

[Gerrit0]

I'm going to track this under the original #1606 issue, since it's the same root cause

[marcj]

@Gerrit0 ok, thanks for your fast feedback!