Typedoc took 2.5 hours in conversions

[Fancy2209]

Search terms

Typedoc takes really long to generate for no apparent reason
Typedoc Hangs
Typedoc takes 2.5 hours to convert
Typedoc Typescript 4.9.5 stuck
Typedoc conversion takes way too long

Expected Behavior

I expected the docs to be generated relatively quickly

Actual Behavior

Using the beta, I tried to compile https://github.com/Fancy2209/docs/tree/New-Typedoc
I am only 4/7 packages in, and typedoc is has taken increasingly longer times, with @awayjs/materials (third package) having taken 47 minutes alone

[debug] Finished conversion in 753401ms
[debug] Finished conversion in 1420293ms
[debug] Finished conversion in 2618209ms
[debug] Finished conversion in 934167ms

Edit: total time was 2.5 hours

[debug] Finished conversion in 753401ms
[debug] Finished conversion in 1420293ms
[debug] Finished conversion in 2618209ms
[debug] Finished conversion in 934167ms
[debug] Finished conversion in 1965765ms
[debug] Finished conversion in 902652ms
[debug] Finished conversion in 416315ms

[debug] Full run took 9047398ms

Full Console Log
ajstypedoc.log

Steps to reproduce the bug

    "@awayjs/core": "^0.9.47",
    "@awayjs/graphics": "^0.5.90",
    "@awayjs/materials": "^0.6.16",
    "@awayjs/renderer": "^0.11.65",
    "@awayjs/scene": "^0.13.279",
    "@awayjs/stage": "^0.11.145",
    "@awayjs/view": "^0.6.64",
    "cp-cli": "^2.0.0",
    "rimraf": "^5.0.7",
    "tslib": "2.3.0",
    "typedoc": "^0.26.0-beta.2",
    "typescript": "^4.9.5"

{
  "files": [],
  "references": [
      {
          "path": "node_modules/@awayjs/core"
      },
      {
          "path": "node_modules/@awayjs/graphics"
      },
      {
          "path": "node_modules/@awayjs/materials"
      },
      {
          "path": "node_modules/@awayjs/renderer"
      },
      {
          "path": "node_modules/@awayjs/scene"
      },
      {
          "path": "node_modules/@awayjs/stage"
      },
      {
          "path": "node_modules/@awayjs/view"
      }
  ]
}

{
    "entryPoints": [
        "node_modules/@awayjs/core",
        "node_modules/@awayjs/graphics",
        "node_modules/@awayjs/materials",
        "node_modules/@awayjs/renderer",
        "node_modules/@awayjs/scene",
        "node_modules/@awayjs/stage",
        "node_modules/@awayjs/view"
    ],
    "name": "AwayJS Docs",
    "entryPointStrategy": "packages",
    "includeVersion": false,
    "out": "./bin",
    "readme": "includes/home.md",

    "packageOptions": {
      "includeVersion": true,
      "entryPoints": ["index.ts"],
      "exclude": [
        "node_modules",
        "dist",
        "tests"
      ],
      "readme": "README.md"
    },

    "logLevel": "Verbose"
}

Running through the docs command

  "scripts": {
    "rimraf": "rimraf",
    "docs": "npm run docs:clear && npm run build && npm run docs:misc",
    "docs:clear": "npm run rimraf -- bin",
    "docs:misc": "cp-cli theme/assets/images/favicon.ico bin/favicon.ico",
    "build": "npx tsc --build && npx typedoc --readme includes/home.md"
  },

Environment

• Typedoc version: 0.26.0-beta.2
• TypeScript version: 4.9.5
• Node.js version: v20.9.0
• OS: Windows 11

[Gerrit0]

Yikes, that's a ridiculously long time... hopefully profiling will quickly reveal the problem when I have time to look this weekend.

[Gerrit0]

I had to rename @awayjs/stage/lib/image/Index.ts to @awayjs/stage/lib/image/index.ts to make the project compile on my box, with a case-sensitive filesystem.

Once I did this, TypeDoc consistently finishes building your project in ~17 seconds, using at peak ~1gb RAM

There might be something weird going on with the case-insensitive system on Windows, I can try a VM later, but...

[Gerrit0]

Decided to try a Windows VM on the off chance it still booted... it did. Slower, taking 55 seconds to run, but it still finished within a reasonable amount of time, without requiring the file rename.

I'm out of ideas beyond something very wrong with your system (out of ram? nearly full disk with an OS trying to compress things to save space? building on a network share that's having issues? insane antivirus?)

[Fancy2209]

Out of RAM? Don't think so but not sure, i'll try running just typedoc
Nearly full disk? I still have ~20GB, I'd hope it's not doing that
building on a network share that's having issues? No, I'm compiling this fully locally
insane antivirus? I just use Windows Defender, though all antivirus are a little bit insane

[Dinnerbone]

I got curious and gave this a try in my usual Windows environment. First project took 7 minutes, which is in line with @Fancy2209's first results.

[debug] Reading tsconfig at ./tsconfig.json
[debug] Using TypeScript 4.9.5 from ./node_modules/typescript/lib
[debug] Expanded ./node_modules/@awayjs/core to:
        ./node_modules/@awayjs/core/package.json
[debug] Expanded ./node_modules/@awayjs/graphics to:
        ./node_modules/@awayjs/graphics/package.json
[debug] Expanded ./node_modules/@awayjs/materials to:
        ./node_modules/@awayjs/materials/package.json
[debug] Expanded ./node_modules/@awayjs/renderer to:
        ./node_modules/@awayjs/renderer/package.json
[debug] Expanded ./node_modules/@awayjs/scene to:
        ./node_modules/@awayjs/scene/package.json
[debug] Expanded ./node_modules/@awayjs/stage to:
        ./node_modules/@awayjs/stage/package.json
[debug] Expanded ./node_modules/@awayjs/view to:
        ./node_modules/@awayjs/view/package.json
[debug] Reading project at ./node_modules/@awayjs/core
[debug] Reading tsconfig at ./node_modules/@awayjs/core/tsconfig.json
[debug] Reading project at ./node_modules/@awayjs/graphics
[debug] Reading tsconfig at ./node_modules/@awayjs/graphics/tsconfig.json
[debug] Reading project at ./node_modules/@awayjs/materials
[debug] Reading tsconfig at ./node_modules/@awayjs/materials/tsconfig.json
[debug] Reading project at ./node_modules/@awayjs/renderer
[debug] Reading tsconfig at ./node_modules/@awayjs/renderer/tsconfig.json
[debug] Reading project at ./node_modules/@awayjs/scene
[debug] Reading tsconfig at ./node_modules/@awayjs/scene/tsconfig.json
[debug] Reading project at ./node_modules/@awayjs/stage
[debug] Reading tsconfig at ./node_modules/@awayjs/stage/tsconfig.json
[debug] Reading project at ./node_modules/@awayjs/view
[debug] Reading tsconfig at ./node_modules/@awayjs/view/tsconfig.json
[info] Converting project at ./node_modules/@awayjs/core
[debug] Using TypeScript 4.9.5 from ./node_modules/typescript/lib
[debug] Expanded ./node_modules/@awayjs/core/index.ts to:
        ./node_modules/@awayjs/core/index.ts
[debug] Converting with 1 programs 1 entry points
[debug] Finished getting entry points in 1221ms
[debug] Begin readme.md/package.json search at ./node_modules/@awayjs/core
./node_modules/@awayjs/core/lib/events/EventDispatcher.ts:19:4 - [warning] Encountered an unknown block tag @method.

19       * @method addEventListener

./node_modules/@awayjs/core/lib/events/EventDispatcher.ts:34:4 - [warning] Encountered an unknown block tag @method.

34       * @method removeEventListener

./node_modules/@awayjs/core/lib/events/EventDispatcher.ts:51:4 - [warning] Encountered an unknown block tag @method.

51       * @method dispatchEvent

./node_modules/@awayjs/core/lib/events/EventDispatcher.ts:66:4 - [warning] Encountered an unknown block tag @method.

66       * @method hasListener

./node_modules/@awayjs/core/lib/events/AssetEvent.ts:6:3 - [warning] Encountered an unknown block tag @export.

6     * @export class away.events.AssetEvent

./node_modules/@awayjs/core/lib/events/EventDispatcher.ts:6:2 - [warning] Encountered an unknown block tag @export.

6    * @export class away.events.EventDispatcher

./node_modules/@awayjs/core/lib/events/IEventDispatcher.ts:13:4 - [warning] Encountered an unknown block tag @method.

13       * @method addEventListener

./node_modules/@awayjs/core/lib/events/IEventDispatcher.ts:21:4 - [warning] Encountered an unknown block tag @method.

21       * @method removeEventListener

./node_modules/@awayjs/core/lib/events/IEventDispatcher.ts:29:4 - [warning] Encountered an unknown block tag @method.

29       * @method dispatchEvent

./node_modules/@awayjs/core/lib/events/IEventDispatcher.ts:36:4 - [warning] Encountered an unknown block tag @method.

36       * @method hasListener

./node_modules/@awayjs/core/lib/library/AbstractionBase.ts:10:3 - [warning] Encountered an unknown block tag @export.

10     * @export class away.pool.AbstractionBase

./node_modules/@awayjs/core/lib/library/IDUtil.ts:21:5 - [warning] Encountered an unknown block tag @langversion.

21       *  @langversion 3.0

./node_modules/@awayjs/core/lib/library/IDUtil.ts:22:5 - [warning] Encountered an unknown block tag @playerversion.

22       *  @playerversion Flash 9

./node_modules/@awayjs/core/lib/library/IDUtil.ts:23:5 - [warning] Encountered an unknown block tag @playerversion.

23       *  @playerversion AIR 1.1

./node_modules/@awayjs/core/lib/library/IDUtil.ts:24:5 - [warning] Encountered an unknown block tag @productversion.

24       *  @productversion Flex 3

(### this is where it hangs for 7 minutes ###)

[warning] The signature ColorTransform.constructor has an @param with name "redMultiplier", which was not used.
[warning] The signature ColorTransform.constructor has an @param with name "greenMultiplier", which was not used.
[warning] The signature ColorTransform.constructor has an @param with name "blueMultiplier", which was not used.
[warning] The signature ColorTransform.constructor has an @param with name "alphaMultiplier", which was not used.
[warning] The signature ColorTransform.constructor has an @param with name "redOffset", which was not used.
[warning] The signature ColorTransform.constructor has an @param with name "greenOffset", which was not used.
[warning] The signature ColorTransform.constructor has an @param with name "blueOffset", which was not used.
[warning] The signature ColorTransform.constructor has an @param with name "alphaOffset", which was not used.
[warning] The signature Transform.lookAt has an @param with name "target", which was not used.
[warning] The signature PartialImplementationError.constructor has an @param with name "message", which was not used.
[warning] The signature Box.setTo has an @param with name "yz", which was not used.
[warning] The signature Matrix.constructor has an @param with name "a", which was not used.
[warning] The signature Matrix.constructor has an @param with name "b", which was not used.
[warning] The signature Matrix.constructor has an @param with name "c", which was not used.
[warning] The signature Matrix.constructor has an @param with name "d", which was not used.
[warning] The signature Matrix.constructor has an @param with name "tx", which was not used.
[warning] The signature Matrix.constructor has an @param with name "ty", which was not used.
[warning] The signature AssetLibraryBundle.constructor has an @param with name "me", which was not used.
[warning] The signature ParserUtils.arrayBufferToImage has an @param with name "image", which was not used.
[warning] The signature ParserUtils.byteArrayToImage has an @param with name "image", which was not used.
[warning] The signature ParserUtils.blobToImage has an @param with name "image", which was not used.
[warning] The signature ParserUtils.blobToAudio has an @param with name "audio", which was not used.
[debug] Finished conversion in 422457ms
[warning] IAsyncService, defined in ./node_modules/@awayjs/core/lib/utils/AsyncServicesLibrary.ts, is referenced by AsyncServicesLibrary.serviceLibraries but not included in the documentation.
[warning] IWaveAudioMeta, defined in ./node_modules/@awayjs/core/lib/audio/WaveAudio.ts, is referenced by WaveAudioData.meta
but not included in the documentation.
[warning] IConfigStore, defined in ./node_modules/@awayjs/core/lib/managers/ConfigManager.ts, is referenced by ConfigManager.addStore.T but not included in the documentation.
[debug] Validation took 3ms
(snip)

I didn't let it continue to verify the rest though.

[Dinnerbone]

The slowness is SourcePath.onBeginResolve()

context.project.reflections has 4135 entries (for the first project), and each iteration takes about 150ms - entirely spent within getRepository().

It's super fast if you don't have git accessible. The disableGit option doesn't seem to work for me, to bypass it.

[Fancy2209]

Adding "disableGit": true, ontypedoc.json works as a temporary fix, it has to be set in packageOptions and not outside of it though
Edit: Nevermind, Typedoc is refusing to emit anything with that on

[Gerrit0]

Fixed with 0.26.0-beta.3, with v0.26.0-beta.4 I also added some tests to hopefully avoid this issue in the future, and improved the performance of the source repo when multiple repos are involved.