monitoring type-check performance on CI

[m-shaka]

What is the feature you are proposing?

Some changes may accidentally damage type-check performance like #3074 (comment)

It would be great if you could know if your change has an impact on type-check performance.

What do you guys think? Please tell me your opinion and possible solutions to this.
A minimum solution that came to me is storing the result of type-check profiling on main branch and comparing it with the one of the PR

[yusukebe]

Hi @m-shaka

I also wanted that! First, we must create the benchmark method to measure the type-check performance. I've used this way ago:

https://github.com/microsoft/TypeScript/wiki/Performance#performance-tracing

It will be good to make some script to show the score as a concrete number: like This took xxx ms.

[m-shaka]

found that --diagnostics option of tsc is useful. It gives you info like this

Files:              247
Lines:           113520
Identifiers:     110226
Symbols:         256205
Types:           214588
Instantiations: 3438524
Memory used:    469468K
I/O read:         0.07s
I/O write:        0.00s
Parse time:       0.45s
Bind time:        0.22s
Check time:       5.11s
Emit time:        0.00s
Total time:       5.79s

Instantiations and Check time are important to Hono.

I'll make POC later

[yusukebe]

When I worked on #2412 I did the following:

• tsc -p tsconfig.json --generateTrace trace
• Open trace/trace.json in Perfetto

It was useful to improve the performance of the RPC.

[m-shaka]

Yeah, that's useful when you're looking for the bottleneck in compilation

But the first thing we need is to be able to notice the possibility that type checking performance has declined, isn't it?

--diagnostics option is easy to use and the result is easy to process via CLI so it's a minimum solution we should take first, I suppose

[yusukebe]

But the first thing we need is to be able to notice the possibility that type checking performance has declined, isn't it?

Yes. Exactly. We have first to measure the performance. --diagnostics seems to be good!