ci: Type check perf

[m-shaka]

The author should do the following, if applicable

part of #3377

• Add tests
• Run tests
• bun run format:fix && bun run lint:fix to format the code
• Add TSDoc/JSDoc to document the code

detail

The main goal is to compare type check performances between main branch and each PR.

What CI jobs I added do is:

• generate Hono app with 200 routes
• compile a script using the app above and take diagnostics
• on main branch, store the result to GitHub actions cache
• on PR, compare the current performance with the latest stored result

I think it'd be nice to post the comparison as a comment on the PR (after making sure caching is working)

question

Should I do something for Codecov?

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.77%. Comparing base (dfbd717) to head (c342b9b).
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3406   +/-   ##
=======================================
  Coverage   95.77%   95.77%           
=======================================
  Files         155      155           
  Lines        9310     9310           
  Branches     2725     2756   +31     
=======================================
  Hits         8917     8917           
  Misses        393      393           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[yusukebe]

Hi @m-shaka !

This is great! It's unsure how this will be effective and how to measure performance accurately, but starting to try using it is important.

One thing about naming. How about creating perf-measures and putting this type-check-perf as type-check? In the future, we may create other scripts for measuring performance like perf-measures/routers.

[yusukebe]

Should I do something for Codecov?

I'll investigate it, but it can be done after merging this.

[m-shaka]

Thank you for the feedback!
Could you explain more about what kind of files you are planning to place in type-check and perf-measures respectively?

[yusukebe]

Could you explain more about what kind of files you are planning to place in type-check and perf-measures respectively?

We may not do that, and we don't have to do that, but I was thinking:

• Router speed
• Req => Res performance
• File/Bundle size like test: introduce build tests #2975
• Other things in the benchmarks

[exoego]

We can ignore coverage for type perf code by adding

type-check-perf/**/*.ts

to

hono/vitest.config.ts

Lines 19 to 30 in dfbd717

	exclude: [
	...(configDefaults.coverage.exclude ?? []),
	'benchmarks',
	'runtime-tests',
	'build.ts',
	'src/test-utils',
	// types are compile-time only, so their coverage cannot be measured
	'src/**/types.ts',
	'src/jsx/intrinsic-elements.ts',
	'src/utils/http-status.ts',
	],

[yusukebe]

We can ignore coverage for type perf code by adding

Yes, we should do it. Thank you for the advice.

[m-shaka]

Got it. You just mean mv type-check-perf perf-measures/type-check, don't you?

[yusukebe]

@m-shaka

Got it. You just mean mv type-check-perf perf-measures/type-check, don't you?

Yes. Exactly!

[m-shaka]

@yusukebe moved the directory and made some refactoring!

[yusukebe]

LGTM!

[yusukebe]

@m-shaka

Thanks! Let's go with this and keep improving it.