Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compiler tests diff runner #275

Merged
merged 147 commits into from
Feb 26, 2025
Merged

Compiler tests diff runner #275

merged 147 commits into from
Feb 26, 2025

Conversation

gabritto
Copy link
Member

@gabritto gabritto commented Feb 4, 2025

No description provided.

@jakebailey
Copy link
Member

jakebailey commented Feb 20, 2025

Comparing CI test time of the runner package before/after AST caching:

  • Regular run: 6m19.137s -> 52.906s
  • With concurrent programs: 6m30.342s -> 1m8.742s
  • In race mode: 2670.664s -> 391.295s

Roughly 6-7x faster.

I still want to get our CI running on better builders for race mode, though. Race mode has about a 5x cost, it seems (race mode does concurrent programs too, of course, so the last two numbers can be directly compared).

@jakebailey
Copy link
Member

Switching to a new pool greatly improved the performance. I mistakenly used an 8 core machine instead of a 16 core, and yet it ran race mode in 86 seconds, which is already like 4x faster even though it's only twice the core count of the default GHA runner. I'll rerun with the 16 core I intended later.

@jakebailey
Copy link
Member

16 core machine nets 76.860s, not much of an improvement, funnily.

@jakebailey
Copy link
Member

Unfortunately after #373, this PR is now crashing on a few tests.

@jakebailey
Copy link
Member

There's one failing test: https://github.com/microsoft/TypeScript/blob/main/tests/cases/conformance/externalModules/typeOnly/chained2.ts

Seemingly in concurrent programs, not all checkers have a symbol at the location for A or B in d.ts.

@gabritto
Copy link
Member Author

Submodule test mappedTypeRecursiveInference.ts relies on the no truncation truncation limit of the type printer (noTruncationMaximumTruncationLength in Strada), otherwise it attempts to print forever. Since we don't have the actual type printer yet, I'm disabling it for now since that's easiest.

@gabritto gabritto merged commit 87b1d82 into main Feb 26, 2025
16 checks passed
@gabritto gabritto deleted the gabritto/diffrunner branch February 26, 2025 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants