Add twoslash repros to issues in this repo #39628
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
typescript-bot
added
Author: Team
For Uncommitted Bug
|
I'm going to merge when green, given that there's really nothing in here. All the code lives in: https://github.com/microsoft/TypeScript-Twoslash-Repro-Action |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #35389
From microsoft/TypeScript-Website#130 (comment)
Compiler-back Bug Repros
source
Keeping on top of compiler bugs is hard work, even just hitting watch on the TypeScript repo gets you many hundreds of emails a day from GitHub. Proving and validating a bug is easier now, thanks to the Playground improvements (supporting many TS versions, a lot more compiler flags etc), but it's still hard to keep on top of and Ryan C basically lives in an almost permanent state of triage in our issues.
To give us the chance to improve our bug reports via automation, I wrote an RFC pitching that we can support letting people write compiler backed bug reports in issues/comments which we can use to know when a bug is fixed, and if it has regressed in the future.
There are two parts to this system:
Making Good Bug Reports
This is done by offering a web-app which gives you real-time information about the bug report you are making. Here's an example of the bug workbench which shows an inconsistency in the way we present a particular type ( #39262 )
The app uses the same twoslash syntax we use in code snippets inside the TypeScript website to highlight type information for a particular identifier. The workbench calls the results of these queries assertions, you can assert things like:
The site's goal is to teach you the syntax, and get you making repros quickly, so there's a lot of documentation in the app:
I've been using it to write code samples for the TypeScript v2 website as it's faster than using my text editor and the CLI.
Verifying Bugs
Once you've got your repro, it gets pasted into an issue or a comment on an issue. From here it gets picker up by the verification tools. This is a GitHub Action you can find at microsoft/TypeScript-Twoslash-Repro-Action and its job is to run nightly and provide useful information about the state of your compiler-backed bug report.
You would write a repro like:
Then overnight the bot would post something like this:
Expanding the historical section, you can see that the verification step has also checked the repro against older builds:
So, it looks like the bug only existed for 3.8 and 3.9. This bug was looked at after I created the repro and is actually fixed now, so the nightly run updates the original comment so that someone looking over the issue can see the new results against historical data.