[ty] Add some completion ranking improvements #20807

BurntSushi · 2025-10-10T18:32:44Z

This PR contains a few improvements for completions:

We now take the text that has been typed into consideration when
generating completions in all cases. Previously, we only did this for
auto-import completions.
The keyword values None, True and False are now included in
scoped completions.
Completions are not shown within comments.
Completions are not shown within strings.
Completions are still shown within the interpolated segments of
f-strings and t-strings.

We add some new evaluation tasks to capture some of these cases. (And
lots of tests for the string detection logic.)

Note: I did make some changes to the TokenFlags representation.
I'm open to other ideas. But I wanted to flag this since TokenFlags
is used within the representation of a Token, which is a fundamental
data type inside of Ruff. Thankfully, even though TokenFlags gets
bigger, Token itself does not (because of padding/alignment).

github-actions · 2025-10-10T18:34:45Z

Diagnostic diff on typing conformance tests

No changes detected when running ty on typing conformance tests ✅

github-actions · 2025-10-10T18:35:39Z

`mypy_primer` results

No ecosystem changes detected ✅
No memory usage changes detected ✅

github-actions · 2025-10-10T18:55:11Z

`ruff-ecosystem` results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

crates/ty_ide/src/completion.rs

AlexWaygood

Nice!

crates/ty_ide/src/completion.rs

crates/ruff_python_parser/src/token.rs

crates/ty_ide/src/completion.rs

sharkdp · 2025-10-13T08:05:33Z

I planned to take a look at this, but it looks like we have two reviews already.

This means that when new characters are typed, the LSP client should send new completion requests to the LSP server. Previously, only one request would be emitted and then the client would filter them. The primary reason to do this is so the server can take what has been typed into account when returning completions. ty is already doing this with auto-import, because it isn't really feasible to do otherwise: always returning *every* symbol throughout an environment isn't a good way to do business. But, we likely also want to take what is typed into account for all completion requests so that we can do our own ranking. Moreover, if we take what is typed into account when doing a completion request, then consider a request that was initiated with `fubar` already typed. If the user hits backspace and gets `fub`, then since the LSP client isn't sending another request, the user won't see all matching completions. This is why we need to mark the response as "incomplete." This will instruct the LSP client to re-send completion requests. The trade-off is that the client will make more completion requests. And if completions are slow, this will magnify that effect for users.

Instead of only extracting this for scoped completions using the AST, we now extract the typed text for all requests in a uniform way directly from the token stream. This doesn't otherwise change any behavior.

This doesn't do any ranking or sorting, but it does filter our completions by the text the user has typed so far. We were already doing this for auto-import. For non-auto-import completions, the number of symbols tends to be much smaller, so I don't really expect this to result in any meaningful performance improvements. This does mean our eval score gets a lot better, and indeed, the eval reflects the real world more accurate since LSP clients do their own filtering.

I had the logic flipped here, which resulted in `show-one` not matching expected tasks in some cases. Basically, when the `--file-name` and `--index` flags aren't given, then the CLI should behave as if all file names and indices are matched.

This is mostly a basic sanity test that we can return completions inside of the interpolated segments of f-strings and t-strings.

This had been annoying me for a while: completions shouldn't be offered inside of comments. We detect this by looking for a `Comment` token before the offset (with only `NonLogicalNewline` being allowed as an intermediate token).

... except when we're in the interpolated segments of an f-string or a t-string. Previously, we would show completions for interpolated segments of an f-string or a t-string. Which is good. But we did this by virtue of never trying to suppress completions when in the non-interpolated part of an f-string/t-string, or any other kind of string. Whcih is bad. Here, we do our best to detect whether the cursor is within a string or not. This seems somewhat straight-forward in the case of a string that is properly opened/closed. But for a string that isn't closed yet---which is likely common while typing---this is less straight-forward. For the most part, I looked at some examples and tried to reverse engineer the kinds of patterns we see in the token stream when there is an unclosed string. It's likely not fully correct, but we can improve it as we go.

Since `None` is a keyword, it isn't currently included in completions. But this is a common papercut that is really annoying, and actually makes typing `None` into the playground nearly impossible. So let's try to fix this.

This doesn't implement fully context aware keyword completions, but it does let us include the 3 keywords that are also values in completions. This helps improve the completion experience when trying to write one of these values in the ty playground. Ranking could still be improved here. We haven't yet devised a mechanism for ranking "builtin" symbols above others. It's even plausible that these _particular_ symbols should get special treatment over and above others.

MichaReiser · 2025-10-15T08:31:28Z

Hmm, I think the evaluation results differ between mac and linux?

crates/ty_python_semantic/src/types.rs

crates/ty_python_semantic/src/types/class.rs

MichaReiser · 2025-10-15T08:56:13Z

Sorry @BurntSushi, I'll squash-merge the commits because @AlexWaygood and I don't have as good a commit hygiene as you.

…rable * origin/main: [ty] Add (unused) `inferable` parameter to type property methods (#20865) Run macos tests on macos (#20889) Remove `release` CI job (#20887) [ty] CI: Faster ecosystem analysis (#20886) Remove `strip` from release profile (#20885) [ty] Sync vendored typeshed stubs (#20876) [ty] Add some completion ranking improvements (#20807) Improved error recovery for unclosed strings (including f- and t-strings) (#20848) Enable lto=fat (#20863) [`pyupgrade`] Extend `UP019` to detect `typing_extensions.Text` (`UP019`) (#20825) [`flake8-bugbear`] Omit annotation in preview fix for `B006` (#20877) fix(docs): Fix typo in `RUF015` description (#20873) [ty] Improve and extend tests for instance attributes redeclared in subclasses (#20866) [ty] Ignore slow seeds as a temporary measure (#20870) Remove parentheses around multiple exception types on Python 3.14+ (#20768) Update Black tests (#20794)

…nt-sets * dcreager/non-non-inferable: (174 commits) [ty] Add (unused) `inferable` parameter to type property methods (#20865) Run macos tests on macos (#20889) Remove `release` CI job (#20887) [ty] CI: Faster ecosystem analysis (#20886) Remove `strip` from release profile (#20885) [ty] Sync vendored typeshed stubs (#20876) [ty] Add some completion ranking improvements (#20807) Improved error recovery for unclosed strings (including f- and t-strings) (#20848) Enable lto=fat (#20863) [`pyupgrade`] Extend `UP019` to detect `typing_extensions.Text` (`UP019`) (#20825) [`flake8-bugbear`] Omit annotation in preview fix for `B006` (#20877) fix(docs): Fix typo in `RUF015` description (#20873) [ty] Improve and extend tests for instance attributes redeclared in subclasses (#20866) [ty] Ignore slow seeds as a temporary measure (#20870) use existing method Remove parentheses around multiple exception types on Python 3.14+ (#20768) Update Black tests (#20794) just the api parts [ty] Fix further issues in `super()` inference logic (#20843) [ty] Document when a rule was added (#20859) ...

BurntSushi requested review from AlexWaygood, MichaReiser, carljm, dcreager, dhruvmanila and sharkdp as code owners October 10, 2025 18:32

BurntSushi removed request for AlexWaygood, carljm and dcreager October 10, 2025 18:35

AlexWaygood added server Related to the LSP server ty Multi-file analysis & type inference labels Oct 10, 2025

BurntSushi force-pushed the ag/completion-improvements branch from 9415542 to a788c47 Compare October 10, 2025 18:44

AlexWaygood reviewed Oct 10, 2025

View reviewed changes

crates/ty_ide/src/completion.rs Outdated Show resolved Hide resolved

BurntSushi force-pushed the ag/completion-improvements branch from a788c47 to 94d68c4 Compare October 10, 2025 22:06

AlexWaygood approved these changes Oct 11, 2025

View reviewed changes

sharkdp removed their request for review October 13, 2025 08:04

BurntSushi and others added 10 commits October 15, 2025 10:21

[ty] Slight refactoring of "typed text" extraction

b856e24

Instead of only extracting this for scoped completions using the AST, we now extract the typed text for all requests in a uniform way directly from the token stream. This doesn't otherwise change any behavior.

[ty] Fix the show-one evaluation command

33a034a

I had the logic flipped here, which resulted in `show-one` not matching expected tasks in some cases. Basically, when the `--file-name` and `--index` flags aren't given, then the CLI should behave as if all file names and indices are matched.

[ty] Add completion task for f/t-strings

6d2f4f5

This is mostly a basic sanity test that we can return completions inside of the interpolated segments of f-strings and t-strings.

[ty] Don't show completions in comments

31c2ebe

This had been annoying me for a while: completions shouldn't be offered inside of comments. We detect this by looking for a `Comment` token before the offset (with only `NonLogicalNewline` being allowed as an intermediate token).

[ty] Add None completion task to evaluation

ba40d81

Since `None` is a keyword, it isn't currently included in completions. But this is a common papercut that is really annoying, and actually makes typing `None` into the playground nearly impossible. So let's try to fix this.

Update evaluation results

7c27f65

Remove is_unclosd_string

7bd9ee2

MichaReiser force-pushed the ag/completion-improvements branch from 94d68c4 to 7bd9ee2 Compare October 15, 2025 08:23

Code review feedback

7850111

AlexWaygood reviewed Oct 15, 2025

View reviewed changes

crates/ty_python_semantic/src/types.rs Outdated Show resolved Hide resolved

crates/ty_python_semantic/src/types/class.rs Outdated Show resolved Hide resolved

AlexWaygood and others added 2 commits October 15, 2025 09:48

These don't need to be pub anymore

761c63f

Ensure completion framework uses project venv over VIRTUAL_ENV

a252c22

MichaReiser mentioned this pull request Oct 15, 2025

numpy-array completion evaluation shows different results between macOS and linux astral-sh/ty#1358

Closed

Update completion results with correct venv

410912f

MichaReiser enabled auto-merge (squash) October 15, 2025 08:55

MichaReiser merged commit 651f796 into main Oct 15, 2025
40 checks passed

MichaReiser deleted the ag/completion-improvements branch October 15, 2025 08:59

sharkdp mentioned this pull request Oct 15, 2025

Code completion astral-sh/ty#86

Open

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[ty] Add some completion ranking improvements #20807

[ty] Add some completion ranking improvements #20807

Uh oh!

BurntSushi commented Oct 10, 2025

Uh oh!

github-actions bot commented Oct 10, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Oct 10, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Oct 10, 2025 •

edited

Loading

Uh oh!

Uh oh!

AlexWaygood left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sharkdp commented Oct 13, 2025

Uh oh!

MichaReiser commented Oct 15, 2025

Uh oh!

Uh oh!

Uh oh!

MichaReiser commented Oct 15, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

[ty] Add some completion ranking improvements #20807

[ty] Add some completion ranking improvements #20807

Uh oh!

Conversation

BurntSushi commented Oct 10, 2025

Uh oh!

github-actions bot commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Diagnostic diff on typing conformance tests

Uh oh!

github-actions bot commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

mypy_primer results

Uh oh!

github-actions bot commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ruff-ecosystem results

Linter (stable)

Linter (preview)

Formatter (stable)

Formatter (preview)

Uh oh!

Uh oh!

AlexWaygood left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sharkdp commented Oct 13, 2025

Uh oh!

MichaReiser commented Oct 15, 2025

Uh oh!

Uh oh!

Uh oh!

MichaReiser commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

github-actions bot commented Oct 10, 2025 •

edited

Loading

github-actions bot commented Oct 10, 2025 •

edited

Loading

`mypy_primer` results

github-actions bot commented Oct 10, 2025 •

edited

Loading

`ruff-ecosystem` results

MichaReiser commented Oct 15, 2025 •

edited

Loading