[ty] Consider `type_check_only` when ranking completions #20910

decorator-factory · 2025-10-16T04:02:53Z

Summary

This pull request enhances Types with information on whether they are "for type checking only", and changes autocompletion ranking so that

This is the first step in solving astral-sh/ty#816

TODO: I only implemented @type_check_only for classes, right now I'm figuring out how to do it for functions. It's a bit puzzling why this decorator supports functions in the first place. The only place I found it used successfully is scipy-stubs, where "imaginary methods" are used to make certain classes match certain protocols.

Test Plan

Added a import_type_check_only_lowers_ranking test case, which used to fail.

For now we only support classes

github-actions · 2025-10-16T07:04:00Z

Diagnostic diff on typing conformance tests

No changes detected when running ty on typing conformance tests ✅

github-actions · 2025-10-16T07:05:14Z

`mypy_primer` results

No ecosystem changes detected ✅
No memory usage changes detected ✅

AlexWaygood · 2025-10-16T10:57:27Z

TODO: I only implemented @type_check_only for classes, right now I'm figuring out how to do it for functions. It's a bit puzzling why this decorator supports functions in the first place. The only place I found it used successfully is scipy-stubs, where "imaginary methods" are used to make certain classes match certain protocols.

Fine to defer functions for now, IMO! I think the vast majority of uses are with classes. Let's get an MVP in first that we can iterate on in followups

decorator-factory · 2025-10-17T00:34:19Z

crates/ty_ide/src/completion.rs

-/// 1) A `_[^_]` prefix sorts last and
-/// 2) A `__` prefix sorts last except before (1)
+/// 1) Names with no underscore prefix
+/// 2) Names starting with `_` but not dunders


Maybe it's out of scope for this PR, but perhaps deprecated items should also ranked last? (and crossed out in the autocompletion list)

Yeah, agree. I think that sounds like a great change for another PR

…nes/ruff into df-type-check-only

decorator-factory · 2025-10-17T00:41:32Z

crates/ty_python_semantic/src/types/function.rs

    }

-    /// Return `true` if `self` is defined in `module` at runtime.
+    /// Return `true` if `self` is defined in `module`


Was it an important detail that it's really defined in the module at runtime? I am not sure, but type_check_only should be the only exception, probably

crates/ty_python_semantic/src/types.rs

MichaReiser

Nice! This looks great to me. I've one question about whether we can compute the is_type_check_only flag lazily rather than storing it on every ClassLiteral.

It would also be great if we could add an integration test for this. See

ruff/crates/ty_completion_eval/README.md

Line 11 in 651f796

```

for how to do that.

crates/ty_python_semantic/src/types/infer/builder.rs

AlexWaygood

Nice, thank you!

AlexWaygood · 2025-10-17T11:55:58Z

crates/ty_ide/src/completion.rs

            module_name: Some(symbol.module.name(db)),
            import: import_action.import().cloned(),
            builtin: false,
+            is_type_check_only: false,


This seems to mean that completions that would require an auto-import are always considered "not type-check-only". Here's a screenshot from a local playground build using your branch (to build the playground locally, use cd playground && npm start --workspace ty-playground from the Ruff repo root):

module.py has these contents, so I would expect Foo to be ranked below Foooo:

from typing import type_check_only @type_check_only class Foo: ... class Foooo: ...

since that's what I get in other situations on your branch (which is great!):

if getting this right is difficult for auto-import completions, it's fine to defer it for now, but we should add a TODO comment here if so

I think let's defer it for now 👍
I added some TODOs in the integration test.

Right now deprecation and type-check-only-ness are stored on types, not symbols, which makes this difficult. (we'd have to eagerly infer types to produce autoimport suggestions if I understand it correctly)

Hmm, so we'd have to infer the types of every symbol that could possibly be imported in order to provide autocomplete suggestions for a single x = Fo<CURSOR> situation... that would indeed undermine the "only lazily infer exactly what we need" architecture of ty 😆

situation... that would indeed undermine the "only lazily infer exactly what we need" architecture of ty 😆

Yeah, let's wait for @BurntSushi on this, but we can definitely infer the type after we've done some filtering

My loose plan at this point was to basically never infer the type of unimported symbols, by virtue of there being so many. It's true that we could try doing that after we've done filtering, but I fear this could still have a negative overall effect where we end needing to type check a substantial portion of third party code just by virtue of using completions. And I of course worry about the latency impact this will have on auto-import completions.

This is why the way auto-import works currently is to basically bypass any type checking and just pick up symbols straight from the AST. The way I would try to tackle type_check_only support here is to see if we can add it to that AST extraction bit and not try to bring in type checking machinery for it.

crates/ty_python_semantic/src/types.rs

decorator-factory · 2025-10-17T13:01:57Z

I've one question about whether we can compute the is_type_check_only flag lazily rather than storing it on every ClassLiteral

I think it is nicely symmetrical with storing FunctionDecorators on function types 😺 . It also allows pre-populating this flag in things we know to be type-check only like @typing.type_check_only itself. (the diff Alex suggested above)

This is something ty would have to check pretty much every time you use a class in a runtime context (like if you do list(something)), so I don't know which way is better for performance there

MichaReiser · 2025-10-22T15:21:37Z

@decorator-factory I'm not sure what the state of this PR is. Would you mind requesting re-review when it's ready for another round of feedback or you can put your PR in draft mode and change it to "ready for review" once it's ready for another round.

Thank you

decorator-factory · 2025-10-22T16:47:44Z

I think it's ready to merge in the current state. As Alex said, the main use case for @type_check_only is classes/protocols that only exist at runtime, so we can fix the TODO tests for methods in a later PR

AlexWaygood

Thanks, and sorry for the delay in getting back to this! Looks good now, just one comment about the tests

AlexWaygood · 2025-10-23T13:03:50Z

crates/ty_completion_eval/truth/import-deprioritizes-type_check_only/main.py

+# TODO: bound methods don't preserve type-check-only-ness, this is a bug
+# Class().meth_[CURSOR:meth_azurous]
+
+# TODO: auto-imports don't take type-check-only-ness into account, this is a bug
+# UniquePrefixA[CURSOR:module.UniquePrefixAzurous]


I believe the idea with the tests in this crate is that for autocomplete suggestions that don't achieve a great result currently, that's simply reflected in a lower score in crates/ty_completion_eval/completion-evaluation-tasks.csv. So that implies that we should uncomment these lines, and then when we improve our ranking in the future that will simply result in us having to update the score in that CSV file to reflect the fact that the ranking has improved.

I see, that's how it seems to work in the numpy-array and scope-existing-over-new-import cases indeed. Fixed in the latest commit

@AlexWaygood Yup that's exactly right! More broadly, it may be the case that some rankings can't improve or can't improve without regressing others.

Initial support for handling type_check_only in completions

3318e96

For now we only support classes

decorator-factory changed the title ~~[ty] Consider type_check_only when ranking completions~~ [ty] WIP: Consider type_check_only when ranking completions Oct 16, 2025

AlexWaygood added the ty Multi-file analysis & type inference label Oct 16, 2025

decorator-factory added 2 commits October 17, 2025 00:28

Support type_check_only functions (for the sake of completions)

12865e4

Merge branch 'main' into df-type-check-only

70ff7ad

decorator-factory commented Oct 17, 2025

View reviewed changes

decorator-factory added 2 commits October 17, 2025 00:37

Merge branch 'df-type-check-only' of github.com:decorator-factory-clo…

974b874

…nes/ruff into df-type-check-only

cargo fmt

45c09d6

decorator-factory marked this pull request as ready for review October 17, 2025 00:38

decorator-factory requested review from AlexWaygood, MichaReiser, carljm, dcreager and sharkdp as code owners October 17, 2025 00:38

decorator-factory changed the title ~~[ty] WIP: Consider type_check_only when ranking completions~~ [ty] Consider type_check_only when ranking completions Oct 17, 2025

decorator-factory commented Oct 17, 2025

View reviewed changes

crates/ty_python_semantic/src/types.rs Outdated Show resolved Hide resolved

MichaReiser reviewed Oct 17, 2025

View reviewed changes

crates/ty_python_semantic/src/types/infer/builder.rs Show resolved Hide resolved

MichaReiser added the server Related to the LSP server label Oct 17, 2025

make clippy happy

7b85ec6

AlexWaygood reviewed Oct 17, 2025

View reviewed changes

decorator-factory added 2 commits October 17, 2025 12:42

Add integration test for type_check_only ranking

4928fbe

Assign type-check-only-ness for @type_check_only itself

a3cfa2d

decorator-factory added 3 commits October 17, 2025 13:12

fix completion integration tests

463a815

fix completion integration tests v2

0a0a3ab

Merge branch 'main' into df-type-check-only

ca26e14

decorator-factory requested review from AlexWaygood and MichaReiser October 22, 2025 16:46

Add TODO comment

49da6cd

AlexWaygood approved these changes Oct 23, 2025

View reviewed changes

Uncomment completion integration tests

3646b1b

AlexWaygood merged commit 4ca7459 into astral-sh:main Oct 23, 2025
40 checks passed

[ty] Consider type_check_only when ranking completions #20910

[ty] Consider type_check_only when ranking completions #20910

Uh oh!

Conversation

decorator-factory commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Plan

Uh oh!

github-actions bot commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Diagnostic diff on typing conformance tests

Uh oh!

github-actions bot commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

mypy_primer results

Uh oh!

AlexWaygood commented Oct 16, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MichaReiser left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

AlexWaygood left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

decorator-factory commented Oct 17, 2025

Uh oh!

MichaReiser commented Oct 22, 2025

Uh oh!

decorator-factory commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlexWaygood left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[ty] Consider `type_check_only` when ranking completions #20910

[ty] Consider `type_check_only` when ranking completions #20910

decorator-factory commented Oct 16, 2025 •

edited

Loading

github-actions bot commented Oct 16, 2025 •

edited

Loading

github-actions bot commented Oct 16, 2025 •

edited

Loading

`mypy_primer` results

decorator-factory commented Oct 22, 2025 •

edited

Loading