Treat imports under "if TYPE_CHECKING" as weak.

[benjyw]

It's not an error if they're not available at runtime (hence the
TYPE_CHECKING test), and at typecheck time they can be provided by
[mypy].extra_type_stubs.

[ci skip-rust]

[ci skip-build-wheels]

[benjyw]

Please weigh in on whether you agree that this is even correct.

[kaos]

Seems correct to me.

[thejcannon]

I'm not sure this is correct. At least without a more concrete example.

My interpretation of extra_type_stubs is precisely that, extra type stubs. Although we could put additional typecheck-only packages in there, they aren't extra type stubs and so belong with normal deps. Otherwise extra_type_stubs should just contain stubs for packages which normal deps.

If your concrete example shows we might need something like this, I suggest we consider renaming extra_type_stubs.

[thejcannon]

(also I'd call this a bugfix for label purposes)

[Eric-Arellano]

I think this is correct. We have two different ways of adding type stubs, based on user feedback:

1. The more accurate modeling of using a python_requirement target and type_stub_modules field.
2. Using [mypy].extra_type_stubs, which is a convenience.

If we did not have #2, then I agree we should not land this PR.

@benjyw can you please update the PR description and comments to explain this nuance?

[Eric-Arellano]

can you please update the PR description and comments to explain this nuance?

Also probably the help for unowned_dependency_behavior, which mentions the try: except exception already.

[thejcannon]

Discussed with @Eric-Arellano offline. They helped alleviate my confusion here.

Essentially, you aren't importing the type stub, you are importing the package. However it is just for type hints sake. I am a bit worried that we still have a gap in our dependency resolution. Consider the following:

if TYPE_CHECKING:
    from alibaba import BaseballBall

def swing_bat_at(ball: BaseballBall):
    ...

If I run ./pants check --only=mypy myfile.py the env it runs in may not contain alibaba, and therefore mypy might not be able to do type checking. That's no different before/after this change. But if I bumped unowned_dependency_behavior to error I'd hope Pants gave me a nudge here.

So I'm not sure I'm still 100% convinced we should do this, but I see more or the use-case. @benjyw can you share the real-world case of this cropping up?

[Eric-Arellano]

If I run ./pants check --only=mypy myfile.py the env it runs in may not contain alibaba, and therefore mypy might not be able to do type checking. That's no different before/after this change. But if I bumped unowned_dependency_behavior to error I'd hope Pants gave me a nudge here.

That is all true. But what is unfortunate is the user might be intending to provide that type hint via [mypy].extra_type_stubs (when it's 3rd-party dep), rather than a python_requirement target. In that scenario, we would never be able to infer a dep, even though it's valid what they're doing.

[benjyw]

AFAICT the BaseballBall scenario will fail at runtime anyway, since there is no import unless TYPE_CHECKING is set? I mean fail at the Python level, nothing to do with Pants. I see, you're only using symbols from that package as a type annotation.

[thejcannon]

So there's two sides to this coin, and I think it's a matter of "which side do we want to support the most"

1. You've added a package to extra_type_stubs, but ARE NOT using the package anywhere. You're only trying to type-check that "yes our code looks like it works with library X", but you never actually use it. This PR makes it so we will no longer error on the import of the package itself.
2. You've added a package to extra_type_stubs, and have forgotten to add the untyped prod package to your requirements. This PR makes it so now mypy might error if configured, or might just be silent. I hope you test your code 😉

Given that case 1 is niche and that the user can specify no-infer-dep, I'm worried we've made life worse for the user in case 2 with no real gain.

[Eric-Arellano]

Another framing of what @thejcannon is saying. We are only giving false positives in this case right now:

• type hints might be for 1st-party and 3rd-party code. Let's just focus on 3rd-party.
• 3rd-party type hints can either come directly from the implementation (e.g. pantsbuild.pants), or from separate stubs packages (types-requests). Let's focus on stubs
• With Pants, you can teach it about stubs for 3rd-party deps in two ways: 1) a normal python_requirement which works w/ dep inference, or 2) [mypy].extra_type_stubs
• When you use [mypy].extra_type_stubs, those are not added to the "universe" / module mapping. They don't participate in dep inference
• It is possible to have use type hints for 3rd-party code w/o actually directly importing the thing at runtime!

if TYPE_CHECKING:
   from pkg_resources import Requirement

def extract_version(req: Requirement) -> str:
    return req.version

In all other cases, I think it is correct to error. Even if you are using [mypy].extra_type_stubs, you usually will want to error if the production python_requirement is not inferrable, i.e. setuptools vs types-setuptools.

--

Given the escape hatch with # pants: no-infer-dep, I no longer am +1 on this change. I think that overall you do want dependencies inferred for if TYPE_CHECKING, and it is a better experience to get the error from Pants rather than from MyPy.

[benjyw]

What I'm grasping towards, weakly, is the following observation, using your example above: If alibaba is present in your lockfile we will (afaict) infer a runtime dep on it even though one is not needed at runtime. So alibaba ends up in your PEX, or in your docker image, or in your wheel requirements. This PR doesn't change that.

But, if alibaba is not present in your lockfile, and we set unowned_dependency_behavior = "error", then the build fails on the unprovided dep. So now we're forced to add it at runtime (or manually remove it via negation in dependencies=) just because of typechecking. I.e., no good deed goes unpunished...

[thejcannon]

But, if alibaba is not present in your lockfile, and we set unowned_dependency_behavior = "error", then the build fails on the unprovided dep. So now we're forced to add it at runtime (or manually remove it via negation in dependencies=) just because of typechecking. I.e., no good deed goes unpunished...

That's what # pants: no-infer-dep is for!

[Eric-Arellano]

Bump on #15334, which will make it more clear what to do when this happens.

[benjyw]

When adopting Pants in a new repo, having to go to every if TYPE_CHECKING location and add a bunch of # pants: no-infer-dep is pretty onerous. That's the tool getting in the way.

[benjyw]

I guess this goes back to dependency scopes... #12794

[Eric-Arellano]

When adopting Pants in a new repo, having to go to every if TYPE_CHECKING location and add a bunch of # pants: no-infer-dep is pretty onerous. That's the tool getting in the way.

@thejcannon and my point is that this should not be the case. Most the time you will have a corresponding implementation to go along with it. If you are using type hints for pkg_resources, it is likely that you are both using types-setuptools and setuptools. So, Pants will be happy to find setuptools

[benjyw]

I will close this PR, as it seems to be a wider issue, totally unrelated to TYPE_CHECKING.

For example, I'm seeing issues where a test has import pytest and the default resolve doesn't provide pytest, so we get the warning/error.

This is tricky because pytest is a "provided" dependency - Pants ensures that it's available at test time regardless of your deps, because it's a tool to run, but it is also a code dependency.

What would be the best practice in cases like this? Add pytest to the default-python resolve, even though it also has its own resolve?

[Eric-Arellano]

What would be the best practice in cases like this? Add pytest to the default-python resolve, even though it also has its own resolve?

#12449, which I started doing prework for in #15186. Getting that green is gonna take a bit more work. I'm happy to pick it up again, it's a very awkward edge

[thejcannon]

FWIW I was originally +1 as well. This is admittedly complex if 4 maintainers' initial reaction was incorrect.

To stretch this further. I'd happily approve a PR which adds a test ensuring TYPE_CHECKING imports aren't weak, and we can put a nice comment explaining why.
That way the next person doesn't have to have this same mental journey

[jsirois]

Isn't this just the very old problem of having 1 dependencies field? dependencies can mean different things in different contexts, but we cram all those contexts into 1 field, which can be a lie. Those contexts are rules, and some rules will have a different notion of dependencies than others. A rule trying to run pytest will not care about dependencies only needed when typechecking, a rule trying to generate a sphinx doc site will care about a different augmented set of dependencies that may mix in css, say, above and beyond symbols that might be referenced in the docs that requires "normal" dependencies to be involved.

[thejcannon]

@jsirois it is. However in the 80% case one field is "good enough" as the deps for type checking are the same for running are the same for testing. However, that leads us here 😅

[jsirois]

And cramming is deemed good because it makes the 80% somehow alot easier / more ergonomic than it would otherwise be possible to do?

[kaos]

Interesting discussions here, and I agree this is a sign of a wider issue. John is onto something important as well.

I have a real case where I had issues with extras for a dependency. It is a single (util) library, and for better or worse, it holds both runtime and test time utility functions. For my tests, I'd like to include the test utils from that library, but I don't want that extra to be included for my app when packaging it up, then it should only include the extras for the runtime utils.

I didn't spend enough time on this particular issue then to really figure out how to solve this, but I feel it is related to this discussion here, as input use case if nothing else.

[benjyw]

Yeah, TYPE_CHECKING seems like a good example of a scoped dependency.

[thejcannon]

Just adding to this sideband discussion, "built" targets are a candidate for 2 dependency classes: build-time and run-time.