boto3-stubs being used as dependency when boto3 is imported

[engnatha]

Describe the bug
I recently added boto3-stubs to my project to make interactions with boto3.client objects easier in my IDE. This broke the dependency inference for my modules that imported boto3. When I run ./pants dependencies path/to/my_module.py, I am now seeing //:boto3-stubs and //:boto3 is absent.

I can remedy this by manually adding //:boto3 as a dependency for my module, but this seems like it should not be necessary. I also tried editing module_mapping at the top level, but this had no effect. I did find some mention of type stub behavior in the docs here, but that seemed useful for picking up type stubs rather than removing them.

Lastly, an observation from this is that it seems like adding type stub libraries will make entities packaged by Pants heavier. The type stubs are only necessary for linting and, in my opinion, should never be included as a dependency unless a goal specifically needs them.

Pants version
2.9.0

OS
Ubuntu 20.04

[Eric-Arellano]

Sorry about this! Thank you for filing. I'm aiming to take this one on this week or early next week.

[Eric-Arellano]

As discussed in Slack, I think the issue is that the requirements.txt had this line:

boto3-stubs>=1.21.0,<2
boto3-stubs[s3,ecr,sts]

This results in two distinct python_requirement targets being created for boto3-stubs. Pants doesn't like that ambiguity, so it stops inferring the dependency.

Instead, the solution is to combine those two entries into boto3-stubs[s3,ecr,sts]>=1.21.0,<2.

@engnatha please feel free to reopen if this does not solve this.

--

@huonw and @achimnol, are you experiencing similar issues? If so, can you share more info please, like your requirements.txt files?

--

Lastly, an observation from this is that it seems like adding type stub libraries will make entities packaged by Pants heavier. The type stubs are only necessary for linting and, in my opinion, should never be included as a dependency unless a goal specifically needs them.

Agreed. See #15454.

[huonw]

We're encountering #15454, I think, so nothing more to add to this specific issue.