Unable to guarantee consistent versions for (transitive) dependencies between tools (particularly mypy) and runtime resolves

[huonw]

Describe the bug

We've got a codebase that uses mypy extensively, and various plugins, and we'd prefer if we had good methods for having the versions of our mypy plugin libraries match the ones used at runtime.

For instance:

# pyproject.toml
[tool.mypy]
plugins = ["pydantic.mypy", "sqlmypy", "strawberry.ext.mypy_plugin"]
...

Two of these come from libraries also used at runtime (pydantic and strawberry-graphql) and thus it'd be good to be using a consistent version globally: if we have mypy using plugins for different versions of our runtime libraries who knows if type checking passing is meaningful.

There's a few options:

• write extra_requirements = ["pydantic==...", ...] etc. but that duplicates the version configuration, and doesn't guarantee identical versions of transitive dependencies without listing them all out by hand.
• use source_plugins pointing to the relevant python_requirement target (which isn't what it's documented to be used for), which at least deduplicates version configuration for those direct requirements, but also doesn't guarantee identical versions of transitive dependencies.

The second is slightly better than the first, so that's what we've tried so far despite it seemingly being not a proper use of source_plugins. For instance:

[GLOBAL]
pants_version = "2.12.0"

backend_packages = [
  "pants.backend.python",
  "pants.backend.python.typecheck.mypy",
]

[python]
interpreter_constraints = ["CPython==3.9.*"]
enable_resolves = true

[mypy]
lockfile = "3rdparty/python/mypy.lock"
source_plugins = [
  "//:pydantic",
]

[anonymous-telemetry]
enabled = false

With top-level BUILD that simulates a transitive dependency typing-extensions that's stuck on an older version:

python_requirement(name="pydantic", requirements=["pydantic==1.9.0"])
python_requirement(name="typing-extensions", requirements=["typing-extensions==4.0.0"])

This mostly works, after running ./pants generate-lockfiles, we have mypy.lock containing:

...
          "project_name": "pydantic",
          "version": "1.9"
...
          "project_name": "typing-extensions",
          "version": "4.3"
...
  "requirements": [
    "mypy==0.910",
    "pydantic==1.9.0"
  ],
...

However, note that typing-extensions is missing from the requirements, and indeed the pinned version of the transitive dependency of pydantic is different to the one in the runtime default resolve (which is just "version": "4").

Notes:

• For this particular typing-extensions dependency, this probably doesn't matter so much, but other transitive dependencies of some plugins may make a difference if the version is significantly different (e.g. hypothetically a plugin that validates that a particular string literal with strict formatting requirements (e.g. "the arg is a valid regex" in a call like re.compile("...")), in addition to the basic runtime check: if the plugin uses a newer version that might support more features, mypy will incorrectly be successful).
• This potentially relates to general issues about ./pants generate-lockfiles always choosing the latest version (if there's been new releases), such as New goal for managing 3rd party dependencies #12880, Add ability to extend a PEX lockfile without modifying existing entries #15704 and Add ability to check if lockfiles are up-to-date with all declared requirements #15723.
• This also potentially relates to Allow [python_tool].version and [python-tool].extra_requirements to point to python_requirement targets? #12449.
• Maybe we want to be saying that mypy can pull plugins from a particular resolve, but that's potentially weird (e.g. the docs for source_plugins call out using a separate resolve for deps of such plugins https://www.pantsbuild.org/docs/reference-mypy#section-source-plugins)

As always, thanks for pants!

Pants version
2.12.0

OS
macOS

Additional info

Reproducing example: https://github.com/huonw/pants-tool-pinning

[benjyw]

This is addressed in a different way by #18481, which allows a tool to be installed from a user resolve. In fact, the version/extra_requirements options are now deprecated, and we want there to just be one kind of lockfile, that can be reused for tools as well as user code.