Warn when dependency inference does not infer something in your resolve

[Eric-Arellano]

Relates to #14864, but that is specifically for the dependencies field and does not tackle dependency inference.

Two competing approaches:

• Make [python-infer].unownded_dependency_behavior default to warn or error. Then enrich the error message to have "Did you mean?" like NoCompatibleResolveException should suggest remedies #14864 (comment)
• add some new check during dependency inference time and warn or error

[stuhood]

Two competing approaches:

I don't know if they're necessarily competing: it's possible that we should both 1) enable unownded_dependency_behavior at warn, 2) when we warn/error, first do a lookup in other resolves.

[Eric-Arellano]

If all deps belong to other resolve

UnownedDependencyError: Pants cannot infer owners for the following imports in the file src/python/pants/util/strutil_test.py (from the target src/python/pants/util/strutil_test.py:tests):

• fake (line: 6)

These imports are already known by Pants, only are not defined in the same "resolve" of foo used by the target src/python/pants/util/strutil_test.py:tests. Instead, they use the following resolves from these targets:

• fake: 'bar' from path/to:tgt, 'baz' from path/to:tgt

Pants can only infer imports on targets that share the same resolve. You may need to set the resolve= field for some of those targets, and often use the parametrize mechanism so that the same import works with multiple resolves. See doc_url().

Otherwise, try the below ideas.

--

If you do not expect an import to be inferrable, add # pants: no-infer-dep to the import line.

Is the import from a third-party dependency (see https://www.pantsbuild.org/v2.12/docs/python-third-party-dependencies)? Some common issues:

• Pants does not know about the requirement, i.e. there is not a python_requirement target for it. You can run ./pants filter --target-type=python_requirement :: to see all python_requirement targets. If you are using requirements.txt, Poetry, or Pipenv, make sure you have python_requirements, poetry_requirements, and pipenv_requirements target generators, respectively. Double check that the requirement is in the respective file like requirements.txt.
• The third-party requirement exposes modules different than Pants's default, e.g. ansicolors exposing colors. Set the modules or module_mapping fields.
• Ambiguity. See below.

Is the import from first-party code? Some common issues:

• The file does not exist.
• The file is missing an owning target like python_sources. Run ./pants list path/to/file.py to see if there are any owners. If not, run ./pants tailor to generate targets.
• The source root is not correctly set up, which is how Pants knows for example that the file src/py/my_project/app.py exposes the module my_project.app. See https://www.pantsbuild.org/v2.12/docs/source-roots.
• Ambiguity. See below.

Some common issues with both first and third-party imports:

• Ambiguity. See above logs if there are warnings and instructions.

If some deps belong to other resolve

Only change is second paragraph

UnownedDependencyError: Pants cannot infer owners for the following imports in the file src/python/pants/util/strutil_test.py (from the target src/python/pants/util/strutil_test.py:tests):

• fake (line: 6)
• another (line: 7)

Some of these imports are already known by Pants, only are not defined in the same "resolve" of foo used by the target src/python/pants/util/strutil_test.py:tests. Instead, they use the following resolves from these targets:

• fake: 'bar' from path/to:tgt, 'baz' from path/to:tgt

Pants can only infer imports on targets that share the same resolve. You may need to set the resolve= field for some of those targets, and often use the parametrize mechanism so that the same import works with multiple resolves. See doc_url().

Otherwise, try the below ideas.

--

If you do not expect an import to be inferrable, add # pants: no-infer-dep to the import line.

Is the import from a third-party dependency (see https://www.pantsbuild.org/v2.12/docs/python-third-party-dependencies)? Some common issues:

• Pants does not know about the requirement, i.e. there is not a python_requirement target for it. You can run ./pants filter --target-type=python_requirement :: to see all python_requirement targets. If you are using requirements.txt, Poetry, or Pipenv, make sure you have python_requirements, poetry_requirements, and pipenv_requirements target generators, respectively. Double check that the requirement is in the respective file like requirements.txt.
• The third-party requirement exposes modules different than Pants's default, e.g. ansicolors exposing colors. Set the modules or module_mapping fields.
• Ambiguity. See below.

Is the import from first-party code? Some common issues:

• The file does not exist.
• The file is missing an owning target like python_sources. Run ./pants list path/to/file.py to see if there are any owners. If not, run ./pants tailor to generate targets.
• The source root is not correctly set up, which is how Pants knows for example that the file src/py/my_project/app.py exposes the module my_project.app. See https://www.pantsbuild.org/v2.12/docs/source-roots.
• Ambiguity. See below.

Some common issues with both first and third-party imports:

• Ambiguity. See above logs if there are warnings and instructions.

[stuhood]

IMO, we should absolutely lean on this being documented on the docsite, and stop after the high level explanation.

UnownedDependencyError: Pants cannot infer owners for the following imports in the file src/python/pants/util/strutil_test.py (from the target src/python/pants/util/strutil_test.py:tests):

fake (line: 6)

These imports are not in the current resolve for the target ($current-resolve), but they were present in other resolves:

fake: 'bar' from path/to:tgt, 'baz' from path/to:tgt

Pants will only infer imports on targets that share the same resolve. You may need to set the resolve= field for some of those targets, or use the parametrize mechanism so that the same import works with multiple resolves. See doc_url().

...and then most of the explanation to the "multiple resolves" section on https://www.pantsbuild.org/docs/python-third-party-dependencies#multiple-lockfiles

[stuhood]

So, I just tried out adding another resolve to example-python, and it looks like we don't trigger these warnings by default in 2.12 and 2.13. I guess that that's because we never got around to changing the default of [python-infer].unowned_dependency_behavior to warn?

@Eric-Arellano , @thejcannon: It seemed like there was consensus around doing that: I can post a change next week, so that at least folks onboarding to 2.14 see it.

EDIT: #16281