Propagate global interpreter constraints when building PEXes with interpreter constraints requested

[Eric-Arellano]

Problem

When running ./pants binary, we do not consider the global interpreter constraints in passing them to the Pex builder.

This resulted in a bug where the interpreter used to build a PEX in CI differed from the Python used to run the PEX. See pex-tool/pex#676. To resolve this issue, we need some way to specify the Pex builder must use the specific Python version 2.7.13 (UCS4). This blocks
#7235.

Solution

Modify PexBuilderWrapper to use compatibility_or_constrains(). This will first try to get the compatibility constraints from the target itself, and then resort to using the PythonSetup subsystem's constraints (for the global instance, resolved from pants.ini, PANTS_PYTHON_SETUP_INTERPRETER_CONSTRAINTS, and --interpreter-constraints).

Result

Users of the function add_interpreter_constraints_from() will now add the global --interpreter-constraints to the Pex builder if the targets do not themselves specify a constraint. At the moment, this only impacts ./pants binary, as python_binary_create.py is the only file which uses this function.

Note this is still not a great solution. It would be better to kill add_interpreter_constraint() and add_interpreter_constraints_from() to instead automatically set the interpreter constraints from the targets' graph. This PR does not make that change to avoid scope creep.

Skipped tests

Due to pex-tool/pex#655, we must now skip several tests that now fail. PEX does not correctly OR interpreter constraints, so these tests complain that they cannot find an appropriate interpreter to satisfy ['CPython>=3.6,<4', 'CPython>=2.7,<3'].

Because this PR blocks #7235, which blocks #7197, we skip these tests until the upstream fix pex-tool/pex#678 is merged into PEX and Pants upgrades its PEX to the new version #7186.

[jsirois]

We have a mess here. The add_interpreter_constraints_from method should probably be using

pants/src/python/pants/backend/python/subsystems/python_setup.py

Lines 146 to 153 in 26b0179

	def compatibility_or_constraints(self, target):
	"""
	Return either the compatibility of the given target, or the interpreter constraints.
	If interpreter constraints are supplied by the CLI flag, return those only.
	"""
	if self.get_options().is_flagged('interpreter_constraints'):
	return tuple(self.interpreter_constraints)
	return tuple(target.compatibility or self.interpreter_constraints)

Bandaids are piled on bandaids it seems.

[stuhood]

This makes sense to me, but would be good to have Chris confirm before landing. I'll ping him.

[Eric-Arellano]

I realized the scope is less significant than I had suggested with the original title and PR description. While we are indeed changing an API function for PexBuilderWrapper, at the moment the only user of add_interpreter_constraints_from() is python_binary_create.py. So the only command that should (theoretically) be impacted by this change is ./pants binary.

As shown in CI with the failing test_pytest_run_integration.py, however, this one change seems to have greater implications even though it does only touch ./pants binary.

[CMLivingston]

LGTM, thanks for catching this and fixing.

[Eric-Arellano]

Reviewers: @cosmicexplorer pointed out the test failures were resulting from pex-tool/pex#655. We now set interpreter constraints for the generated ./pants.pex, so when using the PEX in our CI are running into PEX's issue with not correctly ORing requirements, resulting in a failure to find an appropriate interpreter (example).

We must decide if it is acceptable to skip the failing tests or if we must instead fix upstream first. I would strongly prefer to allow the temporary skip, as this PR is blocking #7235, which itself blocks releasing Python 3 wheels (#7197). However, I want to be careful to not make a rash decision.

[stuhood]

Thanks. Given that the fix is already posted and green upstream, I'm fine with merging this with these tests skipped.