Fix build deps compilation for setuptools < 70.1.0

[chrysle]

See #1681 (comment) for context.

Contributor checklist

• Included tests for the changes.
• PR title is short, clear, and ready to be included in the user-facing changelog.

Maintainer checklist

• Verified one of these labels is present: backwards incompatible, feature, enhancement, deprecation, bug, dependency, docs or skip-changelog as they determine changelog listing.
• Assign the PR to an existing or new milestone for the target version (following Semantic Versioning).

[webknjaz]

This is not a correct fix. It just shoves the problem under the carpet. We need to address the underlying problem, which is revealed not due to pip but because setuptools changed. Yet, the bug is in pip-tools: #2104 (comment). The correct change in the test is to include something like -P 'setuptools < 70.1.0'. But that should be accompanied with a fix for the bug.

[webknjaz]

Alternatively, it's even better to emulate such a situation with artificial package stubs so that it doesn't need to hit the network.

[chrysle]

@webknjaz I reworked the PR; please have a look. Of course there are still the failing Windows tests, can we just drop that horrible Microsoft product ;-)?

[WhyNotHugo]

Slightly changing the order of assertions should help see more details into the errors:

diff --git a/tests/test_cli_compile.py b/tests/test_cli_compile.py
index c5031fc..8db0038 100644
--- a/tests/test_cli_compile.py
+++ b/tests/test_cli_compile.py
@@ -787,13 +787,13 @@ def test_direct_reference_with_extras(runner):
             "pip-tools[testing,coverage] @ git+https://github.com/jazzband/pip-tools@6.2.0"
         )
     out = runner.invoke(cli, ["-n", "--rebuild", "--no-build-isolation"])
-    assert out.exit_code == 0
     assert (
         "pip-tools[coverage,testing] @ git+https://github.com/jazzband/pip-tools@6.2.0"
         in out.stderr
     )
     assert "pytest==" in out.stderr
     assert "pytest-cov==" in out.stderr
+    assert out.exit_code == 0
 
 
 def test_input_file_without_extension(pip_conf, runner):

Generally, I prefer to compare out.stderr before checking the status code in tests; if stderr isn't the expected output, pytest will print it when failing the assertion.

[webknjaz]

Generally, I prefer to compare out.stderr before checking the status code in tests; if stderr isn't the expected output, pytest will print it when failing the assertion.

That won't be necessary if you'll follow the principle of a single assertion per test. Then pytest will print out everything. But personally, I have --showlocals in all configs so that the context is always printed..

[webknjaz]

@chrysle the windows failure is because of the deprecation warning in pip, right?

[chrysle]

Could you elaborate on that? It seems that the resolver only found some pre-versions of a package, only I don't know much because the output is shortened.

[webknjaz]

Yeah, it's weird. Somebody needs to use https://github.com/marketplace/actions/debugging-with-tmate to see what's happening on Windows. The vars in CI seem to suggest that stderr is incomplete or something. Can it be some output buffering specific to windows? Do we need PYTHONUNBUFFERED=1 in tests?

[webknjaz]

Let's not include this hack with more complex logic. This should be better:

Suggested change

	try:
	assert out.exit_code == 0
	assert expected == out.stdout
	except Exception: # pragma: no cover
	print(out.stdout)
	print(out.stderr)
	raise
	assert 0 == out.exit_code and expected == out.stdout, f"\n{out.stdout=}\n\n{out.stderr=}\n\n"

[webknjaz]

FWIW, it'd be better to make it not need internet access. But that's not a blocker, just something to think about for the future.

[webknjaz]

I'd prefer not to include unrelated changes right in this PR. But if you keep this, it's probably better to merge the instructions into a single assert statement. This way, it'd be following the AAA principle to the letter: one test == one assertion.

Suggested change

	assert dependency in out.stderr
	assert out.exit_code == 0
	assert 0 == out.exit_code and dependency in out.stderr

[webknjaz]

@chrysle could you try rebasing?

[webknjaz]

Closed+reopened to re-trigger the CI as the logs got stale and garbage-collected.

[webknjaz]

Of course there are still the failing Windows tests

Interestingly, the “previous” tests pass, even on Windows. But the “latest” crash all over the place because of the relative vs. absolute path differences. “main” fails as well.
“previous” is set to pip==22.2.* currently.

Honestly, I'm leaning towards making the tests more stable by pinning pip in regular push/PR runs and only having it hit newer versions in nightlies...

[webknjaz]

I've converted this to draft to prevent accidental merging ahead of time. I'll undraft it once I'm done experimenting.