[internal] Add generate_all_lockfiles.py to correctly generate default tool lockfiles #12418
Conversation
Eric-Arellano
force-pushed
the
gen-lockfiles-script
branch
2 times, most recently
from
533564f to
6da9162
Compare
…ult tool lockfiles # Rust tests and lints will be skipped. Delete if not intended. [ci skip-rust] # Building wheels and fs_util will be skipped. Delete if not intended. [ci skip-build-wheels]
Eric-Arellano
force-pushed
the
gen-lockfiles-script
branch
from
6da9162 to
b61c07a
Compare
[ci skip-rust] [ci skip-build-wheels] [ci skip-rust] [ci skip-build-wheels]
Eric-Arellano
changed the title
There was a problem hiding this comment.
Thanks!
Will #12418 be the first thing that enforces that these files haven't gone out of sync? Or do we already have a version check that would fail when someone bumped the default_version in this repo without changing the lockfile?
| @@ -72,9 +72,10 @@ class PythonProtobufMypyPlugin(PythonToolRequirementsBase): | |||
| "pants.backend.codegen.protobuf.python", | |||
| "mypy_protobuf_lockfile.txt", | |||
| ) | |||
| default_lockfile_url = git_url( | |||
| default_lockfile_path = ( | |||
There was a problem hiding this comment.
I guess that the default_lockfile_path is intentionally undocumented / not part of the interface?
There was a problem hiding this comment.
Yeah, it's not used by any of our production code, only to facilitate this script. I don't want to require that plugins implement it because they might not need it.
Oh, tricky. Chris's work on #12415 is what will validate lockfiles are not stale for our own internal usage. We still need to validate though that the default lockfiles we're providing are not stale..I think the way to do that is to add an integration test that runs each of the tools using Before that lands, we'll have to rely on code review to ensure this is not stale. |
…-script [ci skip-build-wheels]
If the staleness check works by comparing a hash of inputs to a hash encoded inside the lockfile, then it should work for lockfiles loaded as resources just as well as it should for those loaded via options... seems like it should work...? |
Ah to clarify, the issue is that CI never uses the |
The requirements we use for pantsbuild/pants might be different than what makes sense for the default we provide to our users. For example, we currently use two Flake8 plugins that should not be in the default lockfile.
This script allows us to be confident that the default lockfiles we generate are entirely decoupled from our own internal settings.