-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allows the default git/gitoxide configuration to be obtained from the ENV and config #13687
Conversation
I assume this is meant to resolve the immediate part of #13688 (leaving aside the more general problem). Is that right? |
src/cargo/core/features.rs
Outdated
@@ -908,15 +914,19 @@ fn parse_git(it: impl Iterator<Item = impl AsRef<str>>) -> CargoResult<Option<Gi | |||
#[derive(Debug, Copy, Clone, Default, Deserialize)] | |||
pub struct GitoxideFeatures { | |||
/// All fetches are done with `gitoxide`, which includes git dependencies as well as the crates index. | |||
#[serde(default = "default_true")] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume these were chosen to align with GitoxideFeatures::safe()
and to mirror if you do -Zgitoxide
However, this breaks down if you do CARGO_UNSTABLE_GITOXIDE_FETCH=true
, then almost everything else becomes enabled (we should have a test to show this case).
Some other challenges with these defaults
- Unclear what the motivation is (could have comment pointing to
GitoxideFeature::safe
) - Challenge in keeping in sync with
GitoxideFeature::safe
Should we instead just default everything to false?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What I would love to just work is CARGO_UNSTABLE_GITOXIDE=fetch,list_files
and CARGO_UNSTABLE_GITOXIDE=true
mirroring the CLI flag (which would also imply you could have unstable.gitoxide = "fetch,list_files"
in the config file).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah maybe we can make it a comma separated list to reduce the complexity.
Wonder do people really use individual gitoxide features? I always assume that people would just enable them as a whole, though gitoxide author recommends keeping them separately: #13252 (comment).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What I would love to just work is CARGO_UNSTABLE_GITOXIDE=fetch,list_files and CARGO_UNSTABLE_GITOXIDE=true mirroring the CLI flag (which would also imply you could have unstable.gitoxide = "fetch,list_files" in the config file).
So what we defined as table is going to be replaced with a string? It makes sense, I have no objection to this, but it is not clear that it will break the configuration of the previous version?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's possible to support both, via #[serde(deserialize_with = ...)]
and a custom deserializer that handles a string/list or a struct, like
cargo/src/cargo/util/context/mod.rs
Line 2673 in 88d55a8
fn progress_or_string<'de, D>(deserializer: D) -> Result<Option<ProgressConfig>, D::Error> |
☔ The latest upstream changes (presumably #13696) made this pull request unmergeable. Please resolve the merge conflicts. |
r? weihanglo |
e852649
to
b5fd826
Compare
This comment was marked as outdated.
This comment was marked as outdated.
Yes,this tend to mirror the -Zgit/-Zgitoxide as possible. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for the late review 🙇🏾♂️
src/cargo/util/context/de.rs
Outdated
// then run `CARGO_UNSTABLE_GIT_SHALLOW_INDEX cargo fetch`, here will return `missing config key unstable.git`. | ||
// It seems that anything that starts with CARGO_UNSTABLE_GIT, even like CARGO_UNSTABLE_GITOXIDE can trigger this. | ||
// This is a workaround for now, but should be fixed in the future. | ||
visitor.visit_none() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While this seems fine for now, I am a bit worried about the change.
What about having an enum deriving serde?
#[derive(Debug, Copy, Clone, Default, Deserialize)]
#[serde(untagged)]
pub enum Gitoxide {
#[default]
All,
Some(GitoxideFeatures),
}
so that I assumed it would work well with
[unstable]
gitoxide = true
# or
[unstable.gitoxide]
checkout = true
As well as enviroment variables?
CARGO_UNSTABLE_GITOXIDE=true
CARGO_UNSTABLE_GITOXIDE_CHECKOUT=true
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It doesn't seem to be working. see gitoxide_features_as_table
in the lastest commit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had replaced vistor.visit_none()
with below,
// The effect here is the same as in [`deserialize_option`].
if self.gctx.has_key(&self.key, self.env_prefix_ok)? {
return visitor.visit_some(self);
}
So we can still specify a inner field of a struct in the ENV, like CARGO_UNSTABLE_GITOXIDE_FETCH
, even that CARGO_UNSTABLE_GIOXIDE
doesn't exist.
But that's not enough, It will still warning unstable.git
missing key, Why is this happening? After some groping, Problems gradually surfaced, see https://github.com/rust-lang/cargo/blob/master/src/cargo/util/context/de.rs#L268-L269
if env_key.starts_with(field_key.as_env_key()) {
fields.insert(KeyKind::Normal(field.to_string()));
here will treat CARGO_UNSTABLE_GIT
as prefix of CARGO_UNSTABLE_GIOXIDE_FETCH
, and the unstable.git
and unsable.gitoxide
will be pushed into fields. This is a problem, when the unstable fields is a struct and the fields implement their deserialize_any
method. So any environment variable that starts with the XX character will have missing XX keys.
See test struct_with_overlapping_inner_struct_and_defaults
for more details.
76421f7
to
b4f699e
Compare
☔ The latest upstream changes (presumably #13956) made this pull request unmergeable. Please resolve the merge conflicts. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks reasonable. Just some minor suggestions.
Fix: Skip deserialization of unrelated fields with overlapping name ### What does this PR try to resolve? Split from #13687 (comment) This fixes the overlap of environment variable names: > For example, when env_key is UNSTABLE_GITOXIDE_FETCH and field_key is UNSTABLE_GIT, the field shouldn't be added because `unstable.gitoxide.fetch` doesn't belong to `unstable.git` struct. ### How should we test and review this PR? Updates of test cases `struct_with_overlapping_inner_struct_and_defaults` can be used to compare changes before and after changes ### Additional information r? weihanglo and very appreciate your more optimized code
b4f699e
to
dfc8e24
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One minor error message regression left and we're ready to go
src/cargo/core/features.rs
Outdated
fn expecting() -> String { | ||
let fields = vec!["`fetch`", "`checkout`", "`internal-use-git2`"]; | ||
format!( | ||
"unstable 'gitoxide' only takes {} as valid inputs, for shallow fetches see `shallow-index,shallow-deps`", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"unstable 'gitoxide' only takes {} as valid inputs, for shallow fetches see `shallow-index,shallow-deps`", | |
"unstable 'gitoxide' only takes {} as valid inputs, for shallow fetches see `-Zgit=shallow-index,shallow-deps`", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't it be -Zgitoxide
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like the old tip was wrong, too. See https://doc.rust-lang.org/cargo/reference/unstable.html, let me correct it to -Zgitoxide
also
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, I misunderstand
dfc8e24
to
22ef8cb
Compare
- pass true to enable all pre-definded git/gitoxide features - support parse git/gitoxide as table in Config, if the field is tagged with #[serde(default)], then it can be skipped
22ef8cb
to
ee87c91
Compare
Thanks! @bors r+ |
☀️ Test successful - checks-actions |
Update cargo 9 commits in 7a6fad0984d28c8330974636972aa296b67c4513..34a6a87d8a2330d8c9d578f927489689328a652d 2024-05-31 22:26:03 +0000 to 2024-06-04 15:31:01 +0000 - Silence the warning about forgetting the vendoring (rust-lang/cargo#13886) - fix(vendor): Ensure sort happens for vendor (rust-lang/cargo#14004) - fix(add): Avoid escaping double-quotes by using string literals (rust-lang/cargo#14006) - refactor(source): Split `RecursivePathSource` out of `PathSource` (rust-lang/cargo#13993) - doc: Add README for resolver-tests (rust-lang/cargo#13977) - Allows the default git/gitoxide configuration to be obtained from the ENV and config (rust-lang/cargo#13687) - refactor: Transition direct assertions from cargo-test-support to snapbox (rust-lang/cargo#13980) - Fix: Skip deserialization of unrelated fields with overlapping name (rust-lang/cargo#14000) - chore(deps): update alpine docker tag to v3.20 (rust-lang/cargo#13996) r? ghost
Update cargo 9 commits in 7a6fad0984d28c8330974636972aa296b67c4513..34a6a87d8a2330d8c9d578f927489689328a652d 2024-05-31 22:26:03 +0000 to 2024-06-04 15:31:01 +0000 - Silence the warning about forgetting the vendoring (rust-lang/cargo#13886) - fix(vendor): Ensure sort happens for vendor (rust-lang/cargo#14004) - fix(add): Avoid escaping double-quotes by using string literals (rust-lang/cargo#14006) - refactor(source): Split `RecursivePathSource` out of `PathSource` (rust-lang/cargo#13993) - doc: Add README for resolver-tests (rust-lang/cargo#13977) - Allows the default git/gitoxide configuration to be obtained from the ENV and config (rust-lang/cargo#13687) - refactor: Transition direct assertions from cargo-test-support to snapbox (rust-lang/cargo#13980) - Fix: Skip deserialization of unrelated fields with overlapping name (rust-lang/cargo#14000) - chore(deps): update alpine docker tag to v3.20 (rust-lang/cargo#13996) r? ghost
What does this PR try to resolve?
The default git/gitoxide feautures config can be obtained througt
-Zgit
and-Zgitoxide
.However, it cannot be obtained from environment variables or configurations.
It's not very ergonomic.
How should we test and review this PR?
The previous commit explained the staus quo, the next commit addressed the problem.
Additional information
Inspired by #11813 (comment)
See also #13285 (comment)
Change of usage
Specify the feature you like
The rest fields will use Rust default value. That said, checkout and internal_use_git2 will be false.
Besides, you can pass true to the whole feature to specify the pre-defined features.