-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Avoid reported unsoundness for implied lifetime bounds #12471
base: master
Are you sure you want to change the base?
Conversation
Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @Manishearth (or someone else) some time within the next two weeks. Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (
|
r? @oli-obk would you be willing to review this? |
Failed to set assignee to
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this will be too noisy to use, because it's only a problem when the lifetimes are used again in other arguments and return types, and I don't actually know the details of where it will be problematic. Maybe at minimum check that the lifetimes are used again at all
let TyKind::Ref(mut lifetime, mut mut_ty) = ty.kind else { | ||
return implied_bounds; | ||
}; | ||
while let TyKind::Ref(nested_lifetime, nested_mut_ty) = mut_ty.ty.kind { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
these references may be nested behind other types, too (e.g. &'a (&'b i32, u32)
). I'm also not sure whether references are the only things that exhibit this behaviour. There can be struct Foo<'a>(&'a ())
that then get used as &'a Foo<'b>
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also this doesn't handle type aliases. It may be easier to do this analysis on the ty
datastructures instead of on the hir
datastructures. You can get the ty::FnSig
via cx.tcx.fn_sig(local_def_id)
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also this doesn't handle type aliases. It may be easier to do this analysis on the
ty
datastructures instead of on thehir
datastructures. You can get thety::FnSig
viacx.tcx.fn_sig(local_def_id)
.
I'll try and use the ty::FnSig
instead of the things from hir
to collect the implied bounds. In the example &'a (&'b i32, u32)
I see that this will probably need recursion, and implementing that might take a while.
Having more implied bounds could add to the noise...
Adding any number of implicit lifetime bounds is correct (but should not be necessary, see the reverse lint.) I could try check how often this occurs by using cargo dogfood. I suppose that includes the source code of clippy, The unknown problem in this case is the possible unsoundness when an implicit lifetimes bound is added and the compiler does not complain about the new code.
Checking for lifetimes that are not used again at all sounds like another lint, but I may misunderstand this. In the reported pieces of code, after adding an implicit lifetime the compiler gives an error at later use. |
I mean to not emit the lint unless the lifetime appears twice in the signature. As far as I understood it, that is how you can get the unsoundness. Not with a function that's just missing an implied bound on two lifetimes, but additionally has one (or both?) Lifetimes again elsewhere in the same signature. I don't have a strong opinion on whether to land the lint as is or improve it. I think you'd want to first run it on a few crates to see if it is useful in practice, or if it mainly asks to annotate items that don't have any soundness concerns. It's fine to land with lots of missed situations. I just worry about lots of false alarms. |
ctx, | ||
ADD_REDUNDANT_LIFETIMES_BOUND_NESTED_REF_FN, | ||
generics.span, | ||
&format!( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The one thing I think should happen before landing this is to make it a structured suggestion, so people can just rustfix their crates instead of having to manually adjust possibly hundreds of functions
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I hope I can get this to work as a structured suggestion.
@@ -0,0 +1,222 @@ | |||
/// Lints to help dealing with unsoundness due to a compiler bug described here: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Starting a thread to localize this discussion:
My question is about the motivation of including this into Clippy: If Clippy is able to detect those unsoundness issues, shouldn't this be implemented in the compiler and fix the unsoundness bugs?
One explanation for this approach that I see is, that this lint will have false positives (FPs), so implementing it in the compiler would restrict the language too much. Am I right in that assumption?
If this should be the case, putting the lint into a warn-by-default group might be hard. We definitely should run our lintcheck
tool to look for FPs in popular crates.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Starting a thread to localize this discussion:
My question is about the motivation of including this into Clippy: If Clippy is able to detect those unsoundness issues, shouldn't this be implemented in the compiler and fix the unsoundness bugs?
Ideally yes, and I hope the compiler gets fixed before this is ready.
But the 25860 issue was reported in 2015...
One explanation for this approach that I see is, that this lint will have false positives (FPs), so implementing it in the compiler would restrict the language too much. Am I right in that assumption?
An implied bound is correct because it is implied by the compiler input code. Adding an implied bound is also correct, but it adds redundancy in the input code. I think there is no language restriction here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yea, this isn't implemented in rustc, because it would require bounds in a lot of places, while the new solver that can fix this (citation needed) is still about a year out.
I just wonder how many ppl will use this lint. I don't want you to waste time writing a complex lint that then becomes mostly unused
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Manually adding the implied lifetimes bound fixes the three reported cases: the compiler then gives an error message for them. The compiler could also do this addition internally.
/// pub const fn lifetime_translator<'a, 'b: 'a, T>(val_a: &'a &'b (), val_b: &'b T) -> &'a T {...} | ||
/// ``` | ||
#[clippy::version = "1.78.0"] | ||
pub ADD_REDUNDANT_LIFETIMES_BOUND_NESTED_REF_FN, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Those lint names are a mouthful.
Maybe: IMPLICIT_LIFETIME_BOUNDS
here and EXPLICIT_*
for the lint that removes those again?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Those lint names are a mouthful.
Maybe:
IMPLICIT_LIFETIME_BOUNDS
here andEXPLICIT_*
for the lint that removes those again?
From back to front:
The _FN
suffix is there to indicate to the 25860 issue. The idea was to start off with different suffixes for the three issues but merging the lints into the same name makes them easier to use, so I'll drop the suffix.
LIFETIME_BOUNDS
is easier to read by not accurate: the lint gives an error message for a single bound,
and each bound has two lifetimes.
I'll change the lint names to use IMPLICIT
instead of ADD_REDUNDANT
, and similarly for the explicit and remove redundant.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LIFETIME_BOUNDS
is easier to read by not accurate
Please refer to the lint naming conventions, 2. paragraph:
The basic rule is: the lint name should make sense when read as "allow lint-name" or "allow lint-name items". For example, "allow deprecated items" and "allow dead_code" makes sense, while "allow unsafe_block" is ungrammatical (should be plural).
I think you can keep the _FN
suffix to separate those lints, but I'm not convinced that the NESTED_REF
really has to be part of the lint name.
I'll try and do that for the 25860 issue.
I agree this requires careful treading. |
☔ The latest upstream changes (presumably #12451) made this pull request unmergeable. Please resolve the merge conflicts. |
There are merge commits (commits with multiple parents) in your changes. We have a no merge policy so these commits will need to be removed for this pull request to be merged. You can start a rebase with the following commands:
The following commits are merge commits: |
There are merge commits (commits with multiple parents) in your changes. We have a no merge policy so these commits will need to be removed for this pull request to be merged. You can start a rebase with the following commands:
The following commits are merge commits (since this message was last posted): |
Well, after some back and forth with CI/CD I don´t know the root cause of the last error in the Clippy Test / base (pull_request) above. It's in the code docs, so I'll leave it at that for now. The actual code is for issue 25860: I renamed the lints to IMPLICIT_LIFETIMES_BOUND_NESTED_REF and EXPLICIT_LIFETIMES_BOUND_NESTED_REF. The implicit lifetime bounds are now derived from the fn_sig() as suggested. For the rest, I changed the lifetime bound pair struct to use String for simplicity. Not yet done: Check that a lifetime bound has a lifetime that occurs at least twice in the function signature, Make the lints a structured suggestion to allow automatic fixing, both ways. Add test cases to have better coverage for the Ty recursion: tuple, array and slice types. Remaining cases for lints:
How much code documentation is needed? |
The commits today for the other two went faster than I expected. Please include them for review. |
cargo dev dogfood does not pass here, fixing... |
cargo uitest fails nicely on useless_asref.rs line 23: impl<'a, 'b, 'c> AsRef<&'a &'b &'c MoreRef> for MoreRef with two error messages for missing implied bounds I also made a local version with the IMPLICIT* lint in the category suspicious instead of nursery. |
A possible way to assign these lints to categories over longer periods of time, several releases around the compiler fix for the related unsoundness:
The pedantic category fits nicely with the implication here. |
The Clippy Test / base is failing because I missed a cargo uibless. Fixing... |
Rustc was updated to 1.77.0 today, and I'm using it now. |
Running this on the top 500 crates by downloads, triggered the lint 4 times: clippy 0.1.78 (8732557 2024-03-20) Reports
Stats:
I don't have the time to look into those rn. |
... Thank you. The reported bounds with empty lifetime names indicate a bug here.
|
This is most likely the source code for the reported missing bounds (currently at line 1027 in src/lib.rs of the httparse crate):
Since the argument and return type have the same reference structure, I would not expect this to cause unsoundness. Shall I add a condition for the lint here that the implied bounds from input arguments and from the return type are not the same? |
If it can't cause unsoundness issues and your motivated to add a check for it, I'd say sure, go for it. Sidenote: there's a typo in the error message: |
I added the implied bound manually on its current master, and it still compiles and tests cleanly:
|
Ping @flip1995, do you still want to give this PR a review or reroll? |
Let's reroll. I enjoyed the nice weather over the weekend instead and won't have time this week either. r? clippy I really want to give this another look though. But merging shouldn't be blocked by me. |
r? clippy also rerolling, tad busy today |
3f0f0dc
to
bb6d11d
Compare
On today's push: see the 3 commit messages. Only one CI failure is left, and it indicates an unclosed delimiter for the docs of the main function. However I still cannot find the root cause for this failure. |
865401c
to
979051b
Compare
There are merge commits (commits with multiple parents) in your changes. We have a no merge policy so these commits will need to be removed for this pull request to be merged. You can start a rebase with the following commands:
The following commits are merge commits: |
…Use(). Change the comment on GenericBound::Outlives().
979051b
to
ddf9aa9
Compare
Today's push: no functional changes, only updates to follow the master branch. Meanwhile there were some comments on the unsoundness bugs mentioned above. I think the best next step would be to build this into the compiler to fix these bugs. |
changelog: [
lifetimes_bound_nested_ref
]: Avoid some unsoundness for implied lifetime boundsThe lints here are unusual in that they are related to a compiler bug.
This zulip discussion was my starting point:
https://rust-lang.zulipchat.com/#narrow/stream/122651-general/topic/cve-rs
In the discussion, this overview on implied bounds was mentioned:
https://github.com/rust-lang/rustc-dev-guide/blob/478a77a902f64e5128e7164e4e8a3980cfe4b133/src/traits/implied-bounds.md
and these are reported compiler bugs:
rust-lang/rust#25860
Implied bounds on nested references + variance = soundness hole
rust-lang/rust#25860 (comment)
rust-lang/rust#84591
HRTB on subtrait unsoundly provides HTRB on supertrait with weaker implied bounds
rust-lang/rust#100051
Implied bounds from projections in impl header can be unsound
One lint here suggests to add a generic lifetime bound for each function argument and return value
that is a nested reference with lifetimes, rust-lang/rust#25860 .
This generic lifetime bound is implied by this nested reference,
the nested reference lifetime should outlive the initial reference lifetime.
Without the added lifetime bound the current compiler (1.76.0) can generate unsound code,
as reported above.
However when the generic lifetime bound implied by the nested reference is added,
the compiler produces an error message when the function is used.
Therefore this lint can help to avoid the unsoundness in this case.
The other lint here is the reverse of the first one.
It suggests to delete this generic lifetime bound because it is implied.
This lint can help to clean up code, but only after the corresponding unsoundness is fixed
in the compiler.
I have added the reverse lint because of the unusual situation here.
For the other two cases
rust-lang/rust#84591 and
rust-lang/rust#100051 ,
the situation is similar: when manually adding a generic lifetime bound that is implied
by a nested reference, the compiler gives an error instead of producing unsound code.
Pending the review here, I want to continue with similar pairs of lints for these other two cases.
Is it necessary to document the error messages that the compiler gives when
the generic lifetime bounds are added in the reported examples?