Skip to content

Conversation

niacdoial
Copy link
Contributor

@niacdoial niacdoial commented Sep 6, 2025

This is the second PR in an effort to split #134697 (refactor plus overhaul of the ImproperCTypes family of lints) into individually-mergeable parts.

Contains the changes of the first PR, and splits the core type checking function into several bits, each focused on a specific aspect of FFI-safety.
Some logic which was outside of said core function was also moved into the new functions.

Superset of: #146271

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Sep 6, 2025
Copy link
Contributor

@tgross35 tgross35 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a lot of this makes sense, but aren't there behavior changes here? It looks like tuples and arrays may be treated slightly differently.

Which is probably fine, that would ideally just be split from the refactoring and come with test updates.

View changes since this review

Comment on lines 552 to 618
state: VisitorState,
outer_ty: Option<Ty<'tcx>>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like outer_ty is only used for checking where tuples are used, and taking the entire Ty seems a bit heavyweight for that. Could VisitorState instead get a new flag / flags giving the needed context?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in later commits, it is used to detect, er...

  • direct-use of a ty as an argument/return type/static (here)
  • being a pointee (and through which kind of indirection) (here)
  • if a ty::Slice is indeed a slice or if it is actually a !Sized array (here)

That seems like too much state to add into VisitorState. Unless we could forward the discriminant of outer_ty.kind() and use this? We would also need the mutability when said kind is Ref/RawPtr, though.

Also IIRC "an entire Ty" is only as big as a usize, so... actually I have no idea if that's the point you were making when saying it's "heavyweight".

Copy link
Contributor

@tgross35 tgross35 Sep 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not heavyweight as in actual size, I meant the amount of information. Since you're already looking at the outer type once in the loop, it would be it would be cleaner to extract the relevant info to some boolean flags at that time rather than passing the full type and needing to re-analyze it when you recurse. I think the relevant bits for the linked changes could easily enough be flags e.g. the existing FN_RETURN / IN_MUT_REF, IN_ARRAY, IS_UNSIZED_POINTEE.

The other benefit is not dealing with Ty * Ty possibilities, which means fewer unexpected combinations to possibly bug! on.

Maybe just use a flag for this commit? And if needed later, it can be turned into a context struct.

@niacdoial niacdoial force-pushed the improperctypes-refactor2 branch from 1dbb0e2 to f54061c Compare September 6, 2025 20:31
@niacdoial
Copy link
Contributor Author

I think a lot of this makes sense, but aren't there behavior changes here? It looks like tuples and arrays may be treated slightly differently.

Which is probably fine, that would ideally just be split from the refactoring and come with test updates.

I moved the actual change in behaviour in a later commit.
The rest of the changes here are just an exercise in moving more of the type-checking logic into the visit_* methods.

@rust-log-analyzer

This comment has been minimized.

@niacdoial niacdoial force-pushed the improperctypes-refactor2 branch from f54061c to 66037fd Compare September 6, 2025 21:03
@rust-log-analyzer

This comment has been minimized.

@niacdoial niacdoial force-pushed the improperctypes-refactor2 branch from 66037fd to 2781ebd Compare September 6, 2025 22:23
Comment on lines 435 to 538
// (mid-retcon-commit-chain comment:)
// this is the original fallback behavior, which is wrong
if let ty::Adt(def, args) = ty.kind() {
self.visit_struct_or_union(state, ty, *def, args)
} else {
bug!("ImproperCTypes: this retcon commit was badly written")
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this will be committed, turn the comment into a FIXME(ctypes).

Also, make the else else if cfg!(debug_assertions). I don't think this should run into any problems, but also don't want to introduce a new ICE for an intermediate change.

Comment on lines 643 to 752
// I thought CStr (not CString) here could only be reached in non-compiling code:
// - not using an indirection would cause a compile error (this lint *currently* seems to not get triggered on such non-compiling code)
// - and using one would cause the lint to catch on the indirection before reaching its pointee
// but function *pointers* don't seem to have the same no-unsized-parameters requirement to compile
if let Some(sym::cstring_type | sym::cstr_type) =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you rephrase this comment in terms of the actual behavior, or what the FIXME action item is here? I'm not sure whether it's saying the current behavior makes sense, or if the lint isn't accurate here for some reason.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's not describing the behaviour, it's just justifying that "yes, we can also encounter sym::cstr_type in this piece of code.
(I'm removing the comment for now, it's not relevant until indirections get treated differently)

Comment on lines 392 to 507
fn visit_scalar(&self, ty: Ty<'tcx>) -> FfiResult<'tcx> {
// FIXME(ctypes): for now, this is very incomplete, and seems to assume a x86_64 target
match ty.kind() {
// note: before rust 1.77, 128-bit ints were not FFI-safe on x86_64
ty::Int(..) | ty::Uint(..) | ty::Float(..) => FfiResult::FfiSafe,
ty::Bool => FfiResult::FfiSafe,

ty::Char => FfiResult::FfiUnsafe {
ty,
reason: fluent::lint_improper_ctypes_char_reason,
help: Some(fluent::lint_improper_ctypes_char_help),
},
_ => bug!("visit_scalar is to be called with scalar (char, int, float, bool) types"),
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind linking to the changes this helps facilitate?

Thinking about this a bit more; bool is always FFI-safe and char is always FFI unsafe, so there probably isn't any reason not to move them out of this function into the big match.

(sorry for a bit of back and forth here)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two things that visit_scalar would facilitate, but neither are implemented just yet.

  • look at the safety of these things outside of x86_64
  • lint on possibly-broken value assumptions (if you have a reference or a pattern type as the argument of a extern "C" fn for instance). The new visit_pattern logic would then submit the base type to visit_scalar.

Should I put this as a comment in the function?

that being said, yes I can move bool and char out of there.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

look at the safety of these things outside of x86_64

Which platforms is this a problem for? As far as I know, there shouldn't be any more known incompatibilities for scalars - and if there are, we should probably just fix them.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I... huh.
I kinda assumed there would be some and put off looking into them.

I expected there would be similar situations to u128-on-x86_64 where a type is defined in software (CPU registers can't hold it) and part of the standard but handled differently by different compiler/linker/stdlib stacks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m not aware of any remaining problems with the stable types, so I don’t think there is anything to account for here. I.e., visit_scalar can probably remain inlined. Technically there are ABI mismatches for f16 and f128, but we’re working to fix those before stabilization so won’t be linting on them.

where a type is defined in software (CPU registers can't hold it) and part of the standard but handled differently by different compiler/linker/stdlib stacks.

Fwiw whether or not a type fits in registers doesn’t really determine the compatibility level; the ABI provides a spec for what to do (sometimes types that could fit in registers aren’t even passed that way). The problem with x86 i128 is that there the ABI specified what to do and LLVM wasn’t following it. (Not intending to self-promote, but my post on the subject is worth a read if you haven’t seen it https://blog.rust-lang.org/2024/03/30/i128-layout-update/).

There are platforms that don’t specify __int128 in the ABI (like x86-32). We also won’t be linting on that because GCC/Clang don’t let you use __int128 here, so there isn’t anything to be compatible with (bit more in the last paragraph of the PR description at #137306)

Comment on lines 552 to 618
state: VisitorState,
outer_ty: Option<Ty<'tcx>>,
Copy link
Contributor

@tgross35 tgross35 Sep 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not heavyweight as in actual size, I meant the amount of information. Since you're already looking at the outer type once in the loop, it would be it would be cleaner to extract the relevant info to some boolean flags at that time rather than passing the full type and needing to re-analyze it when you recurse. I think the relevant bits for the linked changes could easily enough be flags e.g. the existing FN_RETURN / IN_MUT_REF, IN_ARRAY, IS_UNSIZED_POINTEE.

The other benefit is not dealing with Ty * Ty possibilities, which means fewer unexpected combinations to possibly bug! on.

Maybe just use a flag for this commit? And if needed later, it can be turned into a context struct.

@tgross35
Copy link
Contributor

tgross35 commented Sep 7, 2025

I moved the actual change in behaviour in a later commit.
The rest of the changes here are just an exercise in moving more of the type-checking logic into the visit_* methods.

"split type visiting into subfunctions" still has some changes right? Array went from just checking the type to checking whether or not it is in a function. Which is probably a reasonable change to make, it should just be its own thing (and come with a test update).

(Possible I'm missing something here)

@niacdoial
Copy link
Contributor Author

Array went from just checking the type to checking whether or not it is in a function.

That's a bit of logic that was moved from check_type to visit_type

Comment on lines 691 to 697
match outer_ty.map(|ty| ty.kind()) {
// C functions can return void
None | Some(ty::FnPtr(..)) => state.is_in_function_return(),
// most of those are not even reachable,
// but let's not worry about checking that here
_ => false,
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is is_in_function_return ever set when the outer type isn't FnPtr? Seems like it should be sufficient to check only that without the match.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's the subtlety here:
if you check, say extern "C" fn create_something() -> &():

  • it will start by looking at the &() type, for which outer_ty is None or Some(ty::FnPtr) (approximately) and state.is_in_function_return() is true.
  • Then, if it looks at the pointee, (), state.is_in_function_return() remains true, but outer_ty becomes Some(&()) (approximately).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, is there any reason to track whether or not we are in a function return at anything other than the first level? FN_RETURN could be renamed to DIRECT_FN_RETURN; save it at the top of visit_ty and .remove it from state (so it is unset if we recurse back to visit_ty via indirection), then use that saved value for the tuple check here.

Relates to #146273 (comment).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, is there any reason to track whether or not we are in a function return at anything other than the first level?

Just checked this. In a later commit in this chain, it will be used when dealing with uninhabited types.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(that being said, I'm working on something along the lines of that comment, and I'm realising that what I called "VisitorState" should have been called something else, so that the new thing could get this name.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like there are effectively some flags that only apply once (i.e. die when you call visit_ty on a nested type) and some flags that need to persist. I don't think it would be unreasonable to have both DIRECT_FN_RETURN that gets reset, and WITHIN_FN_RETURN that sticks around for all calls if you need both.

A clear separation may be useful if there are more:

impl VisitorState {
    const NO_RECURSE_FLAGS: Self = Self::DIRECT_FN_RETURN | Self::SOME_OTHER_FLAG;

    /// Reset flags that shouldn't be persisted through recursion, returning them.
    fn no_recurse_flags(&mut self) -> Self {
        let ret = self.intersection(Self::NO_RECURSE_FLAGS);
        self.remove(Self::NO_RECURSE_FLAGS);
        ret
    }
}

fn visit_ty(mut state, ...) {
    let no_recurse_flags = state.no_recurse_flags();

    // use no_recurse_flags.contains(DIRECT_FN_RETURN) for the tuple check
}

@niacdoial niacdoial force-pushed the improperctypes-refactor2 branch 2 times, most recently from efe195a to 69b0807 Compare September 11, 2025 21:56
@rust-log-analyzer

This comment has been minimized.

@niacdoial niacdoial force-pushed the improperctypes-refactor2 branch from 69b0807 to 64106e6 Compare September 11, 2025 22:10
@tgross35 tgross35 self-assigned this Sep 12, 2025
@tgross35
Copy link
Contributor

Btw if these are ready for a more final review, feel free to un-draft them (just gets them actually into my queue)

Another interal change that shouldn't impact rustc users.
The goal is to break apart the gigantic visit_type function into more
managable and easily-editable bits that focus on specific parts of FFI safety.
Another user-transparent change, unifying outer-type information and the
existing VisitorState flags.
@niacdoial niacdoial force-pushed the improperctypes-refactor2 branch from 64106e6 to 359cb79 Compare September 19, 2025 21:15
@niacdoial
Copy link
Contributor Author

just double-checked:
I'm pretty sure I covered all the things you made reviews on
(the one change in this force-push is renaming IndirectionType->IndirectionKind)

@tgross35
Copy link
Contributor

@niacdoial what exactly are these waiting on? I assume they're close to ready based on your above comment, but they are still marked as drafts.

@niacdoial niacdoial marked this pull request as ready for review September 22, 2025 18:38
@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 22, 2025
@niacdoial
Copy link
Contributor Author

niacdoial commented Sep 22, 2025

ah, I knew I was missing something (talking about the PR still being a draft)

Copy link
Contributor

@tgross35 tgross35 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay here, I've been behind for al little while. Should be catching up now, though

View changes since this review

Comment on lines +328 to +331
/// Flags describing both the immediate and overall context in which the current mir::Ty is
persistent_flags: PersistentStateFlags,
/// Flags describing both the immediate and overall context in which the current mir::Ty is
ephemeral_flags: EphemeralStateFlags,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The two doc comments here are the same

Comment on lines +306 to +307
/// To annotate pointees (through Ref,RawPtr,Box).
const IN_PTR = 0b000001;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does IN_ represent here? Same for IN_ADT

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's building on the mental model of "entering" and "exiting" types as they are visited: the flag IN_ADT tells that the currently checked at this depth is a field to the ADT that's "one layer outside".
same with IN_PTR, saying that we are dealing with the pointee of what's "one layer outside".
...though now that you point it out, the flags' names make them sound like they would also be applied recursively when checking "further in", which is not the case.

const NO_OUTER_TY = 0b100000;
/// For NO_OUTER_TY cases, show that we are being directly used by a FnPtr specifically
/// FIXME(ctypes): this is only used for "bad behaviour" reproduced for compatibility's sake
const NOOUT_FNPTR = 0b1000000;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping the NO_ prefix

        const NO_OUT_FNPTR = 0b1000000;


impl EphemeralStateFlags {
/// modify self to change the ephemeral part of the flags
fn from_outer_ty<'tcx>(ty: Ty<'tcx>) -> Self {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs nit: there is no self here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, tastes like docs I forgot to update as I was working on that abstraction.

Comment on lines +382 to +389
debug_assert!(!matches!(current_ty.kind(), ty::FnPtr(..)));
Self {
persistent_flags: self.persistent_flags,
ephemeral_flags: EphemeralStateFlags::from_outer_ty(current_ty),
}
}
fn get_next_in_fnptr<'tcx>(&self, current_ty: Ty<'tcx>, is_ret: bool) -> Self {
debug_assert!(matches!(current_ty.kind(), ty::FnPtr(..)));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These can be real asserts, they should be easy for the compiler to remove (since the enum variant is always checked before calling).

It would be good to mention the difference between these two functions in the docs for get_next as well.

ephemeral_flags: EphemeralStateFlags::from_outer_ty(current_ty),
}
}
fn get_next_in_fnptr<'tcx>(&self, current_ty: Ty<'tcx>, is_ret: bool) -> Self {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional nit: adding an enum rather than using a bool may come in handy in a few places

enum FnPos {
    Arg,
    Ret,
}

Comment on lines +512 to +514
// - for some reason `Box<_>`es in `extern "ABI" {}` blocks
// (including within FnPtr:s)
// are not treated as pointers but as unsafe structs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs nit

Suggested change
// - for some reason `Box<_>`es in `extern "ABI" {}` blocks
// (including within FnPtr:s)
// are not treated as pointers but as unsafe structs
// - for some reason `Box<_>`es in `extern "ABI" {}` blocks
// (including within FnPtr:s)
// are not treated as pointers but as FFI-unsafe structs

Comment on lines +317 to +319
/// To show that there is no outer type, the current type is directly used by a `static`
/// variable or a function/FnPtr
const NO_OUTER_TY = 0b100000;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be the all zeros case, i.e. the default?

Comment on lines +327 to +332
struct VisitorState {
/// Flags describing both the immediate and overall context in which the current mir::Ty is
persistent_flags: PersistentStateFlags,
/// Flags describing both the immediate and overall context in which the current mir::Ty is
ephemeral_flags: EphemeralStateFlags,
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking out loud: there isn't any need to carry ephemeral flags when there isn't a nested type, is there? And then when there is a nested type, it always has an IndirectionKind right?

I'm wondering if it would be better to combine these two, so we don't need to think about states that don't really make sense:

enum Mutability {
    Const,
    Mut,
}

enum Indirection {
    /// No indirection; we are at a root type.
    None,
    ArrayMember,
    Box { mutability: Mutability },
    Ref { mutability: Mutability },
    RawPtr { mutability: Mutability },
}

struct VisitorState {
    persistent_flags: PersistentStateFlags,
    /// Information about the wrapping type, if any.
    indirection: Indirection,
}

Copy link
Contributor Author

@niacdoial niacdoial Oct 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that would be possible, but many of the checks that use VisitorState.indirection would rely on nontrivial match blocks (something like matches!(_, Box{mutability}|Ref{mutability}|RawPtr{mutability} if mutability)).
So, instead of flags plus a lot of consts that resemble an enum, we would have an enum with a lot of checks that resemble flags.

IndirectionKind is not used for every type of nested type, just for pointers.

I see the point of avoiding duplication between the current EphemeralStateFlags and IndirectionKind, but they describe two different things: the former looks at the "outer type" and the latter at the current one.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The usability argument is fair. However, my concern is that we could accidentally wind up with a meaningless state like MEMORY_INLINED | IN_PTR, and this becomes a lot easier to understand if we don't need to think about that.

I'm realizing that a lot of these flags aren't yet consumed: many are set but only NO_OUTER_TY NOOUT_FNPTR are checked against. So I think we can also start a bit smaller here and build up in the other PRs. How about replacing EphemeralStateFlags with:

/// State representing the parent of immediate type, if any.
enum Indirection {
    /// There is no parent type, or the parent type is not relevant.
    // FIXME(ctypes): there are actually more states we need to worry about
    // here, but these are in the process of being added.
    None,
    /// The type is an argument or return type in a function pointer.
    // FIXME(ctypes): this is only used for compatibility, and will be removed with a behavior fix.
    FnPtr,
}

Then we can fill in the rest as needed, we'll be able to add a Pointer { /* pointer-specific flags */ } variant later on.

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 21, 2025
@niacdoial
Copy link
Contributor Author

no worries for the delay!
I even had the time to think about this whole thing in the background, and find a couple of points I'm no longer happy with, in commits that are only in the original "take 2" PR. (:
I'll try adding a more tests to get full coverage of the linting code, too

/// "Permanent state" flags for VisitorState (kept when visiting new mir::Ty)
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
struct VisitorState: u8 {
struct PersistentStateFlags: u16 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you call this RootDefFlags? They are persistent, but I think the hint about what they are referring to is helpful.

Comment on lines +802 to 819
ty::Tuple(tuple) => {
let empty_and_safe = if tuple.is_empty() {
// C functions can return void
state.is_direct_function_return()
} else {
false
};

ty::RawPtr(ty, _) | ty::Ref(_, ty, _)
if {
(state.is_in_defined_function() || state.is_in_fnptr())
&& ty.is_sized(self.cx.tcx, self.cx.typing_env())
} =>
{
FfiSafe
if empty_and_safe {
FfiSafe
} else {
FfiUnsafe {
ty,
reason: fluent::lint_improper_ctypes_tuple_reason,
help: Some(fluent::lint_improper_ctypes_tuple_help),
}
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logic simplification

            ty::Tuple(tuple) => {
                if tuple.is_empty() && state.is_direct_function_return() {
                    // C functions can return void
                    FfiSafe
                } else {
                    FfiUnsafe {
                        ty,
                        reason: fluent::lint_improper_ctypes_tuple_reason,
                        help: Some(fluent::lint_improper_ctypes_tuple_help),
                    }
                }
}

}

/// Get the proper visitor state for a static variable's type
#[inline]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: no need to #[inline], that will happen automatically on crate-private functions

&& self.is_in_function_return()
}

/// Whether the type is directly used in a function.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// Whether the type is directly used in a function.
/// Whether the type is a function argument or return type, without indirection.

Comment on lines -641 to +875
match self.visit_type(VisitorState::ARGUMENT_TY_IN_FNPTR, *arg) {
match self.visit_type(state.get_next_in_fnptr(ty, false), *arg) {
FfiSafe => {}
r => return r,
}
}

let ret_ty = sig.output();
if ret_ty.is_unit() {
return FfiSafe;
}

self.visit_type(VisitorState::RETURN_TY_IN_FNPTR, ret_ty)
self.visit_type(state.get_next_in_fnptr(ty, true), ret_ty)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that for now, this is_unit check should be left here. If you do this, then there is no need to treat arguments and return types any different, so get_next_in_fnptr can just be folded into get_next.

Later on I think it will probably make sense to just treat function pointers exactly the same as function definitions, i.e. call the same entrypoint. But that probably won't make sense until we start doing the behavior changes.

help: if def.is_struct() {
Some(fluent::lint_improper_ctypes_struct_layout_help)
} else {
// FIXME(ctypes): confirm that this makes sense for unions once #60405 / RFC2645 stabilises
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// FIXME(ctypes): confirm that this makes sense for unions once #60405 / RFC2645 stabilises
// FIXME(#60405): confirm that this makes sense for unions once #60405 / RFC2645 stabilises

Since this will be fixed by something other than the ctypes refactoring

Comment on lines +327 to +332
struct VisitorState {
/// Flags describing both the immediate and overall context in which the current mir::Ty is
persistent_flags: PersistentStateFlags,
/// Flags describing both the immediate and overall context in which the current mir::Ty is
ephemeral_flags: EphemeralStateFlags,
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The usability argument is fair. However, my concern is that we could accidentally wind up with a meaningless state like MEMORY_INLINED | IN_PTR, and this becomes a lot easier to understand if we don't need to think about that.

I'm realizing that a lot of these flags aren't yet consumed: many are set but only NO_OUTER_TY NOOUT_FNPTR are checked against. So I think we can also start a bit smaller here and build up in the other PRs. How about replacing EphemeralStateFlags with:

/// State representing the parent of immediate type, if any.
enum Indirection {
    /// There is no parent type, or the parent type is not relevant.
    // FIXME(ctypes): there are actually more states we need to worry about
    // here, but these are in the process of being added.
    None,
    /// The type is an argument or return type in a function pointer.
    // FIXME(ctypes): this is only used for compatibility, and will be removed with a behavior fix.
    FnPtr,
}

Then we can fill in the rest as needed, we'll be able to add a Pointer { /* pointer-specific flags */ } variant later on.

@tgross35
Copy link
Contributor

I just stumbled upon #66220, that thread is probably worth a read. I think I understand the difference between the two lints better now:

  • improper_ctypes is the original and is more strict, it checks the types used even through indirection
  • improper_ctypes_definitions allows non-FFI-safe types if protected via indirection

Which I think explains some of the current behavior a bit better. Eventually I wonder if it might make sense to eventually add a variant to the result type, so

enum FfiResult<'tcx> {
    FfiSafe,
    FfiPhantom(Ty<'tcx>),
    /// Contains FFI-unsafe types, made okay to use in a signature by a pointer type.
    SafeViaIndirection { ty: Ty<'tcx>, reason: DiagMessage, help: Option<DiagMessage> },
    /// FFI-unsafe
    FfiUnsafe { ty: Ty<'tcx>, reason: DiagMessage, help: Option<DiagMessage> },
}

Sort of as part of what is suggested in #72774. But that would be something for later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants