Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experimentally add ffi_const and ffi_pure extern fn attributes #58327

Closed
wants to merge 11 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions src/doc/unstable-book/src/language-features/c-ffi-const.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# `c_ffi_const`

The `#[c_ffi_const]` attribute applies clang's `const` attribute to foreign
functions declarations.

That is, `#[c_ffi_const]` functions shall have no effects except for its return
value, which can only depend on the values of the function parameters, and is
not affected by changes to the observable state of the program.

The behavior of calling a `#[c_ffi_const]` function that violates these
requirements is undefined.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be UB to even have a wrongly annotated declaration, saying it's UB if the function is called doesn't justify speculatively executing it (e.g., hoisting out of branches) .


This attribute enables Rust to perform common optimizations, like sub-expression
elimination, and it can avoid emitting some calls in repeated invocations of the
function with the same argument values regardless of other operations being
performed in between these functions calls (as opposed to `#[c_ffi_pure]`
functions).

## Pitfalls

A `#[c_ffi_const]` function can only read global memory that would not affect
its return value for the whole execution of the program (e.g. immutable global
memory). `#[c_ffi_const]` functions are referentially-transparent and therefore
more strict than `#[c_ffi_pure]` functions.

A common pitfall involves applying the `#[c_ffi_const]` attribute to a
function that reads memory through pointer arguments which do not necessarily
point to immutable global memory.

A `#[c_ffi_const]` function that returns unit has no effect on the abstract
machine's state, and a `#[c_ffi_const]` function cannot be `#[c_ffi_pure]`.

A diverging and C or C++ `const` function is unreachable. Diverging via a
side-effect (e.g. a call to `abort`) violates `const` pre-conditions. Divergence
without side-effects is undefined behavior in C++ and not possible in C. In C++,
the behavior of infinite loops without side-effects is undefined, while in C
these loops can be assumed to terminate. This would cause a diverging function
to return, invoking undefined behavior.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This paragraph confuses me, I think it might incidentally mislead users too, and I don't even think it is technically correct

  • C's "assumed to terminate" is indistinguishable from "UB if non-terminating", so that last sentence is superfluous (and communicates a wrong picture of UB)
  • C doesn't actually say this about all infinite loops, it has an exception carved out for loops controlled by constant expressions, such as while (1) { ... }

I think we should just say as a separate axiom that functions marked with this attribute must not diverge. GCC's docs don't say this as far as I can see (at least explicitly -- arguably non-termination is an observable effect), but it's kind of the safer assumption because who knows what LLVM's implicit assumptions around readnone/readonly functions are at the moment. It also seems useful to assume termination of const/pure functions to be able to execute them speculatively.


When translating C headers to Rust FFI, it is worth verifying for which targets
the `const` attribute is enabled in those headers, and using the appropriate
`cfg` macros in the Rust side to match those definitions. While the semantics of
`const` are implemented identically by many C and C++ compilers, e.g., clang,
[GCC], [ARM C/C++ compiler], [IBM ILE C/C++], etc. they are not necessarily
implemented in this way on all of them. It is therefore also worth verifying
that the semantics of the C toolchain used to compile the binary being linked
against are compatible with those of the `#[c_ffi_const]`.

[ARM C/C++ compiler]: http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0491c/Cacgigch.html
[GCC]: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-const-function-attribute
[IBM ILE C/C++]: https://www.ibm.com/support/knowledgecenter/fr/ssw_ibm_i_71/rzarg/fn_attrib_const.htm
55 changes: 55 additions & 0 deletions src/doc/unstable-book/src/language-features/c-ffi-pure.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# `c_ffi_pure`

The `#[c_ffi_pure]` attribute applies clang's `pure` attribute to foreign
functions declarations.

That is, `#[c_ffi_pure]` functions shall have no effects except for its return
value, which shall not change across two consecutive function calls and can only
depend on the values of the function parameters and/or global memory.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't sound right. GCC docs specifically state pure functions can read arbitrary non-volatile memory. I don't know for sure what you mean by "global memory" is but certainly pure functions can be passed pointers to a caller's local variables and read them through those pointers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A pure function can read statics, including statics that are pointers, and read memory through these pointers.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it can do all that and even more. And I don't think this paragraph properly conveys that.

Copy link
Contributor Author

@gnzlbg gnzlbg Jun 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have any suggestion?

Being able to read caller local variables via pointers passed to them is AFAICT already covered by "depend on the values of the function parameters", and by global memory I mean any memory that is globally accessible, e.g., via an static, a magic address, etc. (not necessarily something that's passed to the function as a parameter).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Sorry for the delay.)

I think language like in the GCC docs (arbitrary memory) is better. The phrase "values of the function parameters" is not clear for pointer arguments because it can be read as "only depend on the pointer itself, not the contents of the memory it points to" and indeed that's sort of the case for c_ffi_const functions. For example, IIUC uintptr_t tag_pointer(void *, int tag) that does arithmetic on the address can be c_ffi_const and work with pointers to non-constant memory, but of course reading from the pointed-to memory would generally make it non-c_ffi_const.


The behavior of calling a `#[c_ffi_pure]` function that violates these
requirements is undefined.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As for const: it should be UB to have a declaration wrongly annotated c_ffi_pure, even if it's not called.


This attribute enables Rust to perform common optimizations, like sub-expression
elimination and loop optimizations. Some common examples of pure functions are
`strlen` or `memcmp`.

These optimizations only apply across successive invocations of the function,
since any other function could modify global memory read by `#[c_ffi_pure]`
functions, altering their result. The `#[c_ffi_const]` attribute allows
sub-expression elimination regardless of any operations in between the function
calls.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I see what you're getting at here but this wording is misleading. It is harder for the optimizer to apply CSE because it has to show that no relevant state changed between two calls it wants to CSE, but "only apply across successive invocations" only communicates that if it means "directly adjacent without anything/much in between", which is too strong as stated (e.g., it might be known that the pure function doesn't read certain memory, or that another function called in between doesn't write memory).

I think I'd prefer to weaken this note to just briefly describe the difficulties imposed on the optimizer by the weaker rules of pure vs const and not get into the hairy business of predicting what that actually means for when the optimization fires or not.


## Pitfalls

A `#[c_ffi_pure]` function can read global memory through the function
parameters (e.g. pointers), globals, etc. `#[c_ffi_pure]` functions are not
referentially-transparent, and are therefore more relaxed than `#[c_ffi_const]`
functions.

However, accesing global memory through volatile or atomic reads can violate the
requirement that two consecutive function calls shall return the same value.

A `pure` function that returns unit has no effect on the abstract machine's
state.

A diverging and `pure` C or C++ function is unreachable. Diverging via a
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same point about termination as with const.

side-effect (e.g. a call to `abort`) violates `pure` requirements. Divergence
without side-effects is undefined behavior in C++ and not possible in C. In C++,
the behavior of infinite loops without side-effects is undefined, while in C
these loops can be assumed to terminate. This would cause a diverging function
to return, invoking undefined behavior.

When translating C headers to Rust FFI, it is worth verifying for which targets
the `pure` attribute is enabled in those headers, and using the appropriate
`cfg` macros in the Rust side to match those definitions. While the semantics of
`pure` are implemented identically by many C and C++ compilers, e.g., clang,
[GCC], [ARM C/C++ compiler], [IBM ILE C/C++], etc. they are not necessarily
implemented in this way on all of them. It is therefore also worth verifying
that the semantics of the C toolchain used to compile the binary being linked
against are compatible with those of the `#[c_ffi_pure]`.


[ARM C/C++ compiler]: http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0491c/Cacigdac.html
[GCC]: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-pure-function-attribute
[IBM ILE C/C++]: https://www.ibm.com/support/knowledgecenter/fr/ssw_ibm_i_71/rzarg/fn_attrib_pure.htm
6 changes: 6 additions & 0 deletions src/librustc/hir/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2489,6 +2489,12 @@ bitflags! {
/// #[used], indicates that LLVM can't eliminate this function (but the
/// linker can!)
const USED = 1 << 9;
/// #[c_ffi_pure]: applies clang's `pure` attribute to a foreign function
/// declaration.
const C_FFI_PURE = 1 << 10;
/// #[c_ffi_const]: applies clang's `const` attribute to a foreign function
/// declaration.
const C_FFI_CONST = 1 << 11;
}
}

Expand Down
6 changes: 6 additions & 0 deletions src/librustc_codegen_llvm/attributes.rs
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,12 @@ pub fn from_fn_attrs(
if codegen_fn_attrs.flags.contains(CodegenFnAttrFlags::COLD) {
Attribute::Cold.apply_llfn(Function, llfn);
}
if codegen_fn_attrs.flags.contains(CodegenFnAttrFlags::C_FFI_PURE) {
Attribute::ReadOnly.apply_llfn(Function, llfn);
}
if codegen_fn_attrs.flags.contains(CodegenFnAttrFlags::C_FFI_CONST) {
Attribute::ReadNone.apply_llfn(Function, llfn);
}
if codegen_fn_attrs.flags.contains(CodegenFnAttrFlags::NAKED) {
naked(llfn, true);
}
Expand Down
1 change: 1 addition & 0 deletions src/librustc_codegen_llvm/llvm/ffi.rs
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ pub enum Attribute {
SanitizeMemory = 22,
NonLazyBind = 23,
OptimizeNone = 24,
ReadNone = 25,
}

/// LLVMIntPredicate
Expand Down
36 changes: 35 additions & 1 deletion src/librustc_typeck/collect.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2270,7 +2270,41 @@ fn codegen_fn_attrs<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>, id: DefId) -> Codegen
codegen_fn_attrs.flags |= CodegenFnAttrFlags::ALLOCATOR;
} else if attr.check_name("unwind") {
codegen_fn_attrs.flags |= CodegenFnAttrFlags::UNWIND;
} else if attr.check_name("rustc_allocator_nounwind") {
} else if attr.check_name("c_ffi_pure") {
if tcx.is_foreign_item(id) {
if attrs.iter().any(|a| a.check_name("c_ffi_const")) {
// `#[c_ffi_const]` functions cannot be `#[c_ffi_pure]`
struct_span_err!(
tcx.sess,
attr.span,
E0726,
"`#[c_ffi_const]` function cannot be`#[c_ffi_pure]`"
).emit();
} else {
codegen_fn_attrs.flags |= CodegenFnAttrFlags::C_FFI_PURE;
}
} else {
// `#[c_ffi_pure]` is only allowed on foreign functions
struct_span_err!(
tcx.sess,
attr.span,
E0724,
"`#[c_ffi_pure]` may only be used on foreign functions"
).emit();
}
} else if attr.check_name("c_ffi_const") {
if tcx.is_foreign_item(id) {
codegen_fn_attrs.flags |= CodegenFnAttrFlags::C_FFI_CONST;
} else {
// `#[c_ffi_const]` is only allowed on foreign functions
struct_span_err!(
tcx.sess,
attr.span,
E0725,
"`#[c_ffi_const]` may only be used on foreign functions"
).emit();
}
} else if attr.check_name("rustc_allocator_nounwind") {
codegen_fn_attrs.flags |= CodegenFnAttrFlags::RUSTC_ALLOCATOR_NOUNWIND;
} else if attr.check_name("naked") {
codegen_fn_attrs.flags |= CodegenFnAttrFlags::NAKED;
Expand Down
3 changes: 3 additions & 0 deletions src/librustc_typeck/diagnostics.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4720,4 +4720,7 @@ register_diagnostics! {
E0698, // type inside generator must be known in this context
E0719, // duplicate values for associated type binding
E0722, // Malformed #[optimize] attribute
E0724, // `#[c_ffi_pure]` is only allowed on foreign functions
E0725, // `#[c_ffi_const]` is only allowed on foreign functions
E0726, // `#[c_ffi_const]` functions cannot be `#[c_ffi_pure]`
}
16 changes: 16 additions & 0 deletions src/libsyntax/feature_gate.rs
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,12 @@ declare_features! (
// The `repr(i128)` annotation for enums.
(active, repr128, "1.16.0", Some(35118), None),

// Allows the use of `#[c_ffi_pure]` on foreign functions.
(active, c_ffi_pure, "1.34.0", Some(58329), None),

// Allows the use of `#[c_ffi_const]` on foreign functions.
(active, c_ffi_const, "1.34.0", Some(58328), None),

// The `unadjusted` ABI; perma-unstable.
//
// rustc internal
Expand Down Expand Up @@ -1124,6 +1130,16 @@ pub const BUILTIN_ATTRIBUTES: &[(&str, AttributeType, AttributeTemplate, Attribu
"the `#[naked]` attribute \
is an experimental feature",
cfg_fn!(naked_functions))),
("c_ffi_pure", Whitelisted, template!(Word), Gated(Stability::Unstable,
"c_ffi_pure",
"the `#[c_ffi_pure]` attribute \
is an experimental feature",
cfg_fn!(c_ffi_pure))),
("c_ffi_const", Whitelisted, template!(Word), Gated(Stability::Unstable,
"c_ffi_const",
"the `#[c_ffi_const]` attribute \
is an experimental feature",
cfg_fn!(c_ffi_const))),
("target_feature", Whitelisted, template!(List: r#"enable = "name""#), Ungated),
("export_name", Whitelisted, template!(NameValueStr: "name"), Ungated),
("inline", Whitelisted, template!(Word, List: "always|never"), Ungated),
Expand Down
2 changes: 2 additions & 0 deletions src/rustllvm/RustWrapper.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,8 @@ static Attribute::AttrKind fromRust(LLVMRustAttribute Kind) {
return Attribute::OptimizeForSize;
case ReadOnly:
return Attribute::ReadOnly;
case ReadNone:
return Attribute::ReadNone;
case SExt:
return Attribute::SExt;
case StructRet:
Expand Down
1 change: 1 addition & 0 deletions src/rustllvm/rustllvm.h
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ enum LLVMRustAttribute {
SanitizeMemory = 22,
NonLazyBind = 23,
OptimizeNone = 24,
ReadNone = 25,
};

typedef struct OpaqueRustString *RustStringRef;
Expand Down
12 changes: 12 additions & 0 deletions src/test/codegen/c-ffi-const.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
// compile-flags: -C no-prepopulate-passes
#![crate_type = "lib"]
#![feature(c_ffi_const)]

pub fn bar() { unsafe { foo() } }

extern {
// CHECK-LABEL: declare void @foo()
// CHECK-SAME: [[ATTRS:#[0-9]+]]
// CHECK-DAG: attributes [[ATTRS]] = { {{.*}}readnone{{.*}} }
#[c_ffi_const] pub fn foo();
}
12 changes: 12 additions & 0 deletions src/test/codegen/c-ffi-pure.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
// compile-flags: -C no-prepopulate-passes
#![crate_type = "lib"]
#![feature(c_ffi_pure)]

pub fn bar() { unsafe { foo() } }

extern {
// CHECK-LABEL: declare void @foo()
// CHECK-SAME: [[ATTRS:#[0-9]+]]
// CHECK-DAG: attributes [[ATTRS]] = { {{.*}}readonly{{.*}} }
#[c_ffi_pure] pub fn foo();
}
6 changes: 6 additions & 0 deletions src/test/ui/c_ffi_const.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
// ignore-tidy-linelength
#![feature(c_ffi_const, c_ffi_pure)]
#![crate_type = "lib"]

#[c_ffi_const] //~ ERROR `#[c_ffi_const]` may only be used on foreign functions [E0725]
pub fn foo() {}
9 changes: 9 additions & 0 deletions src/test/ui/c_ffi_const.stderr
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
error[E0725]: `#[c_ffi_const]` may only be used on foreign functions
--> $DIR/c_ffi_const.rs:5:1
|
LL | #[c_ffi_const] //~ ERROR `#[c_ffi_const]` may only be used on foreign functions [E0725]
| ^^^^^^^^^^^^^^

error: aborting due to previous error

For more information about this error, try `rustc --explain E0725`.
12 changes: 12 additions & 0 deletions src/test/ui/c_ffi_const2.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
// ignore-tidy-linelength
#![feature(c_ffi_const, c_ffi_pure)]

extern {
#[c_ffi_pure] //~ ERROR `#[c_ffi_const]` function cannot be`#[c_ffi_pure]` [E0726]
#[c_ffi_const]
pub fn baz();
}

fn main() {
unsafe { baz() };
}
9 changes: 9 additions & 0 deletions src/test/ui/c_ffi_const2.stderr
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
error[E0726]: `#[c_ffi_const]` function cannot be`#[c_ffi_pure]`
--> $DIR/c_ffi_const2.rs:5:5
|
LL | #[c_ffi_pure] //~ ERROR `#[c_ffi_const]` function cannot be`#[c_ffi_pure]` [E0726]
| ^^^^^^^^^^^^^

error: aborting due to previous error

For more information about this error, try `rustc --explain E0726`.
6 changes: 6 additions & 0 deletions src/test/ui/c_ffi_pure.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
// ignore-tidy-linelength
#![feature(c_ffi_pure)]
#![crate_type = "lib"]

#[c_ffi_pure] //~ ERROR `#[c_ffi_pure]` may only be used on foreign functions [E0724]
pub fn foo() {}
9 changes: 9 additions & 0 deletions src/test/ui/c_ffi_pure.stderr
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
error[E0724]: `#[c_ffi_pure]` may only be used on foreign functions
--> $DIR/c_ffi_pure.rs:5:1
|
LL | #[c_ffi_pure] //~ ERROR `#[c_ffi_pure]` may only be used on foreign functions [E0724]
| ^^^^^^^^^^^^^

error: aborting due to previous error

For more information about this error, try `rustc --explain E0724`.
7 changes: 7 additions & 0 deletions src/test/ui/feature-gates/feature-gate-c_ffi_const.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
// ignore-tidy-linelength
#![crate_type = "lib"]

extern {
#[c_ffi_const] //~ ERROR the `#[c_ffi_const]` attribute is an experimental feature (see issue #58328)
pub fn foo();
}
11 changes: 11 additions & 0 deletions src/test/ui/feature-gates/feature-gate-c_ffi_const.stderr
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
error[E0658]: the `#[c_ffi_const]` attribute is an experimental feature (see issue #58328)
--> $DIR/feature-gate-c_ffi_const.rs:5:5
|
LL | #[c_ffi_const] //~ ERROR the `#[c_ffi_const]` attribute is an experimental feature (see issue #58328)
| ^^^^^^^^^^^^^^
|
= help: add #![feature(c_ffi_const)] to the crate attributes to enable

error: aborting due to previous error

For more information about this error, try `rustc --explain E0658`.
7 changes: 7 additions & 0 deletions src/test/ui/feature-gates/feature-gate-c_ffi_pure.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
// ignore-tidy-linelength
#![crate_type = "lib"]

extern {
#[c_ffi_pure] //~ ERROR the `#[c_ffi_pure]` attribute is an experimental feature (see issue #58329)
pub fn foo();
}
11 changes: 11 additions & 0 deletions src/test/ui/feature-gates/feature-gate-c_ffi_pure.stderr
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
error[E0658]: the `#[c_ffi_pure]` attribute is an experimental feature (see issue #58329)
--> $DIR/feature-gate-c_ffi_pure.rs:5:5
|
LL | #[c_ffi_pure] //~ ERROR the `#[c_ffi_pure]` attribute is an experimental feature (see issue #58329)
| ^^^^^^^^^^^^^
|
= help: add #![feature(c_ffi_pure)] to the crate attributes to enable

error: aborting due to previous error

For more information about this error, try `rustc --explain E0658`.