RFC for unsafe blocks in unsafe fn

[RalfJung]

No longer treat the body of an unsafe fn as being an unsafe block. To avoid a breaking change, this is a warning now and may become an error in a future edition.

Rendered

The RFC has been adjusted to be a lint; see here for the comment announcing that change and the following discussion.

Cc @rust-lang/wg-unsafe-code-guidelines

[Diggsey]

If this were to be accepted, it would be much better to get the warning in before the 2018 epoch hits stable. If there's no time to get it in for 2018 then I don't think it should be accepted.

[oli-obk]

We can start as allow by default. The fact that const unsafe fn already behaves this way and that clippy can uplift this lint to warn, will already make sure to migrate large parts of the ecosystem.

Oh... you are already mentioning that below in the unresolved questions...

[RalfJung]

Yeah, I had to follow the RFC structure, didn't I? ;)

[RalfJung]

@Diggsey There will be another edition. And once we no longer warn about unsafe blocks in unsafe fn being redundant, we can indeed phase this in more smoothly e.g. with Clippy.

[burdges]

We needed this years ago, so the sooner the better.

[Ixrec]

C# has unsafe blocks in addition to unsafe methods. Though it's not super helpful since I'm not aware of the C# community ever discussing burden of proof issues like this RFC does, probably because 99.99% of the time the answer in that language is "unsafe just isn't worth it". I couldn't even find a C# style guide that mentions the existence of unsafe code, much less has guidelines for making it less dangerous.

[Centril]

Furthermore, while other languages may not have blocks many do have escape hatches (which are relevant to this section)... Examples I found:

1. Haskell: http://hackage.haskell.org/package/base-4.12.0.0/docs/System-IO-Unsafe.html#v:unsafePerformIO and http://hackage.haskell.org/package/base-4.8.0.0/docs/Unsafe-Coerce.html

2. Idris: https://github.com/idris-lang/Idris-dev/blob/master/libs/prelude/Builtins.idr#L189

3. Agda: https://agda.readthedocs.io/en/v2.5.4.2/language/postulates.html

4. Coq: https://github.com/coq/coq/wiki/CoqAndAxioms also, http://adam.chlipala.net/cpdt/html/Universes.html#lab78

5. ReasonML: https://reasonml.github.io/docs/en/interop -- actually, this is sort of a block...

6. OCaml: https://caml.inria.fr/pub/docs/manual-ocaml/libref/Obj.html

7. SML: http://www.smlnj.org/doc/SMLofNJ/pages/unsafe.html

8. Ada: http://www.adaic.org/resources/add_content/docs/95style/html/sec_5/5-9-1.html

[RalfJung]

@Centril But this RFC is specifically about blocks and nesting of unsafe escape hatches. I do not think any of the examples you mention apply there, do they?

@Ixrec Thanks, I had no idea! And it looks like unsafe operations can be used freely in unsafe functions. :/

[Centril]

@RalfJung not with that specificity no; the languages have the "block" form, e.g.:

x = unsafePerformIO $ do
    foo
    bar
    ...

what they lack is the unsafe function form.

[RalfJung]

That's still quite different from Rust. It's just a normal function to the compiler, no checks for "unsafe operations" or so are performed. I do not see a close enough relation to this RFC.

[Centril]

@RalfJung alright; fair enough. Let's leave this bit (the comment thread) open for interested readers who want to see the associated material linked. :)

[Centril]

Other possible drawbacks to list:

1. It will become less ergonomic to write unsafe code (it's justified I think, but worth mentioning...).

2. People might just do this:

unsafe fn frobnicate(x: T, y: U, ...) -> R {
    unsafe {
        ... // Actual code.
    }
}

and then nothing has been gained. I don't know what the risk of this is, but worth mentioning.

[RalfJung]

I added (a variant of) 1.

For 2., I think something has been gained: It is not possible to incrementally improve this function's unsafe locality. Or maybe it is not worth it, then that has at least been explicitly documented in the code.

[Centril]

Yeah I'm not entirely sure 2. is a drawback or not; I usually try to write the section as what someone else might think is a potential drawback (but not necessarily me) -- i.e. this is the section where I try to bring out my inner Dr. Phil / empathy =P

[RalfJung]

The drawbacks section now says

Many unsafe fn are actually rather short (no more than 3 lines) and will
likely end up just being one large unsafe block. This change would make such functions less ergonomic to write.

That should cover 2, right?

[Centril]

@RalfJung the concern is actually slightly different here; it is that people will just go and write one big unsafe { ... } when they shouldn't.

[mark-i-m]

Isn't that already possible today?

[Centril]

@mark-i-m yes; sure -- the concern is that the change we do here might not have any noticable effect cause people could be lazy and...

[pqnet]

Wouldn't it be good that people can explicitly choose to not write a big unsafe block instead of being forced into one automatically?

[Centril]

I've added T-dev-tools for the possible clippy lint for now. If no such clippy lint is proposed in the final version before final review of the RFC I'll remove that team.

[scottmcm]

I think this RFC need more examples of realistic code where this would help, and an explanation of why it helps in enough cases that it's worse the obvious pain in current cases. That seems especially true since "safe" code is just as suspect when it's around unsafe.

Another alternative: an ununsafe block (obvious strawman name) that disallows calling unsafe code again (that can be undone with another unsafe block, of course).

More generally, async fn puts an async block around the body (among other things), so it doesn't seem insane that unsafe fn puts an unsafe block around the body. Though we won't have effect polymorphism any time soon, is there some inconsistency here that should be fixed at a different level?

[mark-i-m]

@scottmcm There are some places in the language where you are forced to use unsafe fn. For example, SIMD or alternate calling conventions or implementing traits with unsafe functions.

Here are some example from a project I worked on:

1. Allocator trait: https://github.com/mark-i-m/os2/blob/47136c645878e0295142213bf63e03fe4e0bca45/kernel/memory/heap.rs#L26-L43
   You might notice that it's very non-obvious from the body of these implementations if they actually use any unsafe operations. IIRC, they don't.

2. Weird calling conventions: https://github.com/mark-i-m/os2/blob/47136c645878e0295142213bf63e03fe4e0bca45/kernel/process/sched.rs#L158-L179
   This function is part of the context-switching code of an OS kernel. The stack is in a very weird state when this is called. In this case, the caller does have a proof obligation (it should only be called from a particular part of the kernel). It also happens that there are one or two patches of unsafe operations. It would be really nice to separate these concerns

[sfackler]

Another concrete use case I find valuable:

Callback functions used when interacting with C libraries are almost always unsafe extern "C" fns, since they're usually passed raw pointers. However, the actual scope of unsafety in the implementations of the callbacks is commonly limited to casting those raw pointers in to Rust references. Currently, that's not called out visually since the entire function body is already an unsafe block but this RFC would enable more tightly scoped blocks.

[RalfJung]

More generally, async fn puts an async block around the body (among other things), so it doesn't seem insane that unsafe fn puts an unsafe block around the body. Though we won't have effect polymorphism any time soon, is there some inconsistency here that should be fixed at a different level?

unsafe is not an effect and behaves nothing like an effect. :)

async says "this function is externally observable to not behave like a normal function, not even calling it works the normal way". unsafe just means "this is a normal function but you have some preconditions". unsafe can be discharged: By proving some things (just proving and checking, not actually doing anything!), you can make an unsafe function safe (think: get_unchecked vs get). This is impossible with effects. You cannot remove async from your function after proving some things about it or adding some assert!.

There is some syntactical similarity between async and unsafe, but semantically speaking they are worlds apart.

[Centril]

unsafe is not an effect and behaves nothing like an effect. :)

well... not everyone agrees (as you know) ;) https://internals.rust-lang.org/t/what-does-unsafe-mean/6696/2
cc @eternaleye

[RalfJung]

Some further examples of longer unsafe fn that look like they would benefit from a more clear demarcation of where the danger is in there:

• align_to and align_to_mut
• Rc helper methods from_raw and allocate_for_ptr
• slice::merge
• The WASM mutex implementation
• do_exec for Fuchsia

Basically any time your unsafe fn contains any non-trivial logic, the implicit unsafe block is not your friend.

However, I also have to admit that the vast majority of unsafe fn are less than 3 lines and just call another unsafe fn or perform a raw ptr deref or so. For all of them, this change would mostly be syntactic noise, which is unfortunate.

[RalfJung]

well... not everyone agrees (as you know) ;)

I am aware. However, I gave my usual arguments above, and AFAIK they have not been refuted yet, so I will keep claiming that everyone who says unsafe is an effect is wrong. ;) In this particular case I think it is actually actively harmful to think of unsafe this way, because it emphasizes a focus on the "additional power" aspect of unsafe, instead of focusing on the "proof obligation" aspect. I think the latter is vastly more important, and the language agrees with me: If the focus was "additional power", we would not let you write unsafe blocks on a safe fn. If the focus was additional power, we would e.g. use unsafe to mark the presence of interior mutability and we would want a guarantee like "calling a safe fn will never write to shared data" (akin to "calling a non-async fn will never yield to another task"). We do not have this guarantee, because this is not what unsafe is for. It is not an effect. We could have an annotation for "writes to shared data", and I agree that would be an effect.

I think whoever claims that unsafe is an effect should formally define what you can then say about a function that is not marked unsafe, in terms of its observable behavior. Because that's what effects make for: To make statements like "does not panic" or "does not allocate" or "does not yield". "Has been manually proven correct" is very, very different from that in that it can be hidden behind an abstraction barrier.

But anyway, this is getting off-topic. ;)

[SimonSapin]

I’m worried about the migration of existing code.

I’d like to see this RFC make it a requirement that rustfix / cargo fix need to fully support automating the necessary code changes, before this lint can warn by default.

But this is tricky, simply wrapping the entire body of a function into a new unsafe block sort of defeats the purpose of this change. On the other hand it would likely be very noisy to minimize the size of generated unsafe blocks as much as possible by wrapping each unsafe fn call (or other operation that needs it) separately. Finding a balance between those likely requires case-by-case subjective human judgment.

[RalfJung]

But this is tricky, simply wrapping the entire body of a function into a new unsafe block sort of defeats the purpose of this change.

I wouldn't necessarily say so. It still provides benefits for new unsafe code written later, and it permits gradual migration of existing unsafe code.

[SimonSapin]

Good point, my "sort of" only applies to existing code. I didn’t meant to argue against this RFC, I was only pondering the merits of different ways to deal with the migration. Sorry if I implied otherwise.

[RalfJung]

It's okay. :) I will add something about migrations to the RFC text.

[newpavlov]

I wonder how many of existing unsafe fns would simply wrap the whole function body with unsafe block with the proposed change. If it's more than 90% (for my code I think its true), I think it will be better to introduce some kind of ununsafe/safe block which will turn on safety checks for a wrapped code. I would hate if code like this will be common:

unsafe fn foo() {
    unsafe {
        // ..
    }
}

And while treating unsafe fn as an effect is debatable (I agree with @RalfJung argumentation here), but I think that the current behaviour is consistent and easy to understand, while the snippet above can be somewhat surprising for new users.

[Manishearth]

I've added T-dev-tools for the possible clippy lint for now. If no such clippy lint is proposed in the final version before final review of the RFC I'll remove that team.

Given that this didn't happen, devtools should be ignored here. Going to tick all our boxes and untag the team (which seems to be the way to handle this without re-FCPing, the "majority of reviewers" calculation will take this into account)

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[pnkfelix]

I don't think this is curious at all. Or at least, I think its very easy to justify: The methods of an unsafe trait are not themselves unsafe fn. So it would be unsound to make their default method definitions implicitly have unsafe bodies.

[RustyYato]

Rationale: We have an unsafe keyword associated to an item with a body (trait, impl, or fn in this case), should the body be logically surrounded by a unsafe block? (all items in the trait/impl or all statements in a fn). Currently the answer differs between trait/impl and fn, and that's curious.

If we apply your logic to unsafe fn, we shouldn't be wrapping the body of an unsafe fn with an unsafe block, because that would be unsound (an unsafe fn is requires the caller to enforce invariants, and says nothing about the callee).

[RalfJung]

So it would be unsound to make their default method definitions implicitly have unsafe bodies.

I think this depends on the particular definition of "unsound". unsafe is still required to cause any UB. And the boy of a fn inside an unsafe impl already does not to be manually reviewed to avoid UB elsewhere (in callers that rely on the unsafe part of the contract).

I agree with you for our conventional definition of "unsound", though.

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

The RFC will be merged soon.

[nikomatsakis]

Huzzah! The @rust-lang/lang team has decided to accept this RFC.

If you'd like to follow its progess towards implementation, please subscribe to the tracking issue.

[Serentty]

I really hope this doesn't become an error in the future, and to be honest I would even be sad to see it be deny-by-default. When the scope of unsafe code is significantly larger than a function, such as for modules full of fiddling with hardware through memory-mapped I/O and such, the verbosity is quite extreme. This seems to me like it punishes breaking up unsafe code into more functions.

[Lokathor]

Hard Error would be completely unacceptable.

Deny By Default wouldn't be a big deal. You just put a single allow at the top of your crate and get on with life.