Add support for setjmp/longjmp on x86_64.

[jeff-davis]

Fix issue #1208

[rust-highfive]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @gnzlbg (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

[gnzlbg]

I honestly have no idea of the semantics that calling setjmp / longjmp can have from rust .

Is it even possible to use these APIs correctly from Rust ?

cc @eddyb @newpavlov @Nemo157

[newpavlov]

IIUC they will be mostly used for FFI, and other uses should be heavily discouraged.

[gnzlbg]

@newpavlov independently of what their intended usage is, i am wondering whether they can be used correctly at all. E.g. longjmp allows you to return from multiple functions without actually going through a return statement (by changing the program counter, stack pointer, etc.). In this PR, this happens by calling an unknown function opaque to LLVM that magically performs these changes.

Is that allowed by LLVM ? In other words: can LLVM perform optimizations that assume that this does not happen? LLVM has its own intrinsics for this: https://llvm.org/docs/ExceptionHandling.html#sjlj-intrinsics, it might well be that LLVM requires this to happen through this intrinsics instead of through a call to an unknown function.

[jeff-davis]

Seems easy enough to answer the LLVM question: I wrote a simple C program using setjmp/longjmp and compiled it with:

clang-6.0 -emit-llvm -S setjmp.c -o setjmp.S

The resulting LLVM has:

  ...
  declare i32 @_setjmp(%struct.__jmp_buf_tag*) #1
  ...
  %3 = call i32 @_setjmp(%struct.__jmp_buf_tag* %2) #4
  ...
  declare void @longjmp(%struct.__jmp_buf_tag*, i32) #3
  ...
  call void @longjmp(%struct.__jmp_buf_tag* %4, i32 0) #5
  ...

So it looks like LLVM is capable of handling a call to setjmp/longjmp.

setjmp.c:

#include <setjmp.h>
#include <stdio.h>

void f1(void);
void f2(jmp_buf jmp);
void f3(jmp_buf jmp);
void f4(jmp_buf jmp);

void f1()
{
  jmp_buf local_jmp;
  if (setjmp(local_jmp) == 0) {
    printf("after setjmp\n");
    f2(local_jmp);
    printf("after f2\n");
  } else {
    printf("caught longjmp\n");
  }
}

void f2(jmp_buf jmp)
{
  printf("f2 start\n");
  f3(jmp);
  printf("f2 after calling f3\n");
}

void f3(jmp_buf jmp)
{
  printf("f3 start\n");
  f4(jmp);
  printf("f3 after calling f4\n");
}

void f4(jmp_buf jmp)
{
  printf("f4 start\n");
  longjmp(jmp, 0);
  printf("f4 after calling longjmp\n");
}

int main()
{
  f1();
}

output when run:

after setjmp
f2 start
f3 start
f4 start
caught longjmp

[jeff-davis]

I also tested it for use in my FFI use case and it works as I want it to: it can catch longjmp()s coming from C, and turn them into rust panics, which then appropriately call the destructors while unwinding.

Is it pretty? No, but working with C is not always pretty.

[gnzlbg]

So it looks like LLVM is capable of handling a call to setjmp/longjmp.

Note that just because it appears to sometimes work does not mean that it always work.
Also, there is a crucial difference between C and Rust: C does not have destructors.

C++, like Rust, does have destructors, and the C++ standard [support.runtime] states:

If any automatic objects would be destroyed by a thrown exception transferring control to another (destination) point in the program, then a call to longjmp(jbuf, val) at the throw point that transfers control to the same (destination) point has undefined behavior.

That is, the C++ standard includes explicit wording that states that calls to longjmp that skip destructors cannot happen (e.g., an optimizer is allowed to remove all such code as unreachable).

To me it is completely unclear how these "functions" should interact with Rust (e.g. how would miri execute them - cc @ralfj @oli-obk). We should at least clarify that calling longjmp isn't always undefined behavior (functions using setjmp being able to return twice within the same activation frame, etc.) before we add it.

[jeff-davis]

So it looks like LLVM is capable of handling a call to setjmp/longjmp.

Note that just because it appears to sometimes work does not mean that it always work.

But in this case clang is generating the calls to libc's setjmp/longjmp. If clang (written by the authors of LLVM) think it's OK to call setjmp/longjmp in LLVM, I don't see how it would be some inherent problem with LLVM.

Also, there is a crucial difference between C and Rust: C does not have destructors.

I understand that. I have no desire to skip over destructors. I only want to use setjmp in the lowest rust frame before an FFI call to catch any longjmp. That protects me from skipping over rust frames, and there is currently no other way to protect me from that.

To me it is completely unclear how these "functions" should interact with Rust (e.g. how would miri execute them - cc @ralfj @oli-obk). We should at least clarify that calling longjmp isn't always undefined behavior before we add it.

longjmp is a fact of life in C. If we ignore it, we are allowing those longjmps to unsafely skip over rust destructors. setjmp is the only solution that I see.

I also added longjmp for two reasons:

1. also helpful to me to turn rust panics back into longjmp before returning to a C function
2. symmetry after adding setjmp.

Though I don't need longjmp() quite so much, and can work around it more easily. If you want to only add setjmp, I am fine with that for now.

[gnzlbg]

I only want to use setjmp in the lowest rust frame before an FFI call to catch any longjmp. That protects me from skipping over rust frames, and there is currently no other way to protect me from that.

Why aren't you doing this in C? That is, why aren't you calling from Rust a C wrapper over postgres that uses setjmp/longjmp to avoid having to use those from Rust ?

If we ignore it, we are allowing those longjmps to unsafely skip over rust destructors.

Adding these functions to libc does not make them magically work (these don't have to be part of libc for you to use it). AFAIK if I merge this PR I am adding a function to libc that always invokes undefined behavior when its called. The only fix for that would be for us to define the behavior of this function.

[jeff-davis]

I only want to use setjmp in the lowest rust frame before an FFI call to catch any longjmp. That protects me from skipping over rust frames, and there is currently no other way to protect me from that.

Why aren't you doing this in C? That is, why aren't you calling from Rust a C wrapper over postgres that uses setjmp/longjmp to avoid having to use those from Rust ?

1. It would require a separate C wrapper for each C function I intend to call. For my use case, which is making it easier for people to write postgres extensions in rust, that would be wrapping hundreds of functions. In rust, I can use a macro, or wrap the function in rust, or do slightly different things for some functions when necessary.
2. It would be a separate build step in C with its own portability issues.

If we ignore it, we are allowing those longjmps to unsafely skip over rust destructors.

Adding these functions to libc does not make them magically work (these don't have to be part of libc for you to use it). AFAIK if I merge this PR I am adding a function to libc that always invokes undefined behavior when its called. The only fix for that would be for us to define the behavior of this function.

There are two functions: setjmp() and longjmp(). Which has undefined behavior, and by what reasoning is it undefined while other FFI calls are defined?

What is the cost of defining this behavior? One of the things that's great about rust is how well it integrates with C or any language that can integrate with C. Adding support for setjmp() and longjmp() seem to fit that spirit.

[gnzlbg]

here are two functions: setjmp() and longjmp(). Which has undefined behavior

longjmp

and by what reasoning is it undefined while other FFI calls are defined?

All functions called via Rust FFI that we support either return their return value to the caller at the call site, or they never return. In particular, we forbid extern functions from performing other actions like unwinding the stack (e.g. if the C code called by Rust calls C++ code that throws an exception, and the exception unwinds into Rust, the behavior is undefined).

longjmp does not return a value to the caller at the callsite but instead jumps to a different point in the program stack - that makes it very different from all C FFI functions that we currently accept.

Adding support for setjmp() and longjmp() seem to fit that spirit.

To me that looks like RFC territory, it is basically adding a new inter-procedural control flow mechanism to rust that lives along return values, panics, generators, etc. but is different to all of them.

[gnzlbg]

Also, C11 7.13.1.1p4 states the following about setjmp:

An invocation of the setjmp macro shall appear only in one of the following contexts:

• the entire controlling expression of a selection or iteration statement;
• one operand of a relational or equality operator with the other operand an integer constant expression, with the resulting expression being the entire controlling expression of a selection or iteration statement;
• the operand of a unary ! operator with the resulting expression being the entire controlling expression of a selection or iteration statement; or
• the entire expression of an expression statement (possibly cast to void).

While C11 7.13.1.1p5 states:

If the invocation appears in any other context, the behavior is undefined.

That is, if you wanted to assign the result of setjmp to a "variable" in Rust, using let x = setjmp(...); the C standard says that at least in C that would be undefined behavior (there are many issues with the state of non-volatile variables in the stack frame). Is that UB in Rust ? No idea, probably? But then we are adding a function with a return type that cannot be used where the return types of functions can normally we used. That sounds like RFC territory as well.

Also, functions that use setjmp (and the extern setjmp function if you want to use it via C FFI) probably need to use LLVM return_twice attribute, which is specified as:

This attribute indicates that this function can return twice. The C setjmp is an example of such a function. The compiler disables some optimizations (like tail calls) in the caller of these functions.

which specifically mentions setjmp and hints at possible mis optimizations if not used (if we add them as extern functions like this PR does, setjmp currently lack this attribute).

---

An example of how easy it is to trigger undefined behavior is given by this program (playground) which works in debug but fails in release due to a mis-optimization:

extern crate libc;
use libc::c_int;

pub type jmp_buf = [i64; 8];

extern "C" {
    fn setjmp(env: &mut jmp_buf) -> c_int;
    fn longjmp(env: &jmp_buf, val: c_int);
}

unsafe fn foo() -> i32 {
    let mut buf: jmp_buf = [0; 8];
    let mut x = 42;
    if setjmp(&mut buf) != 0 {
        // this should always return 13
        return x;
    }
    x = 13;
    longjmp(&mut buf, 1);
    x // this will never be reached
}

fn main() {
    // in debug foo returns 13 correctly, but in release it returns 42
    assert_eq!(unsafe { foo() }, 13); 
}

IIUC, LLVM assumes that the return x; will only be reached before x = 13, so it will always return 42, but longjmp completely breaks LLVM assumptions - in C we can make x volatile to workaround this, but in Rust we would have to do volatile load and stores (e.g. playground).

cc @rkruppe

---

EDIT: basically, if we want to add these, we have to add them to core::intrinsics and treat them specially for anything to work. Also, once one uses a setjmp, many loads / stores might need to become volatile for the code to work correctly.

[jeff-davis]

longjmp

I am willing to remove longjmp() from my PR if setjmp() is acceptable by itself. I think setjmp() is a nice safety feature when working with C.

[gnzlbg]

Using setjmp as proposed in this PR is only ok as long as it only returns once. That is, Rust code calls it, it returns 0, and the program never jumps to it again.

AFAIK that would make setjmp useless, since its raison-d'être is being able to longjmp to it from somewhere else.

For that, AFAICT, it cannot be added as a simple extern function. We'd have to add it as a core::intrinsic to Rust C, and apply the appropriate LLVM attributes to it to prevent mis-optimizations like the one I showed above.

[hanna-kruppe]

Yes, the LLVM IR generated for a call to setjump needs the return_twice attribute. That can be done by a (Rust-level) unstable attribute instead of via a new intrinsic, but some compiler integration is needed. In Clang that is presumably achieved by either special-casing setjmp in its CodeGen, or with a special non-portable attribute or pragma in the header that declares the function.

I assume the severe restrictions the C standard places on where setjmp can be used are intended to ensure (at minimum, there may be more to it) that people don't "wrap" setjmp in new functions but only use it locally. For example, a simple pass-through wrapper around setjmp would have to be special-cased during optimizations just like setjmp itself (e.g. with returns_twice in LLVM), and there's no standard mechanism to annotate functions as such. The same issue applies in Rust as well, so similar restrictions ought to apply. There might also be other issues I can't think of right now, especially with other (maybe historical?) implementation strategies.

And about longjump skipping over destructors: that is indeed a concern because it's easy to write Rust libraries (and indeed people have written such libraries -- e.g., anything with scoped threads) that are unsound if combined with setjmp/longjump. However, adding it as an unsafe function to libc doesn't cause any unsoundness, it just creates yet another spike pit that unsafe code authors have to be aware of. Safe setjmp/longjump is an interesting case study for unsafe code guidelines thinking but doesn't impact this PR or the underlying use case as far as I can tell.

[gnzlbg]

That can be done by a (Rust-level) unstable attribute instead of via a new intrinsic, but some compiler integration is needed.

Note that libc is a normal crate on crates.io. If we do this via an attribute, libc would need to use it, and it will need to become stable at some point. Making setjmp an intrinsic allows us to stabilize the intrinsic without stabilizing the attribute, but as you mention the attribute might be required by any code trying to wrap setjmp anyways..

LLVM also exposes llvm.setjmp/llvm.longjmp intrinsics, so it might be worth it to consider exposing those instead of manually linking to the system symbols.

And about longjump skipping over destructors [...] adding it as an unsafe function to libc doesn't cause any unsoundness,

Rust does not guarantee that destructors run, so skipping them isn't unsound (mem::forget is safe), just annoying for unsafe code that assume that it does not happen. My worry about longjmp is that it is not 100% clear to me which stack state it restores, e.g., could it restore the value of drop flags creating double drops? I don't know, but that would be unsound.

This is not a good example, but it has a double drop (playground):

fn foo() {
    let mut buf: jmp_buf = [0; 8];
    let a = A;
    if unsafe { setjmp(&mut buf) } != 0 {
        return; // drops a again (use-after-move/free, double-drop) => UB
    }
    std::mem::drop(a); // safe: moves and drops a 
    unsafe { longjmp(&mut buf, 1) };
}

Is it possible to construct a similar example without explicitly calling mem::drop(a) where a drop flag is modified before a longjmp and longjmp restores its value to non-dropping (or jumps to a path that does not check it), creating a double drop ?

EDIT: what the example above show is how setjmp and longjmp trump the borrow checker. The example above drops a twice because a is used after-move (after the longjmp a has been moved, but the user is able to use a again) but the borrow checker misses it. The compiler might not be even emitting drop flags here, but it should.

In any case, skipping destructors is fine, running them twice (or more) isn't.

[hanna-kruppe]

Note that libc is a normal crate on crates.io. If we do this via an attribute, libc would need to use it, and it will need to become stable at some point.

Oh, right, hm. Alternatively maybe these could be in the standard library, which sounds odd because they are a C-ism, but OTOH as you note having setjmp/longjump really does extend the language in magical, otherwise-impossible ways. In any case, since something new in rustc is needed and must be stabilized for libc to benefit, I agree that this is RFC territory.

LLVM also exposes llvm.setjmp/llvm.longjmp intrinsics, so it might be worth it to consider exposing those instead of manually linking to the system symbols.

I can't find any such intrinsics, there's only llvm.eh.sjlj.setjmp and llvm.eh.sjlj.longjmp which, as the name indicates, are for exception handling and thus appear to use at least a slightly different data format (it is defined in reference to a GCC built-in function, not the libc symbol). I also would not expect these to be portable, well supported or well maintained since SjLj EH is... not at all widely used.

In any case, skipping destructors is fine, running them twice (or more) isn't.

Good point about running them twice.

Wrt skipping: sorry, safe code can indeed skip destructors of values it owns, what I meant is that carefully written code that never gives up ownership of a value can be assured the value will be dropped unless other code diverges -- but notably it will still be dropped if unwinding happens, which matters because unwinding can be stopped. This assured drop is then used in a lot of libraries that run a user-specified closure to be able to "get in a word" after that closure finishes or unwinds -- e.g., to join a thread.

[gnzlbg]

@jeff-davis I think we should definitely encourage people to "play" with these intrinsics, and for interfacing with C Rust is going to need to provide some kind of support for these.

Would you be ok with adding them behind an experimental cargo feature ? (e.g. unstable_setjmp_longjmp) ? This allows us to test the platform specific details here in CI, and gives those wanting to play with them a semi reliable way to do so. This also allows us to move them to core::intrinsics at some point in the future without breaking stable users.

In the meantime, we should open an issue in rust-lang/rust about what to do about these. We'll probably need to write down what code that uses these is allowed to do to avoid undefined behavior and that's going to be hard.

How does that sound?

[gnzlbg]

So I've opened an issue in the RFC repo: rust-lang/rfcs#2625

@jeff-davis mentioned:

1. It would require a separate C wrapper for each C function I intend to call. For my use case, which is making it easier for people to write postgres extensions in rust, that would be wrapping hundreds of functions. In rust, I can use a macro, or wrap the function in rust, or do slightly different things for some functions when necessary.

2. It would be a separate build step in C with its own portability issues.

Even if we add setjmp to libc in this PR behind a feature gate, using it would be instantaneous undefined behavior, result in mis-optimizations, uses after move, etc. and there would be very little that you could do to avoid that.

The main advantage of writing a C wrapper over postgres that catches postgres exceptions and translates them to Rust error codes (where in Rust you can then raise them again as a panic if you want, or do something else) is that this approach would at least work and that there is experience from doing this same thing to C++ libraries that throw exceptions (we don't have tools in Rust to catch C++ exceptions, C++ thin wrappers that catch them and translate them to error codes need to be written here).

I understand that this solution is sub-optimal, but coming up with ways to make setjmp/longjmp "barely work" will take time.

[burdges]

Is there no way to abuse variable length arguments in C to create one C wrapper function that supports calling all the C functions?

[jeff-davis]

Thank you for the detailed replies. They still leave a few puzzles:

I translated your rust example to C, and I get:

jdavis@jdavis:~/code/c$ ./a.out
foo(): 13
jdavis@jdavis:~/code/c$ clang-6.0 -O2 setjmp2.c
jdavis@jdavis:~/code/c$ ./a.out
foo(): 42
jdavis@jdavis:~/code/c$ gcc setjmp2.c
jdavis@jdavis:~/code/c$ ./a.out
foo(): 13
jdavis@jdavis:~/code/c$ gcc -O2 setjmp2.c
jdavis@jdavis:~/code/c$ ./a.out
foo(): 42

So that means that either (a) the behavior of that pattern of use of setjmp is also undefined in C; or (b) both gcc and clang have the same correctness bug. I have to assume (a), because (b) seems quite unlikely.

The example would be more compelling if you showed a correctness bug in rust for well-defined behavior in C.

But, I agree that we at least should have the returns_twice attribute on setjmp(). I am fine making this an RFC.

[jeff-davis]

@gnzlbg: Yes, the experimental cargo feature sounds like a nice way to proceed for now. Thank you.

[gnzlbg]

I have to assume (a), because (b) seems quite unlikely.

In my translations, both GCC and Clang always return 13 =/

Maybe you forgot to make x volatile ? In C, per C 7.13.2.1p3:

All accessible objects have values, and all other components of the abstract machine249) have state, as of the time the longjmp function was called, except that the values of objects of automatic storage duration that are local to the function containing the invocation of the corresponding setjmp macro that do not have volatile-qualified type and have been changed between the setjmp invocation and longjmp call are indeterminate.

one must make x volatile because otherwise its value is indeterminate. AFAIK Rust does not have a concept of "indeterminate" values, so I don't know whether it makes sense to use that term in Rust.

Also, Rust does not have a "volatile" storage specifier, so x is ""indeterminate"" and there is not much that one can do about that. In this comment I mentioned how one can use volatile pointer load and stores to workaround this issue for that particular example (playground). However, that won't help you with the load / stores that the compiler inserts for code that you don't directly write though (e.g. if the compiler inserts a call to drop, and you use the value inside the Drop impl, loads from &mut self won't be a volatile load).

Basically, there is no way to write Rust code that's equivalent to the correct C code.

[jeff-davis]

I have to assume (a), because (b) seems quite unlikely.

In my translations, both GCC and Clang always return 13 =/

Oh, you're right. I forgot to mark x volatile.