runtime: throw in linked c++ library causes app crash even if caught, on windows platform

[superbaddude]

when compiling go code using cgo to link a c++ application that uses try/catch/throw logic, the golang app crashes immediately when an exception is thrown. on linux, it performs as expected.

code versions:
go version go1.5 windows/amd64
tdm64-gcc-5.1.0-2

the code that demonstrates issue can be found in https://gist.github.com/superbaddude/b52e60f2e8dee0868af2

the output from command on windows:

go test

=== RUN   Test_Callthrow_and_catch_exception
Exception 0x20474343 0x1045e80 0x6161616161616161 0x7fff56cda1c8
PC=0x7fff56cda1c8
signal arrived during external code execution

github.com/superbaddude/testcgo._Cfunc_throw_and_catch_exception()
    github.com/superbaddude/testcgo/_test/_obj_test/_cgo_gotypes.go:37 +0x38
github.com/superbaddude/testcgo.Callthrow_and_catch_exception()
    c:/Code/go/src/github.com/superbaddude/testcgo/throwe.go:9 +0x1b
github.com/superbaddude/testcgo.Test_Callthrow_and_catch_exception(0xc08208e000)
    c:/Code/go/src/github.com/superbaddude/testcgo/throwe_test.go:6 +0x1b
testing.tRunner(0xc08208e000, 0x702d00)
    c:/go/src/testing/testing.go:456 +0x9f
created by testing.RunTests
    c:/go/src/testing/testing.go:561 +0x874

goroutine 1 [chan receive]:
testing.RunTests(0x5e4f28, 0x702d00, 0x1, 0x1, 0x1)
    c:/go/src/testing/testing.go:562 +0x8b4
testing.(*M).Run(0xc082033ee8, 0xc0820583d0)
    c:/go/src/testing/testing.go:494 +0x77
main.main()
    github.com/superbaddude/testcgo/_test/_testmain.go:54 +0x11d

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
    c:/go/src/runtime/asm_amd64.s:1696 +0x1
rax     0x0
rbx     0x1045e80
rcx     0x0
rdi     0x1045ec0
rsi     0x6d6850
rbp     0x0
rsp     0x24fc00
r8      0x10
r9      0x8735d0
r10     0x1
r11     0x3
r12     0x8
r13     0x5e2c36
r14     0x9
r15     0x8
rip     0x7fff56cda1c8
rflags  0x206
cs      0x33
fs      0x53
gs      0x2b
exit status 2
FAIL    github.com/superbaddude/testcgo 0.914s

[alexbrainman]

The crash that you see is Go exception handler in action - exception been raised in the code outside of Go, so Go runtime wouldn't know how to handle it, so it crashes your program.

To accommodate your request, Go exception handler needs to change to co-exist with gcc runtime. I don't know anything about gcc exception handler, so I wouldn't even know if it is possible. Sorry.

Alex

[superbaddude]

i'd suggest updating the golang cgo documentation to explain this limitation on linking external libraries written in c++ on windows. i'd hope to save some time for the next person who encounters this issue.

[alexbrainman]

@superbaddude thank you for suggestion. But I am not sure we should document the bug. Like I said earlier, I don't know how to fix this. But perhaps it can be fixed.

Alex

[superbaddude]

cool, a fix would be much better, especially given the compatibility goals for golang. thanks for the update.

[ianlancetaylor]

I took a look at this. I can throw and catch an exception in a constructor in cgo code. However, I can not throw and catch an exception in cgo code after Go code starts. This is true even if I comment out the call to initExceptionHandler in osinit, so it doesn't have anything to do with the exception handlers that the Go runtime installs. The throw starts, but it is never caught. There is something about the Go runtime, or having Go in the call stack, that causes Windows to be unable to find the installed SEH handler, even when the handler is in the same frame as the throw. I've verified that we are using the external linker.

I do not know how to fix this.

[alexbrainman]

There is something about the Go runtime, or having Go in the call stack, that causes Windows to be unable to find the installed SEH handler, even when the handler is in the same frame as the throw.

We don't use SEH any more. We used to (on windows-386). Now we use single global exception handler (it is installed by calling AddVectoredExceptionHandler). The global exception handler is quite simple: it handles just a few scenarios that Go runtime can recover from (division by zero, int overflow and some others), but otherwise it lets outside code handle the situation (by returning EXCEPTION_CONTINUE_SEARCH from exception handler). It appears that gcc generates code for try/catch in a such way that it cannot handle this scenario.

I don't see how I can help here without understanding what gcc requires for this to work. And I don't. I am sorry.

Alex

[ianlancetaylor]

I understand that Go does not use SEH, but the C++ code, compiled by the C++ compiler, does use SEH. Windows finds the Go exception handlers fine. It fails to find the C++ SEH exception handler, but only after the Go runtime has started, or perhaps only when called from Go. I don't know what the key difference is.

[alexbrainman]

... the C++ code, compiled by the C++ compiler, does use SEH.

As far as I know SEH is not used on Windows 64-bit (windows-amd64 in Go speak). See http://www.ntcore.com/files/vista_x64.htm#Exception_Handling

Also http://blogs.msdn.com/b/freik/archive/2006/01/04/509372.aspx provides some details about exception handling on Windows 64-bit

Windows finds the Go exception handlers fine.

We use standard AddVectoredExceptionHandler Windows API to inform Windows about our handler.

It fails to find the C++ SEH exception handler, but only after the Go runtime has started, or perhaps only when called from Go. I don't know what the key difference is.

If you read the articles above, you will see that Windows requires every stack frame to contain apropriate unwind information. Go does not generate any of that. I am not sure about gcc C++. Maybe gcc generates appropriate unwind information. Maybe Windows requires that full stack has unwind information. I don't know.

Alex

[minux]

Some observations:

1. The exception number (0x20474343) is the one used
   by g++ runtime for g++ exceptions.
2. I changed to C++ function to throw (and catch) an int,
   and the program runs fine.
3. However, if I change the C++ function to throw (and
   catch) new int (or anything that requires memory allocation
   it seems), the program crashes.

[ianlancetaylor]

Windows 64-bit still effectively uses SEH, it just stores the data in the .pdata/.xdata sections rather than generating it at runtime. Anyhow, whether we call it SEH or not doesn't matter; exception information does exist, and GCC generates exception information as Windows expects.

[ianlancetaylor]

@minux Thanks for noticing that. I didn't notice it because when I tried changing the throw statement I also had some iostream output in there. The iostream output before the throw seems to be enough to make the exception lookup fail.

[DemiMarie]

The problem is that the Windows ABI requires that ALL frames through which an exception may propagate have stack unwind information. The only functions that do not require it are functions from which no SEH exception may ever be thrown. Transitively. That means that ANY function that may call into C needs unwind info.

One alternative is to only call into C via an assembler trampoline that ensures no Go code is on the stack when C code is executing.

[minux]

On Apr 7, 2016 5:09 PM, "Demetri Obenour" notifications@github.com wrote:

The problem is that the Windows ABI requires that ALL frames through
which an exception may propagate have stack unwind information. The only
functions that do not require it are functions from which no SEH exception
may ever be thrown. Transitively. That means that ANY function that may
call into C needs unwind info.

One alternative is to only call into C via an assembler trampoline that
ensures no Go code is on the stack when C code is executing.

We already do that. Go code and C code are running on different stacks.
Stack unwind won't work because there is no way to describe unwinding
across two stacks.

[aclements]

Ping @alexbrainman, @ianlancetaylor. Any thoughts on this?

[ianlancetaylor]

I spent some time looking into this over a year ago, but could not figure it out. Something about the Go runtime is causing Windows to fail to do exception lookup as documented, but I don't know what it is. I'm going to unassign myself, as I am not going to be looking into this. I think this needs to go to someone with patience and good access to a Windows machine.

[alexbrainman]

Any thoughts on this?

I do not know much about this subject. There is a lot of guessing in my reply.

Go uses AddVectoredExceptionHandler and AddVectoredContinueHandler to handle exceptions (see runtime.initExceptionHandler). These provide global exception handler - one handler function deals with exception anywhere in the program.

On the other hand c++ runtime provides local exception handlers. These are called by Windows in accordance with exception information generated as part of executable generation. https://msdn.microsoft.com/en-us/library/1eyas8tf(v=vs.100).aspx is the only description I could find. And this is a general description. I have no idea what gcc generates. But I can see, for example, .pdata section:

d:\a\src\issues\issue12516>objdump -h issue12516.test.exe

issue12516.test.exe:     file format pei-x86-64

Sections:
Idx Name          Size      VMA               LMA               File off  Algn
  0 .text         00206508  0000000000401000  0000000000401000  00000600  2**5
                  CONTENTS, ALLOC, LOAD, READONLY, CODE, DATA
  1 .data         000285c0  0000000000608000  0000000000608000  00206c00  2**6
                  CONTENTS, ALLOC, LOAD, DATA
  2 .rdata        00004540  0000000000631000  0000000000631000  0022f200  2**6
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  3 .pdata        000017dc  0000000000636000  0000000000636000  00233800  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  4 .xdata        000019c0  0000000000638000  0000000000638000  00235000  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  5 .bss          00023e00  000000000063a000  000000000063a000  00000000  2**6
                  ALLOC
  6 .idata        00001034  000000000065e000  000000000065e000  00236a00  2**2
                  CONTENTS, ALLOC, LOAD, DATA
  7 .CRT          00000068  0000000000660000  0000000000660000  00237c00  2**3
                  CONTENTS, ALLOC, LOAD, DATA
  8 .tls          00000068  0000000000661000  0000000000661000  00237e00  2**5
                  CONTENTS, ALLOC, LOAD, DATA
  9 .debug_aranges 00000290  0000000000662000  0000000000662000  00238000  2**4
                  CONTENTS, READONLY, DEBUGGING
 10 .debug_pubnames 0000c604  0000000000663000  0000000000663000  00238400  2**0
                  CONTENTS, READONLY, DEBUGGING
 11 .debug_pubtypes 0000e4e2  0000000000670000  0000000000670000  00244c00  2**0
                  CONTENTS, READONLY, DEBUGGING
 12 .debug_info   00077993  000000000067f000  000000000067f000  00253200  2**0
                  CONTENTS, READONLY, DEBUGGING
 13 .debug_abbrev 00001901  00000000006f7000  00000000006f7000  002cac00  2**0
                  CONTENTS, READONLY, DEBUGGING
 14 .debug_line   0001bc42  00000000006f9000  00000000006f9000  002cc600  2**0
                  CONTENTS, READONLY, DEBUGGING
 15 .debug_frame  0001dd10  0000000000715000  0000000000715000  002e8400  2**3
                  CONTENTS, READONLY, DEBUGGING
 16 .debug_str    00000173  0000000000733000  0000000000733000  00306200  2**0
                  CONTENTS, READONLY, DEBUGGING
 17 .debug_loc    000027cc  0000000000734000  0000000000734000  00306400  2**0
                  CONTENTS, READONLY, DEBUGGING
 18 .debug_ranges 000003c0  0000000000737000  0000000000737000  00308c00  2**0
                  CONTENTS, READONLY, DEBUGGING
 19 .debug_gdb_scripts 00000024  0000000000738000  0000000000738000  00309000  2**0
                  CONTENTS, READONLY, DEBUGGING

that pecoff.doc mentions:

The .pdata section contains an array of function table entries used for exception handling and is pointed to by the exception table entry in the image data directory.

Note that .pdata will probably contains all required unwind information for C++ functions, I doubt it contains any for Go functions. Simply because we don't emit any of that.

I also suspect our code that manipulates stacks might be a problem here.

It is big can of worms as far as I am concerned. I am not working on this.

Hope it helps.

Alex

[DemiMarie]

Win64 ABI requires frame unwind information. Windows needs to be able to unwind all the way back to the function pointer passed to CreateThread.

…

On Jun 19, 2017 2:42 AM, "Alex Brainman" ***@***.***> wrote: Any thoughts on this? I do not know much about this subject. There is a lot of guessing in my reply. Go uses AddVectoredExceptionHandler and AddVectoredContinueHandler to handle exceptions (see runtime.initExceptionHandler). These provide global exception handler - one handler function deals with exception anywhere in the program. On the other hand c++ runtime provides local exception handlers. These are called by Windows in accordance with exception information generated as part of executable generation. https://msdn.microsoft.com/en- us/library/1eyas8tf(v=vs.100).aspx is the only description I could find. And this is a general description. I have no idea what gcc generates. But I can see, for example, .pdata section: d:\a\src\issues\issue12516>objdump -h issue12516.test.exe issue12516.test.exe: file format pei-x86-64 Sections: Idx Name Size VMA LMA File off Algn 0 .text 00206508 0000000000401000 0000000000401000 00000600 2**5 CONTENTS, ALLOC, LOAD, READONLY, CODE, DATA 1 .data 000285c0 0000000000608000 0000000000608000 00206c00 2**6 CONTENTS, ALLOC, LOAD, DATA 2 .rdata 00004540 0000000000631000 0000000000631000 0022f200 2**6 CONTENTS, ALLOC, LOAD, READONLY, DATA 3 .pdata 000017dc 0000000000636000 0000000000636000 00233800 2**2 CONTENTS, ALLOC, LOAD, READONLY, DATA 4 .xdata 000019c0 0000000000638000 0000000000638000 00235000 2**2 CONTENTS, ALLOC, LOAD, READONLY, DATA 5 .bss 00023e00 000000000063a000 000000000063a000 00000000 2**6 ALLOC 6 .idata 00001034 000000000065e000 000000000065e000 00236a00 2**2 CONTENTS, ALLOC, LOAD, DATA 7 .CRT 00000068 0000000000660000 0000000000660000 00237c00 2**3 CONTENTS, ALLOC, LOAD, DATA 8 .tls 00000068 0000000000661000 0000000000661000 00237e00 2**5 CONTENTS, ALLOC, LOAD, DATA 9 .debug_aranges 00000290 0000000000662000 0000000000662000 00238000 2**4 CONTENTS, READONLY, DEBUGGING 10 .debug_pubnames 0000c604 0000000000663000 0000000000663000 00238400 2**0 CONTENTS, READONLY, DEBUGGING 11 .debug_pubtypes 0000e4e2 0000000000670000 0000000000670000 00244c00 2**0 CONTENTS, READONLY, DEBUGGING 12 .debug_info 00077993 000000000067f000 000000000067f000 00253200 2**0 CONTENTS, READONLY, DEBUGGING 13 .debug_abbrev 00001901 00000000006f7000 00000000006f7000 002cac00 2**0 CONTENTS, READONLY, DEBUGGING 14 .debug_line 0001bc42 00000000006f9000 00000000006f9000 002cc600 2**0 CONTENTS, READONLY, DEBUGGING 15 .debug_frame 0001dd10 0000000000715000 0000000000715000 002e8400 2**3 CONTENTS, READONLY, DEBUGGING 16 .debug_str 00000173 0000000000733000 0000000000733000 00306200 2**0 CONTENTS, READONLY, DEBUGGING 17 .debug_loc 000027cc 0000000000734000 0000000000734000 00306400 2**0 CONTENTS, READONLY, DEBUGGING 18 .debug_ranges 000003c0 0000000000737000 0000000000737000 00308c00 2**0 CONTENTS, READONLY, DEBUGGING 19 .debug_gdb_scripts 00000024 0000000000738000 0000000000738000 00309000 2**0 CONTENTS, READONLY, DEBUGGING that pecoff.doc mentions: The .pdata section contains an array of function table entries used for exception handling and is pointed to by the exception table entry in the image data directory. Note that .pdata will probably contains all required unwind information for C++ functions, I doubt it contains any for Go functions. Simply because we don't emit any of that. I also suspect our code that manipulates stacks might be a problem here. It is big can of worms as far as I am concerned. I am not working on this. Hope it helps. Alex — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#12516 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGGWB63AJMJBqM7TDsHexBSegvK5ogTHks5sFhhygaJpZM4F4eeo> .

[gm42]

I think this is relevant here: https://stackoverflow.com/a/37371025

Whether -static-libgcc or not is being used might affect the results.

[goomario]

I use AddVectoredExceptionHandler and try {} catch {} resolved.

Code:

auto handle = AddVectoredExceptionHandler(1, callback);
try {
    callThroableFunc();
} catch(std::exception &e) {}
RemoveVectoredExceptionHandler(handle);
 

[fumin]

AddVectoredExceptionHandler

Can confirm with hybridgroup/gocv#274 (comment)

[bloeys]

Thanks for mentioning VectoredExceptionHandlers it really helped me solve this issue! (tested go1.18)

In case it might help someone, what I had is VectoredExceptionHandlers alone wasn't enough. The handler was being called but no matter what I returned from it the Go handler was eventually being called and crashing.

I had to do the following:

void doWork() {
        auto exceptionHandlerHandle = AddVectoredExceptionHandler(1, VectoredExceptionHandler);
	auto continueHandlerHandle = AddVectoredContinueHandler(1, VectoredContinueHandler);

        funcThatThrows();

	RemoveVectoredExceptionHandler(exceptionHandlerHandle);
	RemoveVectoredContinueHandler(continueHandlerHandle);
}

And my handlers were:

#include <Windows.h>
#include <errhandlingapi.h>

// This is the only exception code I wanted to handle. For different codes
// I return EXCEPTION_CONTINUE_SEARCH which causes the Go handler to get called.
#define NS_MS_VC_EXCEPTION 0x406D1388

// This gets called first
LONG VectoredExceptionHandler(_EXCEPTION_POINTERS* exceptionInfo)
{
	if (exceptionInfo->ExceptionRecord->ExceptionCode == NS_MS_VC_EXCEPTION) {
		return EXCEPTION_CONTINUE_EXECUTION;
	}

	return EXCEPTION_CONTINUE_SEARCH;
}

// This gets called if the exception handler returns EXCEPTION_CONTINUE_EXECUTION
LONG VectoredContinueHandler(_EXCEPTION_POINTERS* exceptionInfo)
{
	if (exceptionInfo->ExceptionRecord->ExceptionCode == NS_MS_VC_EXCEPTION) {
		return EXCEPTION_CONTINUE_EXECUTION;
	}

	return EXCEPTION_CONTINUE_SEARCH;
}

[DemiMarie]

The problem is that the Windows ABI requires that ALL frames through
which an exception may propagate have stack unwind information. The only
functions that do not require it are functions from which no SEH exception
may ever be thrown. Transitively. That means that ANY function that may
call into C needs unwind info.
One alternative is to only call into C via an assembler trampoline that
ensures no Go code is on the stack when C code is executing.

We already do that. Go code and C code are running on different stacks. Stack unwind won't work because there is no way to describe unwinding across two stacks.

Could Go pretend that the Go stack was simply not present, and that whatever code switches to the Go stack appears to be the callee of the code that switches from it?

[gopherbot]

Change https://go.dev/cl/457875 mentions this issue: runtime,cmd/link: allow SEH tramps handle non-Go exception