Fatal python error: do_realize_lazy_struct: lost a struct/union!

[lacraig2]

I am attempting to use cffi in out-of-line ABI level.

I am preprocessing my source and simplifying it to the point where cdef can understand its structs and functions.

The compiling to a module seems to work fine, but when I try to instantiate new type I get the following error:

Fatal Python error: do_realize_lazy_struct: lost a struct/union!
Python runtime state: initialized

Current thread 0x00007fd97c46e740 (most recent call first):
  File "/run.py", line 3 in <module>

Extension modules: _cffi_backend (total: 1)
Aborted (core dumped)

This is the stack trace:

#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140737346398016) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140737346398016) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=140737346398016, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x00007ffff79db476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007ffff79c17f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0x00007ffff7c4cec7 in fatal_error_exit (status=-1) at Python/pylifecycle.c:2553
#6  fatal_error (fd=<optimized out>, header=<optimized out>, prefix=<optimized out>, msg=<optimized out>, status=-1) at Python/pylifecycle.c:2664
#7  0x00007ffff7df9dd9 in _Py_FatalErrorFunc (func=func@entry=0x7ffff7281830 <__func__.39> "do_realize_lazy_struct", msg=msg@entry=0x7ffff7280908 "lost a struct/union!") at Python/pylifecycle.c:2750
#8  0x00007ffff7274316 in do_realize_lazy_struct (ct=ct@entry=0x7ffff746fb50) at src/c/realize_c_type.c:727
#9  0x00007ffff72746aa in _realize_c_struct_or_union (builder=builder@entry=0x7ffff72a1138, sindex=<optimized out>) at src/c/realize_c_type.c:406

Some additional context:

cffi/src/c/realize_c_type.c

Lines 725 to 727 in b57a92c

	n = search_in_struct_unions(&builder->ctx, p, strlen(p));
	if (n < 0)
	Py_FatalError("lost a struct/union!");

The lookup name here seems to be $FILE.

I've minimized this to error to:

from libx import ffi

ffi.new("struct _IO_FILE*")

[lacraig2]

I've found a workaround, but I would still like to know if this is correct.

I was able to add the following to ensure that struct _IO_FILE is never defined in stdio.h:

	typedef void* FILE;
	#define __struct_FILE_defined 1

[arigo]

Can you include a reproducible example? The missing information is how you declare your libx module: the cdef() that you applied when building it.

[arigo]

Closing. Feel free to re-open if you have more information!

[lacraig2]

@arigo Sorry for the delay. I don't have permission to re-open the issue.

I put everything required to reproduce things here: https://github.com/lacraig2/repro_cffi

You should only need to run run.py. It'll build a basic so file (without dependencies) because we don't actually get to the point where we use it and otherwise it's harder to make available.

It is building a header file I have committed through a fairly complex process here, but that's not important. The important thing is reproducing and fixing the FILE issue.

[arigo]

The problem likely comes from struct _IO_FILE being redefined explicitly inside the header passed to ffibuilder.cdef(). There is some special-casing to predefine the FILE type for you. I never expected the user to redefine what FILE is. As a quick workaround you can try to remove the struct _IO_FILE { ... }; part. It's still something to be fixed, certainly.

[arigo]

(Ah, I just realized that it's what you said in #149 (comment). Sorry! That comment was correct, it's a viable workaround. I just didn't understand at that time that you'd put these two lines somewhere in the process that produces the .h file that you then feed to ffi_builder.cdef(). That's certainly one way to use CFFI, but not the only way.)