{.byref,exportc.} types are not output into --header file

[exelotl]

When an object type is made available to C code via {.exportc.} in combination with the --header option, the full type definition doesn't appear in the generated <project>.h. (unless the type is small, or is explicitly annotated with {.bycopy.})

Previously this worked fine as long as you also had an {.exportc.}'d procedure that made use of the type, see #7448 (comment).

Example

Compile with nim c -r --nimcache:nimcache --cincludes:nimcache --header main.nim

# main.nim
type
  Foo* {.exportc.} = object
    a*, b*, c*, d*: int

proc dummy(): Foo {.exportc.} = discard

{.compile:"bar.c".}

// bar.c
#include "main.h"

const Foo f = {10, 20, 30, 40};

Current Output

Hint: used config file 'C:\Users\exelotl\Dev\nim-current\config\nim.cfg' [Conf]
Hint: used config file 'C:\Users\exelotl\Dev\nim-current\config\config.nims' [Conf]
.........................................................
CC: bar
CC: stdlib_digitsutils.nim
CC: stdlib_assertions.nim
CC: stdlib_dollars.nim
CC: stdlib_io.nim
CC: stdlib_system.nim
CC: main.nim
C:\Users\exelotl\Dev\nim-tests\exportc-type\bar.c:3:1: error: variable 'f' has initializer but incomplete type
    3 | const Foo f = {10, 20, 30, 40};
      | ^~~~~
C:\Users\exelotl\Dev\nim-tests\exportc-type\bar.c:3:11: error: storage size of 'f' isn't known
    3 | const Foo f = {10, 20, 30, 40};
      |           ^
Error: execution of an external compiler program 'C:\Users\exelotl\Dev\nim-1.6.2\dist\mingw64\bin\gcc.exe -c  -w -fmax-errors=3 -mno-ms-bitfields   -IC:\Users\exelotl\Dev\nim-current\lib -IC:\Users\exelotl\Dev\nim-tests\exportc-type\nimcache -IC:\Users\exelotl\Dev\nim-tests\exportc-type -o C:\Users\exelotl\Dev\nim-tests\exportc-type\nimcache\bar.c.o C:\Users\exelotl\Dev\nim-tests\exportc-type\bar.c' failed with exit code: 1

The generated main.h:

/* Generated by Nim Compiler v1.6.2 */
#ifndef __main__
#define __main__
#define NIM_INTBITS 64

#include "nimbase.h"
#undef LANGUAGE_C
#undef MIPSEB
#undef MIPSEL
#undef PPC
#undef R3000
#undef R4000
#undef i386
#undef linux
#undef mips
#undef near
#undef far
#undef powerpc
#undef unix
typedef struct Foo Foo;                             //  <---- Type is declared but not defined.
N_LIB_PRIVATE N_NOCONV(void, signalHandler)(int sign);
N_LIB_PRIVATE N_NIMCALL(NI, getRefcount)(void* p);
N_LIB_PRIVATE N_NIMCALL(void, printFoo)(Foo* f);
N_CDECL(void, NimMain)(void);
#endif /* __main__ */

Expected Output

Program should compile.

Additional Information

The issue occurs on 1.6.2 and the 1.6.4 RC, but not on 1.6.0 or earlier.

$ nim -v
Nim Compiler Version 1.6.2 [Windows: amd64]
Compiled at 2021-12-17

[exelotl]

Note: in case you're wondering why I'm using the --header feature like this, it's so that I can describe my types in Nim while the data itself is C. This is currently the most straightforward way I've found to ensure my data (which contains pointers and should itself be addressable) goes in ROM, without having to write out the types twice (in C and in Nim). See also: nim-lang/RFCs#257 (comment)

I could potentially work around this by generating data as assembly instead of C, but that comes with its own issues as I'd lose a layer of protection with the C compiler no longer being able to check that the data matches the type, and I'd have to generate different assembly when compiling the project for different platforms, etc.

[Araq]

It's only a problem since it's a regression as --header is not supported.

[ringabout]

I will try git bisect.

[ringabout]

Hello, I cannot compile the program with Nim 1.4.8, Nim 1.6.0 (same errors as 1.6.2 and devel)?

Anything I did wrong?

nim c -r --nimcache:nimcache --cincludes:nimcache --header main.nim

nim -v
Nim Compiler Version 1.6.0 [Windows: amd64]
Compiled at 2021-10-19
Copyright (c) 2006-2021 by Andreas Rumpf

CC: bar
C:\Users\blue\Downloads\Nim\bar.c:4:1: error: variable 'f' has initializer but incomplete type
    4 | const Foo f = {10, 20, 30, 40};
      | ^~~~~
C:\Users\blue\Downloads\Nim\bar.c:4:11: error: storage size of 'f' isn't known
      |           ^

[exelotl]

Huh, you're right, I thought I tested this pretty thoroughly but clearly my example failed to capture the regression. Apologies.

I've updated the example slightly (changed printFoo to a dummy which returns Foo) and am now getting the issue as I described.

Honestly though, don't let this block the 1.6.4 release. It's not that important.

1.6.0:

$ nim -v
Nim Compiler Version 1.6.0 [Windows: amd64]
Compiled at 2021-10-19
Copyright (c) 2006-2021 by Andreas Rumpf

active boot switches: -d:release

$ rm -rf nimcache/ main.exe && nim c -r -f --nimcache:nimcache --cincludes:nimcache --header main.nim
Hint: used config file 'C:\Users\exelotl\Dev\nim-current\config\nim.cfg' [Conf]
Hint: used config file 'C:\Users\exelotl\Dev\nim-current\config\config.nims' [Conf]
.........................................................
CC: bar
CC: stdlib_digitsutils.nim
CC: stdlib_assertions.nim
CC: stdlib_dollars.nim
CC: stdlib_io.nim
CC: stdlib_system.nim
CC: main.nim
Hint:  [Link]
Hint: gc: refc; opt: none (DEBUG BUILD, `-d:release` generates faster code)
26571 lines; 2.985s; 31.609MiB peakmem; proj: C:\Users\exelotl\Dev\nim-tests\exportc-type\main.nim; out: C:\Users\exelotl\Dev\nim-tests\exportc-type\main.exe [SuccessX]
Hint: C:\Users\exelotl\Dev\nim-tests\exportc-type\main.exe  [Exec]

1.6.2:

$ nim -v
Nim Compiler Version 1.6.2 [Windows: amd64]
Compiled at 2021-12-17
Copyright (c) 2006-2021 by Andreas Rumpf   

active boot switches: -d:release

$ rm -rf nimcache/ main.exe && nim c -r -f --nimcache:nimcache --cincludes:nimcache --header main.nim
Hint: used config file 'C:\Users\exelotl\Dev\nim-current\config\nim.cfg' [Conf]    
Hint: used config file 'C:\Users\exelotl\Dev\nim-current\config\config.nims' [Conf]
.........................................................
CC: bar
CC: stdlib_digitsutils.nim
CC: stdlib_assertions.nim
CC: stdlib_dollars.nim   
CC: stdlib_io.nim    
CC: stdlib_system.nim
CC: main.nim
C:\Users\exelotl\Dev\nim-tests\exportc-type\bar.c:3:1: error: variable 'f' has initializer but incomplete type
    3 | const Foo f = {10, 20, 30, 40};
      | ^~~~~
C:\Users\exelotl\Dev\nim-tests\exportc-type\bar.c:3:11: error: storage size of 'f' isn't known
    3 | const Foo f = {10, 20, 30, 40};
      |           ^
Error: execution of an external compiler program 'C:\Users\exelotl\Dev\nim-1.6.2\dist\mingw64\bin\gcc.exe -c  -w -fmax-errors=3 -mno-ms-bitfields   -IC:\Users\exelotl\Dev\nim-current\lib -IC:\Users\exelotl\Dev\nim-tests\exportc-type\nimcache -IC:\Users\exelotl\Dev\nim-tests\exportc-type -o C:\Users\exelotl\Dev\nim-tests\exportc-type\nimcache\bar.c.o C:\Users\exelotl\Dev\nim-tests\exportc-type\bar.c' failed with exit code: 1

[ringabout]

I found the culprit and will push a fix soon.