Windows: Make -fvisibility=public export all defined symbols and import external data

[kinke]

To pave the way for simple DLL generation (end goal: druntime-ldc-shared.dll).

The dllexport storage class for functions definitions is enough; the automatically generated import .lib seems to resolve the regular symbol name to the actual symbol (__imp_*), so dllimport for declarations seems superfluous.

For global variables, things are apparently different unfortunately.

[kinke]

@rainers: Do you happen to have any idea about the data symbols?

[kinke]

Okay, non-TLS globals work in some scenarios now. Trying to access TLS globals across DLL boundaries yields a linking error; both MSVC and clang don't support them in C++ and fail at compile-time:

extern __declspec(dllimport) thread_local int impGlobal;

=>

error C2492: 'impGlobal': data with thread storage duration may not have dll interface

[rainers]

@rainers: Do you happen to have any idea about the data symbols?

I guess you are aware of Benjamin Thaut's work, see for example https://forum.dlang.org/thread/shonvwuauwbhrszysmrz@forum.dlang.org

I agree that an "export all" switch avoids much of the hassle of his DIP. It's also what happens on linux, and there haven't really been any complaints recently AFAICT. It's also what I implemented 11 years ago: https://issues.dlang.org/show_bug.cgi?id=4071 ;-)

dllimported data needs different code to access it (IIRC dmd has support for that, probably not too hard to add the extra indirection).

The more problematic case is imported data referenced from preinitialized data. This needs an additional relocation pass at EXE/DLL initialization time using compiler generated info: https://github.com/Ingrater/druntime/blob/DllSupport80/src/rt/sections_win64.d#L432

An alternative might be to emit initializer functions. IIRC that is what MSVC does, though your comment above suggests it is not supported (anymore?).

[kinke]

Thx Rainer.

The more problematic case is imported data referenced from preinitialized data.

Yep exactly, this seems to account for most Phobos linking failures in #3704. It's especially bad for generated TypeInfos, because they reference e.g. the TypeInfo_Class vtable from druntime and builtin TypeInfos defined in rt.util.typeinfo.
With LLVM here, using dllimport makes no difference for the generated assembly, just like for TLS variables - so if the dllimported global isn't defined in the same DLL/exe, linking errors result. I guess for Linux and Mac this is all done automatically, but Windows would either need duplicate definitions in each DLL (well, wouldn't work in all cases, but could work for builtin TypeInfos) or some form of runtime patching.

Living with no direct TLS globals access across DLLs would IMO be doable, but not being able to access non-TLS globals in initializers for static data in other DLLs seems like the biggest hurdle for DLLs on Windows indeed.

[kinke]

I've added an LLVM pass performing the manual dllimport 'relocation'. The release phobos2-ldc-shared.dll can now be linked successfully against the druntime DLL; debug Phobos still has 1 undefined symbol, a TLS global (core.exception._store; referenced by a staticError instantiation in Phobos).

[rainers]

I've added an LLVM pass performing the manual dllimport 'relocation'.

Having it a in a separate pass is pretty neat :-) For tls-variables, you could export an accessor-function instead that returns a reference/pointer to the variable.

[kinke]

Having it a in a separate pass is pretty neat :-)

:) Yeah, LLVM's rich/heavy IR does have advantages...

For tls-variables, you could export an accessor-function instead that returns a reference/pointer to the variable.

I hope such direct accesses are sufficiently rare that manual workarounds like ldc-developers/druntime@f23246f will do for the/some time being. That was sufficient for the druntime/Phobos DLL boundary; linking all druntime and Phobos DLL variants now works, incl. linking the shared test runner executables (see here).

That should be mostly it from the compiler side, and work can begin on making druntime DLL a truly shared druntime analogous to Posix' _d_dso_registry system, incl. a list of TLS blocks per DLL/executable for each thread. Instead of the CRT ctor/dtor pair in each object file (for the _d_dso_registry() calls, used for Posix), I'd tend towards precompiling a new separate little module and linking that object file automatically with -link-defaultlib-shared on Windows.