core: introduce crossfile arrays (xfa) v3

[kaspar030]

Contribution description

This PR is the third iteration of Cross File Arrays (xfa), which allow typed and sorted arrays to be spread over multiple .c files.

This is basically a rebased version of #9105.

What can this be used for?

This could be useful to organize auto_init functions.

Currently, they're centrally managed in a single auto_init. file (well, multiple if auto_init_saul, _netif etc. should be counted).

With XFA's it is possible to add an initialization function to a global "init_functions" array, within the module's .c file:

 XFA_ADD_PTR(init_functions, 1234, ztimer, &ztimer_init)

auto_init.c could be reduced to this:

XFA_CONST_USE(init_function_t, init_functions);

void auto_init() {
    unsigned n = XFA_LEN(init_function_t, init_functions);
    for (unsigned i = 0; i < n; i++) {
		init_functions[i]();
    }
}

Compared to libc_init_array, XFA's support priority values. All entries get sorted by priority value, at link time.
In the above example, the ztimer init function was added with priority 100. Anything requiring ztimer would need to use a priority value that is higher than that.
The priority space is basically unlimited, the numbers get turned into symbol names, which the linker sorts. (note to self: see if that sort is numerical, or maybe 10 sorts before 2).

This is also not limited to functions. Other candidates for benefiting from non-central arrays are:

• shell commands
• saul drivers
• netif drivers
• threads that should started at boot time

Downsides

Apart from being a bit macro hacky, this currently requires disabling -Warray-bounds for files using XFA's (that include xfa.h).
Also, this needs linker script support. A generic xfa.ld is used unless overridden. Some architectures need slightly different handling due to different section names or linker symbols.

Testing procedure

As is, XFA's are not used anywhere other than in the test.
Ensure that test works, on all platforms.

Issues/PRs references

Taken over #9105.
Initially introduced in #7523.
Not needing ugly hacks since #14754.

[maribu]

Apart from being a bit macro hacky, this currently requires disabling -Warray-bounds for files using XFA's

Would it be possible to add a static inline helper function for accessing the arrays? If so, maybe some #pragma GCC diagnostics dance around that helper function would do?

[kaspar030]

Would it be possible to add a static inline helper function for accessing the arrays? If so, maybe some #pragma GCC diagnostics dance around that helper function would do?

it probably would. also, this could make it possible to not force avr to do this in completely RAM. It would not make it more ergonomic, I think. will need to try.

[kaspar030]

I don't manage to adapt the xfa.ld to the internal msp430 linker script. :( some help would be appreciated.

I see some options:

• someone with more ld-script-fu figures out how to do it
• we extract the msp430 ld scripts and add them to our repo, then add the xfa support there (there are like 5 different ones)
• we make xfa a feature for now, excluding msp430, making it much less usable

[maribu]

The ld-script-force is strong with @benpicco. Maybe he can help?

[benpicco]

I don't have the hardware to test, but when I drop d50d0b6 and apply this patch

diff --git a/makefiles/arch/msp430.inc.mk b/makefiles/arch/msp430.inc.mk
index 011e2778ea..2210919568 100644
--- a/makefiles/arch/msp430.inc.mk
+++ b/makefiles/arch/msp430.inc.mk
@@ -15,6 +15,7 @@ ASFLAGS += $(CFLAGS_CPU) --defsym $(CPU_MODEL)=1 $(CFLAGS_DBG)
 
 LINKFLAGS += $(CFLAGS_CPU) $(CFLAGS_DBG) $(CFLAGS_OPT)
 LINKFLAGS += -Wl,--gc-sections -Wl,-L$(MSP430_SUPPORT_FILES)/include
+LINKFLAGS += -T $(MSP430_SUPPORT_FILES)/include/$(CPU_MODEL).ld
 
 OPTIONAL_CFLAGS_BLACKLIST += -fdiagnostics-color
 OPTIONAL_CFLAGS_BLACKLIST += -Wformat-overflow

it will at least compile for msp430

[jia200x]

I don't have the hardware to test, but when I drop d50d0b6 and apply this patch

I have a z1 lying around. I will try it

[kaspar030]

Seems like the riotdocker llvm is too old for the static test:

llvm-objdump: Unknown command line argument '--syms'.  Try: '/usr/bin/llvm-objdump -help'
llvm-objdump: Did you mean '-dsym'?
llvm-objdump: Unknown command line argument '--syms'.  Try: '/usr/bin/llvm-objdump -help'
llvm-objdump: Did you mean '-dsym'?
Error: Static check of XFA linked const array failed, verify linker flags and try again

The test succeeds with llvm 11, but not with the one in our build container.

[kaspar030]

@kaspar030 Murdock is idle, good time to address those last couple of nitpicks ;)

done & squashed

[fjmolinas]

ACK!

[kaspar030]

Thanks a lot everyone involved and especially @fjmolinas for reviewing, and for keeping me looking at this!

[miri64]

🎉