Introduce new relocation for landing pad

[kito-cheng]

The R_RISCV_LPAD relocation can be used for PLT entry generation and also for
linker relaxation. Additionally, we defined a new mapping symbol type to help
users understand the function signature for the corresponding function.

The addend value is the label value, and it will point to the mapping symbol
placed at the beginning of the function.

e.g.

foo:         # void foo(void)
$sFvvE:
    lpad 123 # R_RISCV_LPAD $sFvvE + 123

We propose two linker relaxations for the landing pad. The first is removing
the entire landing pad, which can be used when symbols have local visibility,
and the address is not taken by any other reference. The second is a landing
pad scheme conversion, designed for backward compatibility (or as a workaround)
for legacy programs that may use functions without declarations.

---

NOTE: This is based on #434

[kito-cheng]

cc @deepak0414 @ved-rivos @mylai-mtk

[aswaterman]

I think this reloc also provides enough info for the linker to fix up direct jumps to skip over landing pads when the landing pad was not relaxed.

Consider I have call foo, and the linker sees that the target of that call still has an R_RISCV_LPAD reloc (i.e. not relaxed). The linker should be able to adjust call foo to call foo+4. This should work whether the target is the PLT or the real function.

[deepak0414]

Q: what happens in case of dynamic linking ?

[kito-cheng]

I gave it some more thought, and this can actually be applied beyond just static linking. However, dependent shared libraries won’t automatically convert along with it. I’ve removed that limitation and added a NOTE to explain the situation.

[rui314]

I think we can explain the behavior more clearly if we avoid mentioning symbol visibility. The only important thing here is whether or not a symbol is visible to other ELF modules, i.e. whether or not the symbol is in the dynamic symbol table. Symbol visibility is just one way to control it.

[kito-cheng]

Good suggestion, applied 02546de :)

[rui314]

What is the "label value of the landing pad?

[rui314]

How should we find a function symbol for a given R_RISCV_LPAD relocation? Should we just look for a function symbol at the same location as the relocation refers to?

[kito-cheng]

Right now, this design requires scanning through the symbol table and relocation table once to figure out which symbols have landing pads. I had previously thought about creating a new section to handle this, but I realized that when dealing with linker relaxation, the best way is still to use relocations to mark them. This approach also avoids introducing a new section format.

[mylai-mtk]

Right now, this design requires scanning through the symbol table and relocation table once to figure out which symbols have landing pads.

Does this mean that O(R + F) time complexity and O(R) space complexity, where R is number of LPAD relocations and F is number of global function symbols in the same section, to build a map from function symbols to lpad labels is expected? This map is needed to synthesize PLT.

Note: The complexities come from the following algorithm:

HashMap<Address, LabelLabel> LpadRelocMap;
HashMap<FunctionSymbol, LpadLabel> LpadLabelMap;
for (auto R: LpadRelocations)
  LpadRelocMap.insert(R.Address, R.addend);

for (auto S: FunctionSymbols)
  if (S.Address in LpadRelocMap) lpadLabelMap.insert(S, LpadRelocMap[S.Address]);

[ved-rivos]

what would be the cases where a label may be computed incorrectly.

[kito-cheng]

Some legacy programs don’t properly declare function prototypes before calling them. In these cases, the compiler will infer a function prototype based on the language standards, but it often ends up being incorrect. One common example is dhrystone[1]. In most versions you find online, Func_2 isn’t declared before it’s called, so the compiler will assume the prototype is int Func_2(char*, char*), but the correct prototype is actually void Func_2(char[31], char[31]).

[1] https://github.com/sifive/benchmark-dhrystone/blob/master/dhry_1.c#L164

Another common potential issue in C is with qsort. Function pointers can be compatible but not perfectly match the expected type. For example, here’s how qsort is declared:

void qsort(void* ptr, size_t count, size_t size, int (*comp)(const void*, const void*));

But in practice, you can pass in a compatible, but not exactly matching, type for the comparison function, and it works in most cases:

#include <stdlib.h>

int compare(int *a, int *b)  // The signature isn’t int (*)(const void*, const void*)
{
    return *(int *)a - *(int *)b;
}

void foo(int *x, size_t count, size_t size)
{
    qsort(x, count, size, compare);  // But in practice, this works fine
}

[mylai-mtk]

But how is the linker expected to know the incorrectness so it can perform this relaxation?

The Zicfilp mechanism is employed when issuing an indirect call through function pointers, and when calling functions through PLT:

In the first case (indirect calls through pointers), to know that an lpad insn needs to be relaxed to lpad 0 due to the original label being incorrect, the linker would need to know where the pointer points to, so the caller's label (the "correct" one) can be checked against the callee's label (the lpad insn). I'm not sure if this is the scenario you're targeting, but if it is, I think this (knowing where the pointer points to, or knowing where the call would come from) is an expectation too high for linkers. Besides, in this scenario, I would also wonder how linkers are expected to retrieve the caller's label (the "correct" one) so it can make the comparison?

In the second case (calls through PLT), the indirect call happens in the PLT, which is generated by linkers. The label which linkers use to generate PLT would come from the addend of the LPAD relocation, which should contain the same label as the one in its referenced lpad insn, so there would be no chance of mismatch and thus incorrectness identifiable by linkers.

The above is my guess and understanding of the intended usage of this relaxation. If we're not on the same page, please do let me know.

[kito-cheng]

Linker never know (or not always know), and also that's not the right layer to analysis (or guess:P ), so I expect that relaxation should only enabled when user pass something like -z force-simple-landing-pad-scheme to linker.

[mylai-mtk]

I don't quite get the purpose of this mapping symbol: It looks like the only reference to these symbols come from the LPAD relocation, but for what? The LPAD relocation already have the 20-bit label stored in its addend, and its link to this %s mapping symbol provides only the additional information of the signature string, which is not needed (as of now) to link the object files.

[kito-cheng]

riscv/riscv-cfi#151 (comment)

It's kinda debugging propose only, so it safe to strip like all other mapping symbols

[mylai-mtk]

If the purpose is to display function signatures when disassembling, this mechanism seems a bit incomplete (?) I suppose since the relocation is a static one, it would not stay in the binary after static linking, thus if a user disassembles a linked ELF, it's still the label numbers instead of signatures that get displayed?

Update: Assuming it's relying on the mapping symbol having the same address as the lpad insn to associate an lpad insn to a function signature (so that the signature can be displayed when disassembling a linked binary), why do relocations refer to these symbols?

[mylai-mtk]

If the purpose of the mapping symbol is to provide debug/disassembling info, I think after the introduction of the .riscv.lpadinfo section, this purpose can be better served by the new section, since it contains the same information, and has a more size-compact format than the symbol table.

[mylai-mtk]

Q: Does the LPAD relocation and/or the $s<func-sig> mapping symbol serve the purpose of providing labels when generating PLT? If so, how is it expected to extract the label for a given function symbol from these new entities?

[rui314]

I don't think I fully understand this proposal, but from the linker's perspective, I believe we just need the following:

• A R_RISCV_LPAD relocation, whose r_offset is at each removable lpad instruction location, and r_sym refers to a function symbol
• If the function's address was not taken (i.e. the function symbol was not referenced by anything but R_RISCV_CALL or R_RISCV_CALL_PLT) and the function symbol is not in the dynamic symbol, the linker removes the lpad instruction.

I don't think we need a mapping symbol for the linker to remove a lpad instruction.

[kito-cheng]

@rui314

I don't think I fully understand this proposal, but from the linker's perspective, I believe we just need the following:

A R_RISCV_LPAD relocation, whose r_offset is at each removable lpad instruction location, and r_sym refers to a function symbol
If the function's address was not taken (i.e. the function symbol was not referenced by anything but R_RISCV_CALL or R_RISCV_CALL_PLT) and the function symbol is not in the dynamic symbol, the linker removes the lpad instruction.
I don't think we need a mapping symbol for the linker to remove a lpad instruction.

The design of the mapping symbol is meant to make debugging easier for users, so it’s actually optional. This was discussed in the CFI spec issue [1]. If we set aside that purpose, I also think it’s a better approach to have r_sym point to the function symbol.

As for using the addend to record the landing pad value, it helps with PLT generation. The linker can get the value by scanning the instruction at that address, but encoding it directly in the addend avoids needing to read the instruction during relocation.

[1] riscv/riscv-cfi#151 (comment)

[mylai-mtk]

Is this relocation only for the func-sig scheme? Based on its description, it looks like so, but the following LPAD relaxation that removes the lpad insn seems also applicable to the unlabeled scheme.

[kito-cheng]

That should also work for unlabeled scheme as well, let me think how to make it clearly.

[mylai-mtk]

@kito-cheng

As for using the addend to record the landing pad value, it helps with PLT generation. The linker can get the value by scanning the instruction at that address, but encoding it directly in the addend avoids needing to read the instruction during relocation.

I think this doesn't provide the lpad label we need for PLT generation? For a call target to be generated in the PLT, the target should reside in a shared library or somewhere that the linker cannot determine during static-time linking. When the target resides in a shared library, it does not have the LPAD relocation associated with it, since the relocation is a static relocation and should be consumed by the static linker that created the shared library?

[kito-cheng]

@mylai-mtk

I think this doesn't provide the lpad label we need for PLT generation? For a call target to be generated in the PLT, the target should reside in a shared library or somewhere that the linker cannot determine during static-time linking. When the target resides in a shared library, it does not have the LPAD relocation associated with it, since the relocation is a static relocation and should be consumed by the static linker that created the shared library?

Hmmmmmmmmmmmmmm, yeah, do you have any better idea that creating a new section to recording the label value for those undefined symbols?

[mylai-mtk]

@kito-cheng

Hmmmmmmmmmmmmmm, yeah, do you have any better idea that creating a new section to recording the label value for those undefined symbols?

Nope. After skimming through existing sections, I don't have ideas better than creating a new section.

My previous prototype of generating extra symbols for every function may work, but there's too many drawbacks with this approach, including bloated symbol table size and slowed down symbol resolution performance, so I would not recommend this approach seriously.

[kito-cheng]

@mylai-mtk added new section. and also new asm directive riscv-non-isa/riscv-asm-manual#113

[mylai-mtk]

Lpad labels are only 20-bit wide, so we can use uint32_t. Same for the 32-bit version above.

[mylai-mtk]

I think you mean "Every function symbol with global or weak"?

[kito-cheng]

Oh yeah, should just restrict to symbol with function type

[mylai-mtk]

Also, we should restrict this to global or pointer-taken symbols, i.e. function symbols with an lpad instruction after relaxation

Update: No need to have the above requirement, since trimming down the size of the .riscv.lpadinfo section is just nice-to-have but not must-have. Having unneeded entries is not ideal, but it would not affect the correctness, so it's not necessary to mandate this in a specification.

[mylai-mtk]

Since you mentioned that "Every symbol with global or weak bind must has a corresponding entry", I think it implies that the lpad labels are provided by the object file that defines the function, right? If this is the case, we can't discard this section after static linking when creating a shared library, since library users would expect to find lpad labels later when linking against this share library.

[kito-cheng]

We can still know the signature/landing pad label value when we reference to a symbol which is undefined yet, because we always need declare the prototype in the source code.

"Every symbol with global or weak bind must has a corresponding entry" -> we didn't exclude the undefined symbol, so we can link to the shared library without that info

[mylai-mtk]

If labels can be provided by the object that uses but doesn't define the function, why require labels to be there in the defining object ("Every symbol with global or weak bind must has a corresponding entry")? For the sake of checking if the use-site and define-site agree on the same lpad label?

[mylai-mtk]

I think it's still better to put the .riscv.lpadinfo section in shared libraries. I can come up with two possible cases in which the C language prototype is hard to obtain:

Case 1: When writing assembly by hand, the pseudo call or jump instruction is used. These instructions would cause the CALL_PLT relocation to be generated in the resulting relocatable, which would lead to PLT entries being generated by the static linker.

Case 2: Compiler inserts calls to builtins or instrumentations. These extra function calls are often not inserted at the C source level, but at the compiler IR level. This makes knowing the C prototype of the called target hard as there would not be a C language data structure to represent the called target in the AST.

In these cases, it's not easy to obtain the C language prototype, as the only assumption in these cases is that there would be a defined symbol to which the linker can resolve the called target. Considering these cases, I would still prefer to have the shared libraries provide the lpad labels for functions defined by them.

[kito-cheng]

I agree with you, so...I just say can be not must be here, that make it strip-able, but it's also legal to leave here

[mylai-mtk]

Hmmm, if this is the intention, I would suggest you make it clear that it means the section is strip-able, and does not mean that the section can be discarded under all circumstances after static linkage. I believe if a user strips his binary, he should know the consequence, so marking a section that is only strip-able under certain situations strip-able is acceptable.

My concern is that if this sentence is considered by a linker implementation to be the spec of linker behavior that uniformly allows the section to be dropped after static linkage, the spec is flawed, since when producing shared libraries, the section cannot be dropped or otherwise the library could risk linkage failure to relocatables.

[mylai-mtk]

Is it beneficial that we also implement this check in the dynamic linker for dynamic symbols? E.g. Abort the program if label mismatches are found at program startup or dlopen()

[mylai-mtk]

This means the signatures stored start with $ and is in the format of $x<function-signature-string>? (Note the additional x there)

If the goal is to save bytes in the string table, I think we can always use Symbol($x<function-signature-string>).st_name + 2 to specify the signature string to keep the referred signature precise without the $x prefix?

[kito-cheng]

Drop mapping symbol so this paragraph will be removed

[mylai-mtk]

If the purpose of the mapping symbol is to provide debug/disassembling info, I think after the introduction of the .riscv.lpadinfo section, this purpose can be better served by the new section, since it contains the same information, and has a more size-compact format than the symbol table.

[mylai-mtk]

Does this LPAD relocation intend to serve the "real" purpose of a relocation? That is, ask linkers to fill-in some value (in this case, the label of the lpad instructions) to some offset at link time.

When prototyping this relocation in LLVM, it appears to me that the LLVM backend assumes places to be relocated have a placeholder value 0 encoded, so to emit the LPAD relocations, 0s would have to be encoded at the label locations in relocatable files. This can of course be changed to encode the correct label along with relocation emitted, but I want to ask if this change is needed, or I can rely on linkers to fill-in the correct labels when relocating at static time.

[mylai-mtk]

We need to specify section header info, including the sh_type, sh_flags field.

[mylai-mtk]

Change to: The landing pad information section is a section that contains the information to generate PLT with landing pads. The static linker is required to use the landing pad values provided by this section when generating lpad instructions for functions listed in this section, unless otherwise specified by the user (e.g. through command line options) or the unlabeled CFI scheme is selected.

This change allows us to:

• Avoid limiting the use of this section to the function-signature scheme
• Be clear that when lpad labels can be inferred from other information sources contained in the source object, the riscv.lpadinfo section would take precedence, unless the linker user commands otherwise clearly (e.g. through the -z force-zicfilp=unlabeled command line option) or the unlabeled CFI scheme is selected (the unlabeled scheme implies an optimized PLT entry sequence, thus 0 must be used for lpad labels.)

Update: Exclude the unlabeled CFI scheme from adopting .riscv.lpadinfo section contents, since it implies an optimized PLT entry sequence, thus 0 must be used for lpad labels.

[mylai-mtk]

Just curious: Why isn't this the index of symbol in the symbol table?

The symbol table index provides a more definite reference to the symbol, considering the case of repeated symbol names (though this may not really happen with function symbols as its an error to globally define a function more than once.)

[mylai-mtk]

I've come up with a case that would cause the current format to fail:

// public_foo.c
void foo() {}

// private_foo_1.c
static int foo(int i) { return i; }
void *get_foo_1() { return foo; }

// private_foo_2.c
static char foo(char c) { return c; }
void *get_foo_2() { return foo; }

Compile them to a shared library, and you get 1 GLOBAL FUNC and 2 LOCAL FUNC foo symbols, and the signature of these 3 foos are different. With the current format, the lpad info entries of these 3 symbols cannot be distinguished, and this would cause problem for the linker.

To support showing lpi_info contents when disassembling, none of the 3 lpad info entries can be omitted, so to distinguish them, symbol table indices should be used in place of string table indices to refer to symbols, and as a consequence, lpi_name should be renamed to lpi_sym or lpi_func (the field does not represent "name" anymore).

[kito-cheng]

TL;DR: I will update to the symbol index in next update.

And let me tell the story why I use the symbol name before: In the earlier version, we have mapping symbol at the beginning of the function, so local function could use that only, but we gonna to remove the mapping symbol, so that means local symbol may also rely this mechanism to record the signature.

Recording the symbol name is fine before since global symbol has unique symbol name (in theory), and local symbol may have duplicated symbol name, but it's not a problem before.

The concern for using the symbol index is: .symtab will gone after strip, so the index is kinda become meaningless, and we have only index for .dynsym, but .dynsym only contain exported global symbol only, however this issue can be resolve if we say the .riscv.lpadinfo should be stripped if .symtab got stripped.

So the conclusion is: I think we should use symbol index here, rather than symbol name as your suggestion.

[mylai-mtk]

Thank you for your explanation, and sorry for ruining your original design about mapping symbol 😥

The main reason I was against the mapping symbol approach was that it's intended usage was obscure. It only had a reference from the LPAD relocation that is intended to provide the disassembling string for the lpad label, but considering that LPAD relocations would be gone after static linking, this solution looks incomplete (it only provides the disassembling string for relocatables, but not for executables and shared libraries), so it didn't convince me that we should adopt the mapping symbol approach.

But if how the mapping symbols were associated to the lpad insns is clarified and unified by dropping the reference in the LPAD relocation, and solely relies on the mapping symbols having the st_value that points to the lpad insns, I think using mapping symbols to provide the disassembling strings actually has its advantages (and is better than using .riscv.lpadinfo):

• It allows decorating lpad insns not only at the beginning of function symbols, but also in the middle of function bodies
• As a kind of debug info, it's automatically stripped away when the the binary is stripped
• Symbol handling in compiler backends and linkers has extensive infrastructure support. Not much work would need to be carried out to have the mapping symbols properly emitted
• If the approach is adopted, we can remove the lpi_info field from the .riscv.lpadinfo section, making the section simpler and probably more compact and concise than the current format
  • Also, separating the optional debug info from the necessary .riscv.lpadinfo simplifies the treatment of the .riscv.lpadinfo section in toolchain (it would be easier to determine if the section needs to kept or can be discarded, and relevant data structures would be easier to maintain)

The only disadvantage (I can come up with) of the updated mapping symbol approach would be a bloated symbol table, but symbol tables are strippable in the final executables/shared libraries, so it does not matter much in production after all.

Saying so much, would you mind to clarify how the mapping symbol approach was supposed to work, and let's reconsider it (and the format .riscv.lpadinfo)?

Thank you 🙇‍♂️

[mylai-mtk]

Is it possible that we design this spec in terms of just labeled and unlabeled schemes, instead of the more constraining func-sig and unlabeled schemes? (Same goes for the PLT draft spec)

The motivation is to increase the versatility of these specs. This idea is based on the observation (and "feeling" during my prototyping) that nearly all components of these specs for the func-sig scheme (including the PLT form, relocations, relaxations, .riscv.lpadinfo section content, mapping symbol, and the %lpad_label("func-sig") assembly operand modifier) are designed with just the assumption that the lpad values can be non-zero, and they come from strings. This implies that the specification of "func-sig" can actually be removed to make these specs more versatile to support all schemes that generates lpad values from strings, which is flexible enough to encode almost all data formats and therefore, most of future possible schemes.