Unable to use LTO when using mixed arm and thumb mode

[DoDoENT]

Description

When using LTO and some source files are compiled in arm mode, while other are compiled in thumb mode, I get following error:

 warning: LLVM gold plugin: Linking two modules of different target triples: CMakeFiles/native-lib.dir/src/main/cpp/native-lib.cpp.o' is 'thumbv7-none-linux-android' whereas 'ld-temp.o' is 'armv7-none-linux-android'
ld.gold: error: treating warnings as errors

Is there a way to workaround the issue?

Here is the minimum project that reproduces the issue

Environment Details

• NDK Version: r14
• Build sytem: cmake
• Host OS: Mac
• Compiler: Clang
• ABI: armeabi-v7a
• STL: c++_static
• NDK API level:9
• Device API level:10

[DanAlbert]

@stephenhines, is this something you could get ARM to look at?

[pirama-arumuga-nainar]

This warning is benign and LLVM will continue to merge the two modules. I think this is the case of a benign warning interacting badly with, I presume, -Wl,--fatal-warnings.

[DanAlbert]

How receptive do you think upstream would be to removing the warning? I don't really see what this one adds (why should the user ever care?), and since the linkers don't offer the same amount of control over warnings that compilers do (can't disable this warning afaik, and can't downgrade it to a warning if you want errors for other warnings), any warning the linker emits needs to be thought of as essentially an error.

[pirama-arumuga-nainar]

Actually, upon further thought, I am not so sure that is a benign warning. LLVM's LTO works by generating LLVM IR in place of the .o files and at the linking step, merging the IR (modules) together, doing LTO and finally code generation. I know that Clang embeds target CPU and features into each function, but I am not sure if it embeds the triple as well. If it doesn't, the triple used while linking would be used to generate code for all the functions in the linked module. This would be bad.

I'll further investigate this issue later this week.

[pirama-arumuga-nainar]

Looked at this further and I don't think ARM and Thumb modes can be mixed during LTO. Like I mentioned earlier, the IR does not bake the selected triple into each function. Even if this is done, I don't think the back end is capable of handling more than one triple. The way LTO is designed, codegen happens only based on the triple and features passed to the linker.

In that sense, I think the warning is good because it catches the mismatch rather than silently doing the wrong thing. The wrong thing being, ignoring the triples in the .o files and generating code for the target/features requested at runtime.

[DoDoENT]

AFAIK, GCC 6 can do that. So, what about adding support for using latest GCC (this brings us back to #26) or fixing the LLVM linker? Because not being able to combine LTO, ARM and Thumb modes in single binary is not acceptable.

[echristo]

It's possible to get LLVM to do it (some other ports do), but the ARM port in particular is problematic here. It needs significant restructuring to consider ARM and Thumb modes to be subtarget features rather than whole module. You should be able to LTO on ARM or Thumb separately in the meantime, you just won't get the full benefit of internalize.

[eugenis]

This is also a problem for ControlFlowIntegrity, which uses LTO to collect info about complete class hierarchies. The "LTO on ARM or Thumb separately" approach does not work for us, we really need everything linked at once.

[echristo]

You'll probably want to look into rewriting the TargetMachine and TargetSubtargetInfo interfaces for the ARM port then. :(

[DoDoENT]

Any progress on this?

[stephenhines]

No, we don't have any progress to share yet on this topic. We can talk to our partners to see if they are interested in working on this in the short term.

[stephenhines]

https://reviews.llvm.org/D33287 is the start of fixing this. I will keep this bug posted as more progress is made.

[DanAlbert]

This is fixed in r17. Sorry for the delays.