Linker error with GCC 7

[shenki]

When building hostboot with GCC 7 we see these errors at link time:

exception caught: Expected 63308 found 3749708 at 3754200
exception caught: Expected 63308 found 3749708 at 3754200
exception caught: Expected 208305 found 1073966513 at 240240
exception caught: Expected 208305 found 1073966513 at 240240

These messages come from here: https://github.com/open-power/hostboot/blob/master/src/build/linker/linker.C#L834

The build command is this:

../src/build/linker/linker ../img/hbirt_test.bin ../img/hbirt_test.elf         ../img/libtrace_rt.so ../img/liberrl_rt.so ../img/libtargeting_rt.so ../img/libutil_rt.so ../img/libdevicefw_rt.so ../img/libxscom_rt.so ../img/libibscom_rt.so ../img/libscom_rt.so ../img/libvpd_rt.so ../img/libi2c_rt.so ../img/libhtmgt_rt.so ../img/libattn_rt.so ../img/libprdf_rt.so ../img/libipmi_rt.so ../img/libpm_rt.so ../img/libpnor_rt.so ../img/libfapi2_rt.so ../img/libsecureboot_rt.so ../img/libp9_cpuWkup.so ../img/libfsi_rt.so ../img/libsbeio_rt.so ../img/libtod_rt.so ../img/libnestmemutils.so ../img/libimageprocs_rt.so ../img/libcxxtest_rt.so ../img/libtestsyslib_rt.so ../img/libtestxscom_rt.so ../img/libtesterrl_rt.so ../img/libtestdevicefw_rt.so ../img/libtestscom_rt.so ../img/libtestutil_rt.so ../img/libtestvpd_rt.so ../img/libtestprdf_rt.so ../img/libtestattn_rt.so ../img/libtestipmi_rt.so ../img/libtestpnor_rt.so ../img/libtestfapi2_rt.so ../img/libtestsecureboot_rt.so ../img/libtesttargeting_rt.so ../img/libtestsbeio_rt.so ../img/libtestpm_rt.so ../img/p9_nimbus.prf ../img/p9_cumulus.prf ../img/p9_eq.prf ../img/p9_ex.prf ../img/p9_ec.prf ../img/p9_capp.prf ../img/p9_pec.prf ../img/p9_phb.prf ../img/p9_obus.prf ../img/p9_xbus.prf ../img/p9_mcbist.prf ../img/p9_mcs.prf ../img/p9_mca.prf ../img/p9_mc.prf ../img/p9_mi.prf ../img/p9_dmi.prf ../img/cen_centaur.prf ../img/cen_mba.prf

It is very verbose. Just before the failure, it's doing this:

File ../img/libp9_cpuWkup.so

        Relocated 10f38 at 10070 to 394f38
        Symbol: _ZZNK5fapi26TargetILNS_10TargetTypeE1024EPN9TARGETING6TargetEE9getParentILS1_2097152EEENS0_IXT_ES4_EEvE20__traceData_codeInfo
        Relocated 11490 at 10080 to 395490
        Symbol: _ZZNK5fapi26TargetILNS_10TargetTypeE1024EPN9TARGETING6TargetEE9getParentILS1_2097152EEENS0_IXT_ES4_EEvE20__traceData_codeInfo_0
        Relocated 11484 at 10098 to 395484
        Symbol: _ZZNK5fapi26TargetILNS_10TargetTypeE1024EPN9TARGETING6TargetEE9getParentILS1_2097152EEENS0_IXT_ES4_EEvE20__traceData_codeInfo_1
        Relocated 11478 at 100a0 to 395478
        Symbol: _ZZNK5fapi26TargetILNS_10TargetTypeE1024EPN9TARGETING6TargetEE9getParentILS1_2097152EEENS0_IXT_ES4_EEvE20__traceData_codeInfo_2
        Relocated 1146c at 100a8 to 39546c
        Symbol: *ABS*
        Relocated 4c0 at 100c0 to 3844c0
        Symbol: *ABS*
        Relocated 17600 at 100c8 to 39b600
        Symbol: *ABS*
        Relocated 510 at 100d8 to 384510
        Symbol: *ABS*
        Relocated 17600 at 100e0 to 39b600
        Symbol: *ABS*
        Relocated 730 at 100f0 to 384730
        Symbol: *ABS*
        Relocated 17600 at 100f8 to 39b600

Then we see this:

        Symbol: *ABS*
        Relocated f74c at 108c0 to 39374c
        Symbol: .rodata
        Relocated f74c at 108cc to 39374c
        Symbol: *ABS*
        Relocated f74c at 108d8 to 39374c
        Symbol: *ABS*
Expected 63308 found 3749708 at 3754200
exception caught: Expected 63308 found 3749708 at 3754200

The "working" print is showing:

        cout << "\tRelocated " << i->addend << " at " << i->address << " to "
            << relocation << endl;

The error case looks like this:

            oss << "Expected " << i->addend << " found " << address
                << " at " << (offset + i->address);

Back to the bad entry, hex(63308) = f74c

powerpc64-linux-gnu-objdump -R img/libp9_cpuWkup.so  | grep -i f74c
0000000000010860 R_PPC64_RELATIVE  *ABS*+0x000000000000f74c
0000000000010878 R_PPC64_RELATIVE  *ABS*+0x000000000000f74c
00000000000108c0 R_PPC64_RELATIVE  *ABS*+0x000000000000f74c
00000000000108d8 R_PPC64_RELATIVE  *ABS*+0x000000000000f74c
00000000000108d8 R_PPC64_RELATIVE  *ABS*+0x000000000000f74c
0000000000010908 R_PPC64_RELATIVE  *ABS*+0x000000000000f74c
0000000000010938 R_PPC64_RELATIVE  *ABS*+0x000000000000f74c
0000000000010950 R_PPC64_RELATIVE  *ABS*+0x000000000000f74c
(continues)

[dcrowell77]

Do you happen to know anything knew about GCC7 that would lead to new symbol names?

How much tweaking is required to compile with GCC7 locally?

[grooverdan]

For ubuntu, https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test there is gcc-7/gcc-8 (and other) packages that are quite stable. There's also docker images for gcc.

[ghost]

Daniel Black <notifications@github.com> writes:

For ubuntu, https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test there is gcc-7/gcc-8 (and other) packages that are quite stable. There's also docker images for gcc.

For hostboot you need the custom compiled linker though (to then build the custom linker), so I think the best route is probably tweaking the config in an op-build to get you a more recent gcc. (it'd be good if we could not require a custom built linker at some point though).

…

-- Stewart Smith OPAL Architect, IBM.

[shenki]

Thanks @grooverdan and @stewart-ibm.

@dcrowell77 To do a GCC 7 build, make this change to your defconfig (eg openpower/configs/romulus_defconfig):

-BR2_GCC_VERSION_6_X=y
+BR2_GCC_VERSION_7_X=y

Otherwise, you can grab my gcc-7 branch from here which has that change already applied:

https://github.com/shenki/op-build/tree/gcc-7

git clone --recursive -b gcc-7 https://github.com/shenki/op-build/
cd op-build
. op-build-env
op-build romulus_defconfig
op-build

[shenki]

I spoke to @segher, one of our toolchain people, and they said it was luck that previous versions of the compiler did not emit the R_PPC64_RELATIVE entries. They said there was nothing specifically that changed between GCC 6 and GCC 7 that would have caused it to start doing that.

[dcrowell77]

(it'd be good if we could not require a custom built linker at some point though).

Noted... but there are things that it does that the regular linker doesn't do since we have a static memory space with no dynamic linking.

For planning purposes, how important is moving to GCC7?

[segher]

So what are the GCC 6 and GCC 7 compiler versions? gcc -v

And, what does gcc -Q --help=common | grep -- -fpi say for both?

(My suspicion is your GCC 7 enables -fpie by default -- terrible plan, etc., but some distros do that).

[shenki]

We use buildroot to generate the compiler. Neither appear to have -fpie by default:

powerpc64le-buildroot-linux-gnu-gcc-7.3.0 -Q --help=common | grep -- -fpi
  -fpic                       		[disabled]
  -fpie                       		[disabled]

  -fpic                       		[disabled]
  -fpie                       		[disabled]

[shenki]

@dcrowell77 can you explain why "static memory space with no dynamic linking" stops hostboot from being able to use the binutils linker?

Being stuck on GCC 6 means the rest of the firmware doesn't get to benefit from the past couple of years of compiler development. This isn't ideal, but there are no known compiler issues affecting the firmware at this stage.

We need a solution for before buildroot removes support for GCC 6. I don't think they plan to do this with their next quarterly release (September), but it would be on the cards.

[dcrowell77]

can you explain why...

Sadly no... This was all built by someone (Patrick) who isn't here anymore and I don't have the background without digging into things for awhile. He is a smart guy so I assume there was a good reason to be non-standard.

A couple things I do see:

• we generate some extra data that is consumed by our VFS code since we are running a custom module loader
• there is a weak symbol checker
• we have 2 images (base and extended) with some different relocation settings (I think)
• we have some non ELF images that are included as virtual 'files' inside our image
• there is some checking done to ensure that a given symbol is available to the module that is consuming it

I have no familiarity with binutils either so I can't give a quick diff between them.