Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ld64.lld: error: relocation BRANCH26 is out of range: -165194492 is not in [-134217728, 134217727]; references _calloc #52767

Closed
glandium opened this issue Dec 17, 2021 · 13 comments · Fixed by #99052
Closed

ld64.lld: error: relocation BRANCH26 is out of range: -165194492 is not in [-134217728, 134217727]; references _calloc #52767

glandium opened this issue Dec 17, 2021 · 13 comments · Fixed by #99052
Labels
lld:MachO

Comments

@glandium
Copy link
Contributor

glandium commented Dec 17, 2021 •
edited
Loading

Building Firefox for arm64 macos with disabled optimizations and with lld fails with:

ld64.lld: error: relocation BRANCH26 is out of range: -165194492 is not in [-134217728, 134217727]; references _calloc
ld64.lld: error: relocation BRANCH26 is out of range: -165196704 is not in [-134217728, 134217727]; references _calloc
ld64.lld: error: relocation BRANCH26 is out of range: -165196676 is not in [-134217728, 134217727]; references _calloc
ld64.lld: error: relocation BRANCH26 is out of range: -165196144 is not in [-134217728, 134217727]; references _calloc
ld64.lld: error: relocation BRANCH26 is out of range: -165196116 is not in [-134217728, 134217727]; references _calloc
ld64.lld: error: relocation BRANCH26 is out of range: -165196088 is not in [-134217728, 134217727]; references _calloc
ld64.lld: error: relocation BRANCH26 is out of range: -165195984 is not in [-134217728, 134217727]; references _calloc
ld64.lld: error: relocation BRANCH26 is out of range: -165299108 is not in [-134217728, 134217727]; references _malloc
ld64.lld: error: relocation BRANCH26 is out of range: -165308144 is not in [-134217728, 134217727]; references _malloc
ld64.lld: error: relocation BRANCH26 is out of range: -165208212 is not in [-134217728, 134217727]; references _calloc
ld64.lld: error: relocation BRANCH26 is out of range: -165311620 is not in [-134217728, 134217727]; references _malloc

STR:

  • download https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Fald_IyiRO-ljidFJwvZig/runs/0/artifacts/public/build/testcase.tar.zstd
  • tar --zstd -xf testcase.tar.zstd
  • cd testcase/toolkit/library/build
  • clang++ --sysroot /path/to/MacOSX11.0.sdk -F/path/to/MacOSX11.0.sdk/System/Library/PrivateFrameworks -mmacosx-version-min=11.0 -stdlib=libc++ --target=aarch64-apple-darwin -o XUL -Wl,-filelist,XUL.list -fuse-ld=lld -framework Cocoa -lobjc -framework AudioToolbox -lresolv ../../../security/nss/lib/crmf/crmf_crmf/libcrmf.a ../../../js/src/build/libjs_static.a ../../../aarch64-apple-darwin/debug/libgkrust.a ../../../security/libnss3.dylib ../../../config/external/lgpllibs/liblgpllibs.dylib ../../../mozglue/build/libmozglue.dylib -dynamiclib -lbsm -framework IOSurface -framework IOKit -framework AudioToolbox -framework CoreMedia -framework VideoToolbox -framework LocalAuthentication -framework Security -lm -framework OpenGL -framework SystemConfiguration -framework AVFoundation -framework CoreUI -framework CoreSymbolication -lcups -framework Foundation -framework CoreFoundation -framework CoreLocation -framework QuartzCore -framework Carbon -framework CoreAudio -framework CoreVideo -framework AudioToolbox -framework AudioUnit -framework AddressBook -framework OpenGL -framework Security -framework ServiceManagement -framework CoreServices -framework ApplicationServices -framework AppKit -weak_framework Metal -weak_framework MediaPlayer

Cc: @nico @int3 @keith
@keith
Copy link
Member

keith commented Dec 17, 2021

I am able to repro, note I had to add -F/Applications/Xcode-13.2.0-RC1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/PrivateFrameworks to the args to find CoreUI

@keith
Copy link
Member

keith commented Jan 5, 2022

@thevinster is this the same as what https://reviews.llvm.org/D116705 should fix?

@smeenai
Copy link
Collaborator

smeenai commented Jan 6, 2022

I patched that locally and it didn't fix this issue :( The error sounds like a thunk isn't getting created at all, which is strange. I imagine _calloc is just a reference to the C standard library function, in which case this would be a thunk pointing to a stub; perhaps that's a case we hadn't run into before?

@glandium
Copy link
Contributor Author

glandium commented Jan 6, 2022

BTW, the testcase tarball expires next week, should I upload it somewhere else?

@smeenai
Copy link
Collaborator

smeenai commented Jan 7, 2022

BTW, the testcase tarball expires next week, should I upload it somewhere else?

Yup, I think that'd be helpful.

CC @nico, who I think looked at the range extension thunk code recently.

@glandium
Copy link
Contributor Author

Here's the same testcase, which shouldn't expire until next year: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Fald_IyiRO-ljidFJwvZig/runs/0/artifacts/public/build/testcase.tar.zstd (I'll update the original comment with this url too)

@keith
Copy link
Member

keith commented Jan 29, 2022

I noticed this also doesn't link with ld64:

ld: b(l) ARM64 branch out of range (195733676 max is +/-128MB): from _mdb_env_stat.island (0x07C1413C) to _mdb_env_stat (0x136BE9E8) in '_mdb_env_stat.island' for architecture arm64

@int3
Copy link
Contributor

int3 commented Feb 3, 2022

This issue inspired me to finally implement better diagnostic info for LLD :)

This is what I've got:

ld64.lld: error: ../../../aarch64-apple-darwin/debug/libgkrust.a(mdb.o):(symbol _mdb_env_create+0x20): relocation BRANCH26 is out of range: -165194492 is not in [-134217728, 134217727]; references _calloc
ld64.lld: error: ../../../aarch64-apple-darwin/debug/libgkrust.a(mdb.o):(symbol _mdb_env_open+0x3d4): relocation BRANCH26 is out of range: -165196704 is not in [-134217728, 134217727]; references _calloc
ld64.lld: error: ../../../aarch64-apple-darwin/debug/libgkrust.a(mdb.o):(symbol _mdb_env_open+0x3b8): relocation BRANCH26 is out of range: -165196676 is not in [-134217728, 134217727]; references _calloc
ld64.lld: error: ../../../aarch64-apple-darwin/debug/libgkrust.a(mdb.o):(symbol _mdb_env_open+0x1a4): relocation BRANCH26 is out of range: -165196144 is not in [-134217728, 134217727]; references _calloc
ld64.lld: error: ../../../aarch64-apple-darwin/debug/libgkrust.a(mdb.o):(symbol _mdb_env_open+0x188): relocation BRANCH26 is out of range: -165196116 is not in [-134217728, 134217727]; references _calloc
ld64.lld: error: ../../../aarch64-apple-darwin/debug/libgkrust.a(mdb.o):(symbol _mdb_env_open+0x16c): relocation BRANCH26 is out of range: -165196088 is not in [-134217728, 134217727]; references _calloc
ld64.lld: error: ../../../aarch64-apple-darwin/debug/libgkrust.a(mdb.o):(symbol _mdb_env_open+0x104): relocation BRANCH26 is out of range: -165195984 is not in [-134217728, 134217727]; references _calloc
ld64.lld: error: ../../../aarch64-apple-darwin/debug/libgkrust.a(mdb.o):(symbol _mdb_fname_init+0x90): relocation BRANCH26 is out of range: -165299108 is not in [-134217728, 134217727]; references _malloc
ld64.lld: error: ../../../aarch64-apple-darwin/debug/libgkrust.a(mdb.o):(symbol _mdb_reader_check0+0xa0): relocation BRANCH26 is out of range: -165308144 is not in [-134217728, 134217727]; references _malloc
ld64.lld: error: ../../../aarch64-apple-darwin/debug/libgkrust.a(mdb.o):(symbol _mdb_env_init_meta+0x54): relocation BRANCH26 is out of range: -165208212 is not in [-134217728, 134217727]; references _calloc
ld64.lld: error: ../../../aarch64-apple-darwin/debug/libgkrust.a(mdb.o):(symbol _mdb_env_cwalk+0x120): relocation BRANCH26 is out of range: -165311620 is not in [-134217728, 134217727]; references _malloc

I noticed that all the problematic symbols were in a section called text_env. This section gets placed late in __TEXT, after the __stubs section. I believe our branch island code assumes that __stubs always occurs after the sections with branches, but in fact this is only true of the __text section.

I wouldn't be surprised if ld64's implementation has a similar (wrong) assumption...

int3 added a commit that referenced this issue Mar 1, 2022
@int3
[lld-macho] Have relocation address included in range-check error mes…
a552fb2 
…sage

This makes it easier to debug those errors. See e.g. #52767 (comment)

We take the approach of 'reverse-engineering' the InputSection from the
output buffer offset. This provides for a cleaner Target API, and is
similar to LLD-ELF's implementation of getErrorPlace().

Reviewed By: #lld-macho, Roger

Differential Revision: https://reviews.llvm.org/D118903
@stheophil
Copy link

We hit the same issue when we compile a large C++ project with Xcode, enable Asan/UBSan, and place some code in dedicated sections.

@int3
Copy link
Contributor

int3 commented Jun 8, 2022

@stheophil does ld64 link the same project successfully?

@stheophil
Copy link

No, ld64 is the Xcode linker right? So we use ld64 and it fails to link.

mem-frob pushed a commit to draperlaboratory/hope-llvm-project that referenced this issue Oct 7, 2022
@int3
[lld-macho] Have relocation address included in range-check error mes…
08b66e2 
…sage

This makes it easier to debug those errors. See e.g. llvm/llvm-project#52767 (comment)

We take the approach of 'reverse-engineering' the InputSection from the
output buffer offset. This provides for a cleaner Target API, and is
similar to LLD-ELF's implementation of getErrorPlace().

Reviewed By: #lld-macho, Roger

Differential Revision: https://reviews.llvm.org/D118903
gbaraldi pushed a commit to JuliaLang/llvm-project that referenced this issue Jan 6, 2023
@int3 @gbaraldi
[lld-macho] Have relocation address included in range-check error mes…
f32c8a0 
…sage

This makes it easier to debug those errors. See e.g. llvm#52767 (comment)

We take the approach of 'reverse-engineering' the InputSection from the
output buffer offset. This provides for a cleaner Target API, and is
similar to LLD-ELF's implementation of getErrorPlace().

Reviewed By: #lld-macho, Roger

Differential Revision: https://reviews.llvm.org/D118903
@calvin2021y
Copy link

Try 15.0.7 still has this kind problem.  any way to work around ?

@nullhook
Copy link

i just came across this issue. has anyone found a workaround yet?

@speednoisemovement speednoisemovement mentioned this issue Apr 5, 2024
Closed
@speednoisemovement speednoisemovement mentioned this issue Jul 16, 2024
Merged
yuxuanchen1997 pushed a commit that referenced this issue Jul 25, 2024
@speednoisemovement @yuxuanchen1997
[lld-macho] Fix thunks for non-__text TEXT sections (#99052)
052ffa3 
Summary:
This supersedes #87818 and
fixes #52767

When calculating arm64 thunks, we make a few assumptions that may not
hold when considering code sections outside of `__text`:

1. That a section needs thunks only if its size is larger than the
branch range.
2. That any calls into `__stubs` are necessarily forward jumps (that is,
the section with the jump is ordered before `__stubs`)

Sections like this exist in the wild, most prominently the
`__lcxx_overrides` section introduced in
#69498

This change:
- Ensures that if one section in `__TEXT` gets thunks, all of them do.
- Makes all code sections in `__TEXT` contiguous (and guaranteed to be
placed before `__stubs`)

Test Plan: 

Reviewers: 

Subscribers: 

Tasks: 

Tags: 


Differential Revision: https://phabricator.intern.facebook.com/D60251191
@RyanGlScott RyanGlScott mentioned this issue Aug 20, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
lld:MachO
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[lld-macho] Fix thunks for non-__text TEXT sections speednoisemovement/llvm-project
9 participants
@int3 @keith @sylvestre @glandium @smeenai @stheophil @nullhook @EugeneZelenko @calvin2021y