Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make a disable-jemalloc build work #43589

Merged
merged 2 commits into from
Aug 11, 2017

Conversation

aidanhs
Copy link
Member

@aidanhs aidanhs commented Aug 1, 2017

Fixes #43510. I've tested this up to building a stage1 compiler.

r? @alexcrichton

cc @cuviper @vorner

@cuviper your fix was almost correct, you just had a stray ! in there which caused the second error you saw.

@alexcrichton
Copy link
Member

@bors: r+

@bors
Copy link
Contributor

bors commented Aug 1, 2017

📌 Commit 947cfb3 has been approved by alexcrichton

@cuviper
Copy link
Member

cuviper commented Aug 1, 2017

Oh, I see, I copied the ! from the feature line. Thanks!

@bors
Copy link
Contributor

bors commented Aug 2, 2017

⌛ Testing commit 947cfb3d003b41a64c5ee754dbf518a53d54d4b4 with merge 899c27a18bcf2e6044af87d64afa5865dada3d17...

@bors
Copy link
Contributor

bors commented Aug 2, 2017

💔 Test failed - status-appveyor

@kennytm
Copy link
Member

kennytm commented Aug 2, 2017

x86_64-pc-windows-msvc failed to link term in stage0-libtest. Given the topic involved in this PR: Legit.

   Compiling term v0.0.0 (file:///C:/projects/rust/src/libterm)
error: linking with `link.exe` failed: exit code: 1120
  |
  = note: "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\bin\\amd64\\link.exe" "/NOLOGO" "/NXCOMPAT" "/LIBPATH:C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage0-sysroot\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage0-test\\x86_64-pc-windows-msvc\\release\\deps\\term-c6d433459a54f839.0.o" "/OUT:C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage0-test\\x86_64-pc-windows-msvc\\release\\deps\\term-c6d433459a54f839.dll" "/DEF:C:\\Users\\appveyor\\AppData\\Local\\Temp\\1\\rustc.eYw8sZPlb2Lc\\lib.def" "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage0-test\\x86_64-pc-windows-msvc\\release\\deps\\term-c6d433459a54f839.crate.metadata.o" "C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage0-test\\x86_64-pc-windows-msvc\\release\\deps\\term-c6d433459a54f839.crate.allocator.o" "/OPT:REF,ICF" "/DEBUG" "/LIBPATH:C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage0-test\\x86_64-pc-windows-msvc\\release\\deps" "/LIBPATH:C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage0-test\\release\\deps" "/LIBPATH:C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage0-sysroot\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "kernel32.lib" "/LIBPATH:C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage0-sysroot\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "std-6894941a948c9efd.dll.lib" "C:\\Users\\appveyor\\AppData\\Local\\Temp\\1\\rustc.eYw8sZPlb2Lc\\libcompiler_builtins-dd1b2e32329767b3.rlib" "advapi32.lib" "ws2_32.lib" "userenv.lib" "shell32.lib" "libcmt.lib" "/DLL" "/IMPLIB:C:\\projects\\rust\\build\\x86_64-pc-windows-msvc\\stage0-test\\x86_64-pc-windows-msvc\\release\\deps\\term-c6d433459a54f839.dll.lib"
  = note:    Creating library C:\projects\rust\build\x86_64-pc-windows-msvc\stage0-test\x86_64-pc-windows-msvc\release\deps\term-c6d433459a54f839.dll.lib and object C:\projects\rust\build\x86_64-pc-windows-msvc\stage0-test\x86_64-pc-windows-msvc\release\deps\term-c6d433459a54f839.dll.exp
          term-c6d433459a54f839.crate.allocator.o : error LNK2019: unresolved external symbol __rg_alloc referenced in function __rust_alloc
          term-c6d433459a54f839.crate.allocator.o : error LNK2019: unresolved external symbol __rg_oom referenced in function __rust_oom
          term-c6d433459a54f839.crate.allocator.o : error LNK2019: unresolved external symbol __rg_dealloc referenced in function __rust_dealloc
          term-c6d433459a54f839.crate.allocator.o : error LNK2019: unresolved external symbol __rg_usable_size referenced in function __rust_usable_size
          term-c6d433459a54f839.crate.allocator.o : error LNK2019: unresolved external symbol __rg_realloc referenced in function __rust_realloc
          term-c6d433459a54f839.crate.allocator.o : error LNK2019: unresolved external symbol __rg_alloc_zeroed referenced in function __rust_alloc_zeroed
          term-c6d433459a54f839.crate.allocator.o : error LNK2019: unresolved external symbol __rg_alloc_excess referenced in function __rust_alloc_excess
          term-c6d433459a54f839.crate.allocator.o : error LNK2019: unresolved external symbol __rg_realloc_excess referenced in function __rust_realloc_excess
          term-c6d433459a54f839.crate.allocator.o : error LNK2019: unresolved external symbol __rg_grow_in_place referenced in function __rust_grow_in_place
          term-c6d433459a54f839.crate.allocator.o : error LNK2019: unresolved external symbol __rg_shrink_in_place referenced in function __rust_shrink_in_place
          C:\projects\rust\build\x86_64-pc-windows-msvc\stage0-test\x86_64-pc-windows-msvc\release\deps\term-c6d433459a54f839.dll : fatal error LNK1120: 10 unresolved externals
          
error: aborting due to previous error
error: Could not compile `term`.

@aidanhs
Copy link
Member Author

aidanhs commented Aug 2, 2017

Hum, I'll have to have a think about this, looks like it worked fine on all the osx and linux variants, but not on windows and I'm not 100% sure why - in theory global_allocator works the same way (right?) and the allocator tests work on windows so I don't have any immediate ideas.

Thoughts from onlookers appreciated.

@cuviper
Copy link
Member

cuviper commented Aug 2, 2017

I'm no Windows expert, but I think it's more particular about resolving symbols only from directly-specified libraries. I'm not sure what that implies in this case.

It looks like the msvc builds always disable jemalloc anyway, so maybe this workaround can just be skipped there. Appveyor didn't go far enough to see if windows-gnu has a similar link error.

@cuviper
Copy link
Member

cuviper commented Aug 2, 2017

AFAICS none of the Windows targets change exe_allocation_crate from the default None, so maybe it just defaults to alloc_system even when windows-gnu does ship alloc_jemalloc?

@kennytm
Copy link
Member

kennytm commented Aug 3, 2017

@bors retry (Queue is empty, and let's see if windows-gnu have any messages.)

@bors
Copy link
Contributor

bors commented Aug 3, 2017

⌛ Testing commit 947cfb3d003b41a64c5ee754dbf518a53d54d4b4 with merge ed77d9fbc01ca9ded17300f6256558c0ea891d88...

@bors
Copy link
Contributor

bors commented Aug 3, 2017

💔 Test failed - status-appveyor

@aidanhs aidanhs added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Aug 3, 2017
@alexcrichton
Copy link
Member

Errors like this are typically indicative of a missing "dllexport" attribute of some shape/form when the object is compiled. We have to apply dllexport manually and IIRC we don't do this correctly for allocator attributes.

I think, though, the fix may not be too too hard if you'd like to pioneer it here. The boolean on whether to do this is here, [here] is where allocator modules are trans'd, and I think here is roughly the right location for where to say "this symbol is a C symbol," e.g. needs a dllexport.

@aidanhs
Copy link
Member Author

aidanhs commented Aug 5, 2017

Ok, I've had a look at this and it's a bit humbling to realise how little I know about linking on Windows! Fortunately I have access to Windows so I'll play around this weekend and I think I have a route forward - looking at the export details, I think I might be able to just add extern to the allocator symbols during trans, rather than special casing them? We shall see anyway.

@alexcrichton
Copy link
Member

@aidanhs ah yeah that sounds like a fine strategy to me as well!

@aidanhs aidanhs force-pushed the aphs-fix-system-malloc branch 5 times, most recently from a2b1347 to 62231b1 Compare August 10, 2017 14:55
@aidanhs
Copy link
Member Author

aidanhs commented Aug 10, 2017

This is the best I've come up with after wrestling with it for a bit. Note that because the stage0 snapshot has the broken behavior on msvc, I've had to cfg the global allocator stuff until the next snapshot update (which also means a temporary tidy check change).

In terms of approach:

  • I can't use the 'iterate through exports of crates' because the global allocator code isn't attached to a crate (I think?), it's just inserted directly into the LLVM module.
  • I think a Rust level export is fine because it's only referenced by rust code, i.e. when something depends on an rlib or dylib (rather than cdylib).

This gets past building stage0 term locally on Windows (obviously, because the new code is cfged out) but I've been unable to test further than that because building LLVM on Windows is being problematic. In particular, I'm not sure I've got the correct number of underscores.

@alexcrichton
Copy link
Member

@bors: r+

That all sounds good to me! I think you're right about the "Rust level" as well instead of "C"

@bors
Copy link
Contributor

bors commented Aug 10, 2017

📌 Commit 56a0753 has been approved by alexcrichton

@bors
Copy link
Contributor

bors commented Aug 10, 2017

⌛ Testing commit 56a0753 with merge 6c5212f...

bors added a commit that referenced this pull request Aug 10, 2017
Make a disable-jemalloc build work

Fixes #43510. I've tested this up to building a stage1 compiler.

r? @alexcrichton

cc @cuviper @vorner

@cuviper your fix was almost correct, you just had a stray `!` in there which caused the second error you saw.
@bors
Copy link
Contributor

bors commented Aug 11, 2017

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing 6c5212f to master...

@bors bors merged commit 56a0753 into rust-lang:master Aug 11, 2017
@aidanhs aidanhs deleted the aphs-fix-system-malloc branch August 11, 2017 11:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

stage0 libstd fails to build when jemalloc is disabled
5 participants