Skip to content

Build fails with --disable-jemalloc #21526

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
AndrewScull opened this issue Jan 22, 2015 · 9 comments
Closed

Build fails with --disable-jemalloc #21526

AndrewScull opened this issue Jan 22, 2015 · 9 comments
Labels
I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics.

Comments

@AndrewScull
Copy link
Contributor

Configuring with ../configure --disable-docs --enable-ccache --disable-jemalloc causes the build to fail with:

x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore
Illegal instruction (core dumped)

Usually it is Illegal instruction however sometimes a bad or double free is reported.

Building with jemalloc enabled (../configure --disable-docs --enable-ccache) has no problem.
@huonw huonw added the I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics. label Jan 22, 2015
@huonw
Copy link
Member

huonw commented Jan 22, 2015

I can reproduce. Running the aborting command under valgrind gives hundreds of errors like:

==26307== Invalid read of size 8
==26307==    at 0x4E4B4D5: collections..vec..Vec$LT$syntax..ptr..P$LT$syntax..codemap..Spanned$LT$syntax..ast..MetaItem_$GT$$GT$$GT$::glue_drop.12051::h50b8b9a8c4be2bb3 (in /home/huon/rust3/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-4e7c5e5c.so)
==26307==    by 0xF90490F: ???
==26307==    by 0x2F: ???
==26307==    by 0xD461B9F: ???
==26307==    by 0xD52354F: ???
==26307==    by 0xF5F963F: ???
==26307==    by 0x4E4E8FF: syntax..codemap..Spanned$LT$syntax..ast..Attribute_$GT$::glue_drop.12756::hb78b33cc126cc208 (in /home/huon/rust3/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-4e7c5e5c.so)
==26307==    by 0x8097101EF: ???
==26307==    by 0xC9B2F8F: ???
==26307==    by 0xD461B6F: ???
==26307==    by 0x4EA9E2E: iter::FlatMap$LT$A$C$$u{20}B$C$$u{20}I$C$$u{20}U$C$$u{20}F$GT$.Iterator::next::h11721808144370039366 (in /home/huon/rust3/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-4e7c5e5c.so)
==26307==    by 0x4E9746C: fold::noop_fold_item::h5513282176331594077 (in /home/huon/rust3/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-4e7c5e5c.so)
==26307==  Address 0xe5f3240 is 0 bytes inside a block of size 80 free'd
==26307==    at 0x4C29E90: free (vg_replace_malloc.c:473)
==26307==    by 0x4E4B597: collections..vec..Vec$LT$syntax..ptr..P$LT$syntax..codemap..Spanned$LT$syntax..ast..MetaItem_$GT$$GT$$GT$::glue_drop.12051::h50b8b9a8c4be2bb3 (in /home/huon/rust3/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-4e7c5e5c.so)
==26307==    by 0xF90490F: ???
==26307==    by 0xD46183F: ???
==26307==    by 0xF5F963F: ???
==26307==    by 0xD52354F: ???
==26307==    by 0xC9B2EFF: ???
==26307==    by 0x4E4E8FF: syntax..codemap..Spanned$LT$syntax..ast..Attribute_$GT$::glue_drop.12756::hb78b33cc126cc208 (in /home/huon/rust3/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-4e7c5e5c.so)
==26307==    by 0x4F: ???
==26307==    by 0xC9B2F8F: ???
==26307==    by 0xD4618AF: ???
==26307==    by 0x4EAADA8: fold::noop_fold_impl_item::h7121156065768389895 (in /home/huon/rust3/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-4e7c5e5c.so)
==26307== 
==26307== Invalid read of size 8
==26307==    at 0x4E4B55F: collections..vec..Vec$LT$syntax..ptr..P$LT$syntax..codemap..Spanned$LT$syntax..ast..MetaItem_$GT$$GT$$GT$::glue_drop.12051::h50b8b9a8c4be2bb3 (in /home/huon/rust3/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-4e7c5e5c.so)
==26307==    by 0xF90490F: ???
==26307==    by 0x2F: ???
==26307==    by 0xD461B9F: ???
==26307==    by 0xD52354F: ???
==26307==    by 0xF5F963F: ???
==26307==    by 0x4E4E8FF: syntax..codemap..Spanned$LT$syntax..ast..Attribute_$GT$::glue_drop.12756::hb78b33cc126cc208 (in /home/huon/rust3/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-4e7c5e5c.so)
==26307==    by 0x8097101EF: ???
==26307==    by 0xC9B2F8F: ???
==26307==    by 0xD461B6F: ???
==26307==    by 0x4EA9E2E: iter::FlatMap$LT$A$C$$u{20}B$C$$u{20}I$C$$u{20}U$C$$u{20}F$GT$.Iterator::next::h11721808144370039366 (in /home/huon/rust3/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-4e7c5e5c.so)
==26307==    by 0x4E9746C: fold::noop_fold_item::h5513282176331594077 (in /home/huon/rust3/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-4e7c5e5c.so)
==26307==  Address 0xe5f3248 is 8 bytes inside a block of size 80 free'd
==26307==    at 0x4C29E90: free (vg_replace_malloc.c:473)
==26307==    by 0x4E4B597: collections..vec..Vec$LT$syntax..ptr..P$LT$syntax..codemap..Spanned$LT$syntax..ast..MetaItem_$GT$$GT$$GT$::glue_drop.12051::h50b8b9a8c4be2bb3 (in /home/huon/rust3/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-4e7c5e5c.so)
==26307==    by 0xF90490F: ???
==26307==    by 0xD46183F: ???
==26307==    by 0xF5F963F: ???
==26307==    by 0xD52354F: ???
==26307==    by 0xC9B2EFF: ???
==26307==    by 0x4E4E8FF: syntax..codemap..Spanned$LT$syntax..ast..Attribute_$GT$::glue_drop.12756::hb78b33cc126cc208 (in /home/huon/rust3/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-4e7c5e5c.so)
==26307==    by 0x4F: ???
==26307==    by 0xC9B2F8F: ???
==26307==    by 0xD4618AF: ???
==26307==    by 0x4EAADA8: fold::noop_fold_impl_item::h7121156065768389895 (in /home/huon/rust3/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-4e7c5e5c.so)

@AndrewScull
Copy link
Contributor Author

I have tracked down the introduction of this crash to the commit: 98d4d49

@huonw
Copy link
Member

huonw commented Jan 24, 2015

Wow, thanks for tracking it down, although it's surprising to me, since that commit seems fairly inoccuous. I wonder if one of the other commits in the rollup #21213 may be playing a part too (since those sort of PRs aren't guaranteed to have every commit compiling).

semarie added a commit to jasperla/openbsd-wip that referenced this issue Jan 25, 2015
@semarie
lang/rustc: upgrade to snapshot 2015-01-20 9006c3c
cd5aaf4 
- adapt openbsd patches for 2015-01-20 9006c3c
- switch rustc build to use jemalloc (and patch jemalloc) due to bug in
  rustc without it (see rust-lang/rust#21526)
- regen patches & distinfo (new bootstrap)
@edwardw edwardw mentioned this issue Jan 27, 2015
Closed
@aidancully
Copy link
Contributor

For what it's worth, I believe the issue is in the stage0 snapshot generating code that can have use-after-free issues. If it's a use-after-free issue (which the valgrind output seems to indicate), the problem's manifestation will be unrelated to the actual patch that generated the panic. While trying to track this down locally, this patch made the problem disappear so that the build ran to completion:

diff --git a/src/libsyntax/fold.rs b/src/libsyntax/fold.rs
index 2a70434..f138ddf 100644
--- a/src/libsyntax/fold.rs
+++ b/src/libsyntax/fold.rs
@@ -1159,6 +1159,7 @@ pub fn noop_fold_item<T: Folder>(i: P<Item>, folder: &mut T) -> SmallVector<P<It
 // fold one item into exactly one item
 pub fn noop_fold_item_simple<T: Folder>(Item {id, ident, attrs, node, vis, span}: Item,
                                         folder: &mut T) -> Item {
+    debug!("noop_fold_item_simple");
     let id = folder.new_id(id);
     let node = folder.fold_item_underscore(node);
     let ident = match node {

@semarie semarie mentioned this issue Feb 14, 2015
Closed
@semarie
Copy link
Contributor

semarie commented Feb 14, 2015

@aidancully what is your last known to work build with this workaround ?

@semarie
Copy link
Contributor

semarie commented Feb 14, 2015

Starting from 985fc7d (PR #22200) , the workaround seems to no work anymore for me.

@aidancully
Copy link
Contributor

@semarie: I don't know, I needed --disable-jemalloc for fakeCargo, but I haven't been using fakeCargo since cargo seems to have stopped changing as frequently. So I've been using jemalloc -- only so many hours in the day. That "workaround" was never intended to solve the issue, but rather just to show that the issue seems to be in internal Rust memory management. There is no way that that patch should in any way address the actual issue. (Incidentally, I think this is a serious issue, and suspect that we're just lucky that jemalloc works differently internally than system-malloc does.)

@pnkfelix
Copy link
Member

cc me

@alexcrichton
Copy link
Member

Seeing how this hasn't cropped up in awhile, I'm gonna close as "presumably fixed" at some point

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@alexcrichton @pnkfelix @aidancully @huonw @AndrewScull @semarie