nightly-mac bot is regularly segfaulting #16489

alexcrichton · 2014-08-14T04:33:42Z

logs: http://buildbot.rust-lang.org/builders/nightly-mac

relevant bits:

rustc: x86_64-apple-darwin/stage1/lib/rustlib/i686-apple-darwin/lib/libcore
rustc: x86_64-apple-darwin/stage1/lib/rustlib/i686-apple-darwin/lib/liblibc
rustc: x86_64-apple-darwin/stage1/lib/rustlib/x86_64-apple-darwin/lib/libcore
rustc: x86_64-apple-darwin/stage1/lib/rustlib/x86_64-apple-darwin/lib/liblibc
rustc: x86_64-apple-darwin/stage1/lib/rustlib/i686-apple-darwin/lib/librand
rustc: x86_64-apple-darwin/stage1/lib/rustlib/i686-apple-darwin/lib/liballoc
rustc: x86_64-apple-darwin/stage1/lib/rustlib/i686-apple-darwin/lib/libunicode
rustc: x86_64-apple-darwin/stage1/lib/rustlib/x86_64-apple-darwin/lib/librand
rustc: x86_64-apple-darwin/stage1/lib/rustlib/x86_64-apple-darwin/lib/liballoc
rustc: x86_64-apple-darwin/stage1/lib/rustlib/x86_64-apple-darwin/lib/libunicode
rustc: x86_64-apple-darwin/stage1/lib/rustlib/i686-apple-darwin/lib/libcollections
make: *** [x86_64-apple-darwin/stage1/lib/rustlib/i686-apple-darwin/lib/stamp.rand] Segmentation fault: 11 (core dumped)
make: *** Waiting for unfinished jobs....

I wish I knew more sadly :(

The text was updated successfully, but these errors were encountered:

alexcrichton · 2014-08-14T04:39:38Z

This might be the backtrace (not sure if it's the right core dump)

  thread #2: tid = 0x0001, 0x0000000110bf8450 librbml-4e7c5e5c.dylib`reader::maybe_get_doc::h49f0a6e62ed2aa2eMMa + 192, stop reason = signal SIGSTOP
    frame #0: 0x0000000110bf8450 librbml-4e7c5e5c.dylib`reader::maybe_get_doc::h49f0a6e62ed2aa2eMMa + 192
    frame #1: 0x00000000017fb0d4
    frame #2: 0x000000010ea5a4ae librustc-4e7c5e5c.dylib`metadata::creader::PluginMetadataReader$LT$$x27a$GT$::read_plugin_metadata::h8a67b9643df81f1enCt + 2174
    frame #3: 0x000000010eb152d3 librustc-4e7c5e5c.dylib`plugin::load::PluginLoader$LT$$x27a$GT$.Visitor$LT$$LP$$RP$$GT$::visit_view_item::h63e14bd2cb2a16fdF4A + 611
    frame #4: 0x000000010eaab91f librustc-4e7c5e5c.dylib`plugin::load::load_plugins::hc01fc2451d60e1b0r3A + 319
    frame #5: 0x000000010e8f5b7d librustc-4e7c5e5c.dylib`driver::driver::phase_2_configure_and_expand::hc69692654be8f218WBw + 1981
    frame #6: 0x000000010ea6d9ff librustc-4e7c5e5c.dylib`driver::driver::compile_input::h558ac2baf30310beuvw + 943
    frame #7: 0x000000010eb01d62 librustc-4e7c5e5c.dylib`driver::main_args::closure.136228 + 6082
    frame #8: 0x000000010eb0e9cb librustc-4e7c5e5c.dylib`task::TaskBuilder$LT$S$GT$::try_future::closure.137382 + 75
    frame #9: 0x000000010eb0e8f7 librustc-4e7c5e5c.dylib`task::TaskBuilder$LT$S$GT$::spawn_internal::closure.137346 + 215
    frame #10: 0x000000010e2ec53a libnative-4e7c5e5c.dylib`task::spawn_opts::closure.8378 + 74
    frame #11: 0x00000001117c2a7c librustrt-4e7c5e5c.dylib`rust_try_inner + 12
    frame #12: 0x00007f818a500000
    frame #13: 0x000000010e2ec3d0 libnative-4e7c5e5c.dylib`task::spawn_opts::closure.8324 + 240
    frame #14: 0x000000011175e490 librustrt-4e7c5e5c.dylib`thread::thread_start::hf75b60e3600e3439Fqd + 32
    frame #15: 0x00007fff9234d8bf libsystem_c.dylib`_pthread_start + 335
    frame #16: 0x00007fff92350b75 libsystem_c.dylib`thread_start + 13

alexcrichton · 2014-08-14T04:59:32Z

Yes, that is the backtrace. I'm going to create a new snapshot and pray that one of @luqmana's recent fixes for codegen fixed it!

Hopefully this will fix rust-lang#16489!

Hopefully this will fix #16489!

alexcrichton · 2014-08-17T05:44:50Z

Nope, today's nightly segfaulted: http://buildbot.rust-lang.org/builders/nightly-mac/builds/77

rustc: x86_64-apple-darwin/stage1/lib/rustlib/i686-apple-darwin/lib/libcore
rustc: x86_64-apple-darwin/stage1/lib/rustlib/i686-apple-darwin/lib/liblibc
rustc: x86_64-apple-darwin/stage1/lib/rustlib/x86_64-apple-darwin/lib/libcore
rustc: x86_64-apple-darwin/stage1/lib/rustlib/x86_64-apple-darwin/lib/liblibc
rustc: x86_64-apple-darwin/stage1/lib/rustlib/i686-apple-darwin/lib/librand
rustc: x86_64-apple-darwin/stage1/lib/rustlib/i686-apple-darwin/lib/liballoc
rustc: x86_64-apple-darwin/stage1/lib/rustlib/i686-apple-darwin/lib/libunicode
rustc: x86_64-apple-darwin/stage1/lib/rustlib/x86_64-apple-darwin/lib/librand
rustc: x86_64-apple-darwin/stage1/lib/rustlib/x86_64-apple-darwin/lib/liballoc
rustc: x86_64-apple-darwin/stage1/lib/rustlib/x86_64-apple-darwin/lib/libunicode
make: *** [x86_64-apple-darwin/stage1/lib/rustlib/i686-apple-darwin/lib/stamp.rand] Segmentation fault: 11 (core dumped)
make: *** Waiting for unfinished jobs....
rustc: x86_64-apple-darwin/stage1/lib/rustlib/i686-apple-darwin/lib/libcollections
program finished with exit code 2
elapsedTime=539.708832

brson · 2014-08-27T00:21:30Z

Nominating. I would not be comfortable releasing 1.0 with this mystery.

pnkfelix · 2014-08-28T21:43:39Z

We need to investigate further to be sure that this is a bug in rust (and not, e.g., a problem in our internal machines/infrastructure).

Classifying as P-high, 1.0, on the basis of internal pressure to determine whether this is indeed a rust bug.

brson · 2014-08-28T21:44:00Z

Snaps are run on the 'mac3' builder only, which has a failing hard drive. I will try to remove that machine from the rotation to hopefully fix the problem.

alexcrichton · 2014-08-28T21:50:15Z

@brson, you mentioned this may be disks going bad, and one (maybe plausible) explanation is that we're mmaping the dynamic library to read the metadata and then the mmap is going wrong after the intial success, but that's just a wild guess assuming that bad disks are the actual underlying cause.

brson · 2014-09-02T18:59:59Z

I've switched the snapshots to mac4 and the problem persists. Seems to be real.

alexcrichton · 2014-09-17T03:21:28Z

New theory:

64-bit compiler builds 32-bit libcore
64-bit compiler builds 64-bit libcore
64-bit compiler builds 32-bit librand
1. 64-bit compiler looks for #[phase(plugin)] libcore for 32-bit librand
2. 64-bit compiler first looks for 64-bit libcore (that's the host arch)
3. 64-bit compiler mmaps a partially written 64-bit libcore (step 2 is still in flight)
4. 64-bit compiler reads corrupt metadata, causing unsafe code code to segfault

I think what's happening here is that the compiler reading a half-written library somehow lets it pick up corrupt state and causes something else to go awry down the line. A possible to attempt to explain the symptoms:

This only happens when compiling librand on a cross-build for OSX. The only way the compiler will read a half-written library is if it crosses architectures (otherwise the deps are all in order). Also librand is the first library to be built after libcore, and the first library to use #[phase(plugin)].
This happens fairly regularly. It looks like the scheduling of librand/libcore and host/target always shows up the same on OSX, causing a "repeatable" error.
Builds sporadically succeed. This is a timing-sensitive explanation, allowing races to go either way.

The symptom I can't explain is that this only happens on OSX. Not sure what's going on there. I'm going to disable -j2 for the nightly builders for now and see if this improves. If so, I'm going to close this in the future.

alexcrichton · 2014-09-17T03:27:09Z

Also that specific unsafe code is probably not at fault. I would be more suspicious of this transmute.

shadowmint · 2014-09-17T05:56:59Z

fwiw although the nightly-mac builder has succeeded:

Sep 17 02:47        success     #124    Build successful
http://buildbot.rust-lang.org/builders/nightly-mac/builds/124

Its now 3 hours later, and the nightly snapshot hasn't been updated:

doug-2:~ doug$ curl https://static.rust-lang.org/rustup.sh | sudo bash
...
install: verifying installed binaries are executable

Rust is ready to roll.

install: verifying installed binaries are executable

Cargo is ready to roll.

doug-2:~ doug$ rustc --version
rustc 0.12.0-pre-nightly (21d1f4d7c 2014-09-14 10:36:08 +0000)
doug-2:~ doug$ cargo --version
cargo 0.0.1-pre-nightly (31c4baf 2014-09-15 19:13:42 +0000)
doug-2:~ doug$ date
Wed 17 Sep 2014 13:54:05 WST

So there may be an issue with this build even when it succeed on the build. I'm not sure if you want to track that separately or as part of this ticket; if it's simply an artifact of the intermittently failing builds.

alexcrichton · 2014-09-17T13:34:57Z

That issue (syncing with cloudfare) is tracked in #16649.

alexcrichton · 2014-09-19T18:00:35Z

Last few nights haven't segfaulted, closing this and will reopen if necessary.

…as, r=Veykril feat!: create alias when renaming an import. ![gif](https://github.com/rust-lang/rust-analyzer/assets/57047985/c593d9a8-b8a0-4e13-9e50-a69c7d0d8749) Closes rust-lang#15858 Implemented: - [x] - Prevent using `reserved` keywords (e.g self) and `_`. - [x] - Rename other modules that might be referencing the import. - [x] - Fix "broken" tests. - [ ] - Rename **only** "direct" references. - [ ] - Test more cases. Future possibilities: 1. Also support `extern crate <name>` syntax. 2. Allow aliasing `self` when it is inside an `UseTreeList`. ~3. If import path already has an alias, "rename" the alias.~ ~[4. Create alias even if path is not the last path segment.](rust-lang/rust-analyzer#16489 (comment)

alexcrichton added A-mac-osx labels Aug 14, 2014

alexcrichton added a commit to alexcrichton/rust that referenced this issue Aug 16, 2014

Register new snapshots

8f16aa7

Hopefully this will fix rust-lang#16489!

alexcrichton mentioned this issue Aug 16, 2014

Register new snapshots #16531

Merged

bors added a commit that referenced this issue Aug 16, 2014

auto merge of #16531 : alexcrichton/rust/snapshots, r=luqmana

85fd37f

Hopefully this will fix #16489!

bors closed this as completed in #16531 Aug 16, 2014

alexcrichton reopened this Aug 17, 2014

huonw mentioned this issue Aug 17, 2014

static.rust-lang.org/rustup.sh may be stale for Mac rust-lang/cargo#392

Closed

brson added the I-nominated label Aug 27, 2014

brson assigned thestinger and brson and unassigned thestinger Aug 28, 2014

pnkfelix added P-high and removed I-nominated labels Aug 28, 2014

pnkfelix added this to the 1.0 milestone Aug 28, 2014

alexcrichton mentioned this issue Sep 17, 2014

Rust nightly build for OSX is failing #17328

Closed

brson mentioned this issue Sep 17, 2014

Automation metabug #17356

Closed

65 tasks

alexcrichton closed this as completed Sep 19, 2014

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nightly-mac bot is regularly segfaulting #16489

nightly-mac bot is regularly segfaulting #16489

alexcrichton commented Aug 14, 2014

alexcrichton commented Aug 14, 2014

alexcrichton commented Aug 14, 2014

alexcrichton commented Aug 17, 2014

brson commented Aug 27, 2014

pnkfelix commented Aug 28, 2014

brson commented Aug 28, 2014

alexcrichton commented Aug 28, 2014

brson commented Sep 2, 2014

alexcrichton commented Sep 17, 2014

alexcrichton commented Sep 17, 2014

shadowmint commented Sep 17, 2014

alexcrichton commented Sep 17, 2014

alexcrichton commented Sep 19, 2014

nightly-mac bot is regularly segfaulting #16489

nightly-mac bot is regularly segfaulting #16489

Comments

alexcrichton commented Aug 14, 2014

alexcrichton commented Aug 14, 2014

alexcrichton commented Aug 14, 2014

alexcrichton commented Aug 17, 2014

brson commented Aug 27, 2014

pnkfelix commented Aug 28, 2014

brson commented Aug 28, 2014

alexcrichton commented Aug 28, 2014

brson commented Sep 2, 2014

alexcrichton commented Sep 17, 2014

alexcrichton commented Sep 17, 2014

shadowmint commented Sep 17, 2014

alexcrichton commented Sep 17, 2014

alexcrichton commented Sep 19, 2014