internal compiler error: 'index out of bounds'

[oblitum]

Building rust-lang/cargo@31c4baf on 63eaba2 but got this:

Could not compile `libgit2`.

--- stdout
task 'rustc' failed at 'index out of bounds: the len is 56 but the index is 406', /private/tmp/rust-2WTYvO/src/libsyntax/lib.rs:1

stack backtrace:
   1:        0x112809249 - rt::backtrace::imp::write::h1903589be5ac481cdPq
   2:        0x11280c5a1 - failure::on_fail::h77d0ffb2719b85cdR5q
   3:        0x112aab955 - unwind::begin_unwind_inner::hea9898af96e0a19fPQd
   4:        0x112aab608 - unwind::begin_unwind_fmt::h5d7843f827787588hOd
   5:        0x112aab2f2 - rust_begin_unwind
   6:        0x112afdf8c - failure::begin_unwind::heb0c8adb522059ea1mk
   7:        0x112afde5e - failure::fail_bounds_check::hba050a8e3d6df883skk
   8:        0x11a078684 - util::interner::StrInterner::get::h99d136e0eccf641c0pa
   9:        0x11a08f968 - ast::Name::as_str::h58412323700bd16aTMb
  10:        0x11a139681 - parse::parser::Parser<'a>::lit_from_token::h599eab467f4278f5ChK
  11:        0x11a13dc2c - parse::parser::Parser<'a>::parse_lit::hd28284011a2b4b93GjK
  12:        0x11a142603 - parse::parser::Parser<'a>::parse_bottom_expr::h2ee1d373c8341700lBK
  13:        0x11a14a146 - parse::parser::Parser<'a>::parse_prefix_expr::h6dc3a17b37d46ec4wfL
  14:        0x11a14a7fd - parse::parser::Parser<'a>::parse_assign_expr::h953c9b592ca189495nL
  15:        0x11a07e572 - parse::parser::Parser<'a>::parse_expr::h7d6bce249db03716syL
  16:        0x11793238e - parse_string::h676aa2a77ed1f92a7ua
  17:        0x11792f6d4 - expand_link_config::hb827bede6339e9c4Cba
  18:        0x110bc915a - ext::base::MacroExpanderFn.TTMacroExpander::expand::h83ec5aed6913e433m80
  19:        0x110c3eef6 - ext::expand::expand_item_mac::h3d741d0c6f4a7ab6kX7
  20:        0x110c3c07e - ext::expand::expand_item::hc11b8daef767cb165O7
  21:        0x110c48b87 - fold::noop_fold_mod::closure.52184
  22:        0x110c48980 - iter::Iterator::collect::h14144417837773372411
  23:        0x110c46c0f - fold::Folder::fold_mod::h9466688161727339124
  24:        0x110ca07bd - ext::expand::expand_crate::h8b0c4df0d18f5931vC8
  25:        0x11003132b - driver::driver::phase_2_configure_and_expand::closure.143781
  26:        0x10fe35475 - driver::driver::phase_2_configure_and_expand::h1a0cbe5b588f3d2eoou
  27:        0x10fff5262 - driver::driver::compile_input::h1ee1e1d504016d0evhu
  28:        0x11009f0b4 - driver::run_compiler::h4e5f868faf7a819d17x
  29:        0x11009d2e6 - driver::main_args::closure.147625
  30:        0x10f7af13b - task::TaskBuilder<S>::try_future::closure.100304
  31:        0x10f7af033 - task::TaskBuilder<S>::spawn_internal::closure.100275
  32:        0x10f72359d - task::spawn_opts::closure.8507
  33:        0x112b0dfcc - rust_try_inner
  34:        0x112b0dfb6 - rust_try
  35:        0x112aa8cd7 - unwind::try::hc47eaaee76e18794xFd
  36:        0x112aa8b4c - task::Task::run::h4aaec2a6b65d966fPVc
  37:        0x10f7233f2 - task::spawn_opts::closure.8446
  38:        0x112aaa80a - thread::thread_start::h8561d3c1c3e6984d1fd
  39:     0x7fff88377899 - _pthread_body
  40:     0x7fff8837772a - _pthread_struct_init


--- stderr
error: internal compiler error: unexpected failure
note: the compiler hit an unexpected failure path. this is a bug.
note: we would appreciate a bug report: http://doc.rust-lang.org/complement-bugreport.html
note: run with `RUST_BACKTRACE=1` for a backtrace


To learn more, run the command again with --verbose.
make: *** [cargo-x86_64-apple-darwin] Error 101

[bkoropoff]

This might be a problem in link-config.

[fyhuang]

I am also seeing this problem. rust-lang/cargo@5ff469f and "make" says it's using rustc 0.12.0-pre.

[nerdrew]

Are you both seeing this problem on macs or linux? I have the same crash when testing a small macro (copied from the rust repo).

https://gist.github.com/nerdrew/b460bd73f8b63d632d15

Edit: I'm on a mac

[oblitum]

@nerdrew I'm on mac.

[nerdrew]

The index out of bounds for me refers to interned strings I think. I printed them out; there were 56 of them. They look like language keywords only.

Here's the code to print them:

fn bam(cx: &mut ExtCtxt, _: Span, tts: &[TokenTree]) -> Box<MacResult+'static> {
    // Parse an expression and emit it unchanged.
    let mut parser = parse::new_parser_from_tts(cx.parse_sess(), cx.cfg(), Vec::from_slice(tts));

    let interner = get_ident_interner();
    print!("{}\n", interner.len());
    for i in range(0, 56) {
        let n = Name(i);
        print!("{}\n", interner.get(n));
    }

    let expr = parser.parse_expr();
    MacExpr::new(expr)
}

here's the output:

self
static
'static
tt
matchers
__rust_abi
<opaque>
<unnamed_field>
Self
prelude_import
as
break
crate
else
enum
extern
false
fn
for
if
impl
in
let
loop
match
mod
mut
once
pub
ref
return
struct
super
true
trait
type
unsafe
use
virtual
while
continue
proc
box
const
where
alignof
be
offsetof
priv
pure
sizeof
typeof
unsized
yield
do

[huonw]

(@nerdrew's example above doesn't fail for me on linux.)

This may be thread local storage getting junked up? (Or some other reason that causes the interner of the syntax extension to be disconnected from the interner of the main compiler, since 56 is exactly the number of default interned strings we have, and @nerdrew's list is exactly those strings.)

Maybe @alexcrichton could deduce more (you enjoy playing with TLS, right? 😝 ).

[fyhuang]

@nerdrew Yup, OSX.

[nerdrew]

Any movement on this? I can't compile the postgres library, which makes me sad :(

[oblitum]

still happening in 3be6a2f with rust-lang/cargo@ee02b9a.

[nerdrew]

Hmmmm. @alexcrichton any updates on this? I want to experiment with rust and the postgres library, but I can't build it :(

I'm willing to look at the code and see, but some guidance would help a lot.

[oblitum]

@nerdrew @fyhuang just a shot in the dark but recently another bug I've posted was uncovered and it was related to usage of Homebrew. As we're all having this issue on OS X, are you guys checking out Rust through brew --HEAD?

[nerdrew]

I've been installing the rust-nightly with my own homebrew tap: https://github.com/nerdrew/homebrew-tap/blob/master/rust-nightly.rb

I'll investigate

[fyhuang]

@oblitum Yes! I was indeed installing rust through brew --HEAD.

[nerdrew]

Bam! Setting DYLD_LIBRARY_PATH fixed it for me with the nightly, linked by homebrew using my custom homebrew tap.

export DYLD_LIBRARY_PATH=/usr/local/lib/rustlib/x86_64-apple-darwin/lib

Woooohoooo!

[oblitum]

not a shot in the dark at all it seems =D

[alexcrichton]

I think this is a dupe of #16029, but I'm not 100% sure. Note, however, @jgallagher wrote up an excellent explanation of that issue with some workarounds: #16029 (comment)

[nerdrew]

The workaround from #16029 (comment) fixed it for me. I can't speak to the cause, but the solution indicates it's a dupe.

[ashee]

The DYLD trick did not work for me - I am on OSX 10.9.5

$ export DYLD_LIBRARY_PATH=/usr/local/lib/rustlib/x86_64-apple-darwin/lib


touch target/snapshot/bin/cargo
"/usr/local/bin/rustc" -v
rustc 0.12.0-dev
target/snapshot/bin/cargo build --target x86_64-apple-darwin  
   Compiling toml v0.1.0 (https://github.com/alexcrichton/toml-rs#05c1eb42)
   Compiling semver v0.1.0 (https://github.com/rust-lang/semver#5dee9181)
   Compiling openssl-static-sys v0.0.1 (https://github.com/alexcrichton/openssl-static-sys#b0180139)
   Compiling curl-sys v0.0.1 (https://github.com/alexcrichton/curl-rust?ref=bundle#1d43e08f)
   Compiling flate2 v0.0.1 (https://github.com/alexcrichton/flate2-rs#68971ae7)
   Compiling encoding v0.1.0 (https://github.com/lifthrasiir/rust-encoding#ff55916f)
   Compiling glob v0.0.1 (https://github.com/rust-lang/glob#27338cbd)
   Compiling link-config v0.0.1 (https://github.com/alexcrichton/link-config#0afbf50b)
   Compiling tar v0.0.1 (https://github.com/alexcrichton/tar-rs#943d7c01)
   Compiling libssh2-static-sys v0.0.1 (https://github.com/alexcrichton/libssh2-static-sys#80e71a30)
   Compiling docopt v0.6.4 (https://github.com/docopt/docopt.rs#21ce1190)
   Compiling url v0.1.0 (https://github.com/servo/rust-url#55b18b79)
   Compiling curl v0.0.1 (https://github.com/alexcrichton/curl-rust?ref=bundle#1d43e08f)
   Compiling docopt_macros v0.6.4 (https://github.com/docopt/docopt.rs#21ce1190)
   Compiling libgit2 v0.0.1 (https://github.com/alexcrichton/git2-rs#c01b0b27)
Could not compile `libgit2`.

--- stderr
/Users/amitava/.cargo/git/checkouts/git2-rs-cf258e7bcbaee34d/master/libgit2/src/lib.rs:79:9: 79:14 error: no rules expected the token `const`
/Users/amitava/.cargo/git/checkouts/git2-rs-cf258e7bcbaee34d/master/libgit2/src/lib.rs:79         const GIT_REVPARSE_SINGLE = 1 << 0,

[jgallagher]

I think error: no rules expected the token const indicates your rust compiler is too old to build libgit2

[ashee]

I am on rust-0.12-dev (installed via homebrew)

extracting cargo-nightly-x86_64-apple-darwin/bin/cargo
touch target/snapshot/bin/cargo
"/usr/local/bin/rustc" -v
rustc 0.12.0-dev

[jgallagher]

@ashee const was introduced on October 10, which is probably later than your installed version. I've been using rust-nightly via:

brew tap nerdrew/tap
brew install rust-nightly

which uses this homebrew tap to download the nightly build for Mac. You have to reinstall it to update, but it lets you track the latest and greatest, which a lot of active rust projects are doing.

[nerdrew]

@jgallagher I was thinking of changing how my rust-nightly formula works. Instead of using head I was thinking of using the date as the version. Something like rust-nightly-2014-10-23. It updates nicely with homebrew then, but it always looks out of date by the next day :/

Thoughts?

[jgallagher]

@nerdrew Personally I would like that. I didn't realize it was possible (but know very little about Homebrew so no surprise there).

[nerdrew]

I'll try to push that up tonight. I hope it doesn't break your head install.

[ashee]

Thanks @jgallagher - I built rust from source and built cargo against it. Reminder - do not forget to set DYLD_LIBRARY_PATH if you used a custom --prefix during configure

export DYLD_LIBRARY_PATH=$RUST_PREFIX/lib

[tamird]

Is this good to close?

[nerdrew]

I think so. I had to unset DYLD_LIBRARY_PATH to get a recent-ish version to work (recent-ish = a couple of weeks ago).

[steveklabnik]

I do too. If anyone has something new, please just open a new issue, thanks!

[oblitum]

#16029 should be included in this analysis of issue closing.