ICE: failed to get layout

[remram44]

   Compiling rs-web-tk v0.1.0 (file:///Users/remram/Documents/programming/rs-web-tk)
error: internal compiler error: src/librustc_trans/context.rs:739: failed to get layout for `&mut futures::future::join::MaybeDone<tk_listen::Listen<futures::stream::Map<tk_listen::SleepOnError<tokio_core::net::Incoming>, [closure@src/main.rs:85:14: 136:10 sender:&std::sync::Arc<futures_mpsc::Sender<std::string::String>>, outsockets:&std::sync::Arc<std::sync::Mutex<(std::collections::HashMap<i32, tk_bufstream::WriteFramed<tokio_core::net::TcpStream, tk_http::websocket::ServerCodec>>, i32)>>, cfg:&std::sync::Arc<tk_http::server::Config>, h1:&tokio_core::reactor::Handle]>>>`: the type `futures::future::join::MaybeDone<tk_listen::Listen<futures::stream::Map<tk_listen::SleepOnError<tokio_core::net::Incoming>, [closure@src/main.rs:85:14: 136:10 sender:&std::sync::Arc<futures_mpsc::Sender<std::string::String>>, outsockets:&std::sync::Arc<std::sync::Mutex<(std::collections::HashMap<i32, tk_bufstream::WriteFramed<tokio_core::net::TcpStream, tk_http::websocket::ServerCodec>>, i32)>>, cfg:&std::sync::Arc<tk_http::server::Config>, h1:&tokio_core::reactor::Handle]>>>` has an unknown layout

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.20.0-nightly (720c596ec 2017-07-08) running on x86_64-apple-darwin

note: run with `RUST_BACKTRACE=1` for a backtrace

thread 'rustc' panicked at 'Box<Any>', src/librustc_errors/lib.rs:489:8
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
   1: std::panicking::default_hook::{{closure}}
   2: std::panicking::default_hook
   3: std::panicking::rust_panic_with_hook
   4: std::panicking::begin_panic_new
   5: rustc_errors::Handler::bug
   6: rustc::session::opt_span_bug_fmt::{{closure}}
   7: rustc::session::opt_span_bug_fmt
   8: rustc::session::bug_fmt
   9: <&'a rustc_trans::context::SharedCrateContext<'a, 'tcx> as rustc::ty::layout::LayoutTyper<'tcx>>::layout_of
  10: rustc_trans::abi::FnType::unadjusted::{{closure}}
  11: rustc_trans::abi::FnType::unadjusted
  12: rustc_trans::declare::declare_fn
  13: rustc_trans::trans_item::TransItem::predefine
  14: rustc_trans::base::trans_crate
  15: rustc_driver::driver::phase_4_translate_to_llvm
  16: rustc_driver::driver::compile_input::{{closure}}
  17: rustc_driver::driver::phase_3_run_analysis_passes::{{closure}}
  18: rustc_driver::driver::phase_3_run_analysis_passes
  19: rustc_driver::driver::compile_input
  20: rustc_driver::run_compiler

error: Could not compile `rs-web-tk`.

Code: https://gitlab.com/remram44/rs-web/blob/bf5066037ffb627c98b4988ff7d46b054ce33e40/src/main.rs#L92

Using versions (similar crash; logs):
rustc 1.20.0-nightly (720c596 2017-07-08)
rustc 1.18.0 (03fc9d6 2017-06-06)

I don't know too much about the compiler internals, but a bug search for "layout_of" didn't bring up anything that seemed relevant. So here you go.

Seems similar to #37096 but I'm not using impl Trait?

[est31]

Minimized a bit (52 lines):

(expand)

extern crate futures;
extern crate futures_mpsc;
extern crate tk_http;
extern crate tk_listen;
extern crate tokio_core;

use std::sync::Arc;

use futures::{Future, Sink, Stream};
use futures::future::FutureResult;
use futures::stream::empty as empty_stream;
use tokio_core::reactor::Core;
use tk_http::server::buffered::{Request, BufferedDispatcher};
use tk_http::server::{Encoder, EncoderDone, Config, Proto, Error};
use tk_listen::ListenExt;

use tokio_core::net::TcpStream;
use std::net::SocketAddr;

fn http<S>(_: Request, _: Encoder<S>)
    -> FutureResult<EncoderDone<S>, Error>
{
    unimplemented!()
}

fn main() {
    let h = Core::new().unwrap().handle();

    let (sender, receiver) = futures_mpsc::channel::<()>(1);
    let sender = Arc::new(sender);

    let _ = empty_stream::<(TcpStream, SocketAddr), ()>()
        .map(|(socket, addr)| {
            let sender = sender.clone();
            Proto::new(socket, &Config::new().done(),
                BufferedDispatcher::new_with_websockets(addr, &h,
                    http,
                    move |_, _| {
                        let sender = (*sender).clone();
                        futures::future::ok(()).and_then(move |_| {
                            empty_stream::<(), String>().fold(sender, |sender, _| {
                                sender.send(()).map_err(|_| "")
                            })
                        })
                        .then(|_| Ok(()))
                    }),
                &h)
            .then(|_| Ok(()))
        })
        .listen(1000)
        .join(receiver.for_each(|_| Ok(())));
}

[alexcrichton]

This showed up in the futures crate's test suite as well

[alexcrichton]

A smaller version of what I believe is the same ICE:

extern crate futures;

use futures::{Future, Stream, Sink};
use futures::stream::iter;

fn foo<T>() -> T {
    loop {}
}

fn main() {
    iter(foo::<std::iter::Cloned<std::slice::Iter<i32>>>().map(Ok))
        .forward(
            foo::<Box<Sink<SinkItem = i32, SinkError = ()>>>()
        )
        .join(
            foo::<Box<Stream<Item = i32, Error = ()>>>()
                .for_each(|_| Ok(()))
        );
}

[est31]

@alexcrichton yeah now that I look at it is that on stable OP's example has a different ICE error than on nightly, very alike to #40274, while the one on nightly matches your example.

[est31]

Bisected the nightly ICE using bisect-rust to #42840 (9b85e1c) -- cc @arielb1 @nikomatsakis

[arielb1]

Looks like the projection cache is dropping normalization obligations on the floor at https://github.com/rust-lang/rust/blob/master/src/librustc/traits/project.rs#L1392:

fn opt_normalize_projection_type<'a, 'b, 'gcx, 'tcx>( //
// ...
            let result =             // ...
                Normalized {
                    value: normalized_ty,
                    obligations,
                } // ...
            infcx.projection_cache.borrow_mut()
                                  .complete(projection_ty, &result, cacheable);
            Some(result)
// ...
    fn complete(&mut self,
                key: ty::ProjectionTy<'tcx>,
                value: &NormalizedTy<'tcx>,
                cacheable: bool) {
        let fresh_key = if cacheable {
            debug!("ProjectionCacheEntry::complete: adding cache entry: key={:?}, value={:?}",
                   key, value);
            // <-- WHERE DID value.obligations JUST DISAPPEAR TO?
            self.map.insert(key, ProjectionCacheEntry::NormalizedTy(value.value))
        } else {
            debug!("ProjectionCacheEntry::complete: cannot cache: key={:?}, value={:?}",
                   key, value);
            !self.map.remove(key)
        };

        assert!(!fresh_key, "never started projecting `{:?}`", key);
    }

This causes problems with my new evaluation code because it causes an earlier call into evaluation, which allows the cache pollution to proceed (I think).

[arielb1]

Actually, the problem is that the projection cache is incompatible with 1-pass evaluation. Not sure how to solve this.

[alexcrichton]

@arielb1 do you have further thoughts on this? The regression is now on beta :(

[aturon]

cc @eddyb

[eddyb]

I don't have any specific knowledge of how the caches involved here interact. It seems like layout code triggers this but the problem could arise in a number of other ways.

[arielb1]

I have to talk to @nikomatsakis about this.

[nikomatsakis]

I can look into this, but not until Saturday.

[nikomatsakis]

triage: P-high

[nikomatsakis]

So I have a fix pending in this branch but I'm not sure what to use for a regression test, since I don't want to add a dependency on futures.

[alexcrichton]

@nikomatsakis here's a "reasonably sized" test case which reproduces the error here.

[est31]

@nikomatsakis I've minimized @alexcrichton 's example to roughly the half here.

[remram44]

The nightly is out and I no longer run into that ICE! On the other hand compiling this simple program now takes 8 minutes. The previous commit (before I introduced the ICE) is up from 6s to 1m51s.

[est31]

@remram44 I suggest you file a separate bug report for this.

[alexcrichton]

@remram44 hm I can't seem to get the commit linked above to compile? What compiler are you using to compile it "pre ICE"?

[remram44]

I'm compiling using stable and the nightly from the 28th from rustup... not sure why this would fail for you?

[alexcrichton]

@remram44 sorry I still can't get the code to compile. Can you list the exact commands you're using, along with the exact toolchain in rustup-style syntax?

[remram44]

The toolchains I tried are nightly-2017-08-01-x86_64-apple-darwin and nightly-2017-08-01-x86_64-unknown-linux-gnu.

Whatever your environment you can reproduce this from scratch in Docker with:

% docker run -ti --rm ubuntu:16.04
$ apt update && apt install curl gcc git
$ curl https://sh.rustup.rs -sSf | sh
$ . ~/.cargo/env
$ rustup toolchain install nightly-2017-07-28
$ rustup toolchain install nightly-2017-08-01
$ git clone https://gitlab.com/remram44/rs-web --single-branch --branch ICE
$ cd rs-web
$ cargo +nightly-2017-07-28 build
$ # Observe ICE
$ cargo +nightly-2017-08-01 build
$ # Observe no ICE, build time 8 minutes
$ git checkout 8fca3d957f91115349aaeb3ab58056b92d13b953
$ # Now on previous version, that always worked
$ cargo +nightly-2017-07-28 build
$ # Observe build time 6 seconds
$ cargo +nightly-2017-08-01 build
$ # Observe build time 1 minute 51 seconds

[alexcrichton]

Ok thanks for the repro @remram44! Want to open a new issue with that so we can be sure to track it?

[remram44]

Will do!

[edit: #43787]