panic when using rustdoc on NFS

[ms140569]

I did:

RUST_BACKTRACE=1 cargo doc --verbose

with:

rustc 1.17.0-nightly (134c4a0f0 2017-03-20)
cargo 0.19.0-nightly (c995e9eb5 2017-03-17)

on:

Darwin uranos.fritz.box 14.5.0 Darwin Kernel Version 14.5.0: Sun Sep 25 22:07:15 PDT 2016; root:xnu-2782.50.9~1/RELEASE_X86_64 x86_64

and this cargofile:

[package]
name = "rpx"
version = "0.0.1"
authors = [ "Matthias Schmidt <******>" ]

[dependencies]
getopts = "0.2"

and got:

 Documenting getopts v0.2.14
     Running `rustdoc --crate-name getopts /Users/schmidtm/.cargo/registry/src/github.com-1ecc6299db9ec823/getopts-0.2.14/src/lib.rs -o /Volumes/server/data/Matthias/prj/rpx/target/doc -L dependency=/Volumes/server/data/Matthias/prj/rpx/target/debug/deps`
error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: run with `RUST_BACKTRACE=1` for a backtrace

thread 'rustc' panicked at 'could not lock `/Volumes/server/data/Matthias/prj/rpx/target/doc/.lock`: Operation not supported (os error 45)', src/librustc_data_structures/flock.rs:362
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: rustc_data_structures::flock::<impl rustc_data_structures::flock::imp::Lock>::panicking_new
   1: rustdoc::html::render::write_shared
   2: rustdoc::html::render::run
   3: rustdoc::rust_input::{{closure}}
   4: std::panicking::try::do_call
   5: __rust_maybe_catch_panic
   6: <F as alloc::boxed::FnBox<A>>::call_box
   7: std::sys::imp::thread::Thread::new::thread_start
   8: _pthread_body
   9: _pthread_start

error: Could not document `getopts`.

Caused by:
  process didn't exit successfully: `rustdoc --crate-name getopts /Users/schmidtm/.cargo/registry/src/github.com-1ecc6299db9ec823/getopts-0.2.14/src/lib.rs -o /Volumes/server/data/Matthias/prj/rpx/target/doc -L dependency=/Volumes/server/data/Matthias/prj/rpx/target/debug/deps` (exit code: 101)

HTH,

Matthias Schmidt.

Drop me a mail, if i could help: matthias dot schmidt at gmail dot com

[sfackler]

Looks like your filesystem doesn't support file locks? I know Cargo has a some logic to deal with that which should rustdoc should have as well.

[nagisa]

Looks like your filesystem doesn't support file locks?

/Volumes/server/data

This looks a lot like a network filesystem (SMB/NFS), which indeed do not support locking.

EDIT: I tried to reproduce but my remote mountpoint has a : in it which resulted in a lot of salty tears.

[ms140569]

Yepp, it's a Samba Server on my Linux Box. Thanks for the evaluation. Cheers, Matthias Am 26.03.2017 10:52 vorm. schrieb "Simonas Kazlauskas" < notifications@github.com>:

…

Looks like your filesystem doesn't support file locks? /Volumes/server/data This looks a lot like a network filesystem (SMB/NFS), which indeed do not support locking. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#40830 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAS0pfZ6-gF4MtfphphXTMSkMjTQEjUxks5rpidXgaJpZM4MpOiv> .

[areilly]

FWIW this bug also hits "cargo doc --open" under the same circumstances. The backtrace shows that this is just running rustdoc, so it's the same thing, just later versions. I'm up to "guessing_game" in Rust Lang second edition, so it was a bit of a shock to hit breakage so soon.

Yes, /User/andrew/Documents is a symlink to /Volumes/andrew, which is an SMB2 (or 3?) file share on Samba 4.7.4 running on FreeBSD-11.1-STABLE. I understand that macOS+SMB doesn't support fcntl() locks, and flock() locks are dodgy. How about using NFS-style dot-locks?

Backtrace

 Documenting libc v0.2.34
error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.22.1 (05e2e1c41 2017-11-22) running on x86_64-apple-darwin

note: run with `RUST_BACKTRACE=1` for a backtrace

thread 'rustc' panicked at 'could not lock `/Volumes/andrew/Documents/rusty_projects/guessing_game/target/doc/.lock`: Operation not supported (os error 45)', src/librustc_data_structures/flock.rs:363:12
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
   1: std::sys_common::backtrace::_print
   2: std::panicking::default_hook::{{closure}}
   3: std::panicking::default_hook
   4: std::panicking::rust_panic_with_hook
   5: std::panicking::begin_panic
   6: std::panicking::begin_panic_fmt
   7: rustc_data_structures::flock::<impl rustc_data_structures::flock::imp::Lock>::panicking_new
   8: rustdoc::html::render::write_shared
   9: rustdoc::html::render::run

error: Could not document `libc`.

Caused by:
  process didn't exit successfully: `rustdoc --crate-name libc /Users/andrew/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.34/src/lib.rs -o /Volumes/andrew/Documents/rusty_projects/guessing_game/target/doc --cfg feature="default" --cfg feature="use_std" -L dependency=/Volumes/andrew/Documents/rusty_projects/guessing_game/target/debug/deps` (exit code: 101)

</details>

[kzys]

Cargo is ignoring locks on NFS. Maybe we can do the same.

https://github.com/rust-lang/cargo/blob/ce455cd2aa3d11bab3818b8fdee87a11ffceb79b/src/cargo/util/flock.rs#L269

[kzys]

Actually I am not so sure why do we need to have this lock.

All crates' docs are generated in parallel, but it would be synchronized at the end of rustdoc execution. Wouldn't be possible to run write_shared() at the end, from the main thread, instead of using flock()?

[HankB]

Looks like I ran into the same issue trying to compile with source on an NFS share. (Both host and NFS server running Debian Stretch.)
There are errors in the source but they seem to have nothing to do with the issue (*). cargo build ends with

thread 'main' panicked at 'librustc/session/mod.rs:802: Trying to get session directory from IncrCompSession `NotInitialized`', librustc/session/mod.rs:1296:26
note: Run with `RUST_BACKTRACE=1` for a backtrace.
error: aborting due to 3 previous errors


error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.29.1 (b801ae664 2018-09-20) running on x86_64-unknown-linux-gnu

note: compiler flags: -C debuginfo=2 -C incremental --crate-type lib

note: some of the compiler flags provided by cargo are hidden

error: Could not compile `tower`.

To learn more, run the command again with --verbose.
hbarta@olive:/mnt/share/projects/tower$ 

Impact is minimal for me as I can just move the files to a local filesystem and proceed. If there is any benefit to me providing further information, let me know. (It seems like the source of the problem is already known.)

(*) I've tried code that compiles cleanly and it produces the same error.

[jettylab]

Not sure if this is specifically relevant to this issue, but I asked a Rust forum question about randomly failing builds on an NFS mounted volume under MacOS: https://users.rust-lang.org/t/can-rust-builds-be-done-in-an-nfs-mounted-volume/28191.

The advice from @Kornel on redirecting CARGO_TARGET_DIR is an effective work-around for me for now.

[jyn514]

It looks like flock::new now returns an error instead of panicking:

rust/compiler/rustc_data_structures/src/flock.rs

Line 210 in 0c11b93

Err(io::Error::new(io::ErrorKind::Other, msg))

. Is anyone still running into this issue?

[ms140569]

Hi guys - I'm still facing this issue when compiling anything hosted on my SAMBA/SMB Fileserver:

This is:
Mac OSX 10.14.6
rustc 1.48.0 (7eac88a 2020-11-16)

Stacktrace below:

   Compiling cynon v0.0.1 (/Volumes/server/data/Matthias/prj/rust/cynon)
     Running `rustc --crate-name build_script_build --edition=2018 build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=f8228d0fe8773068 -C extra-filename=-f8228d0fe8773068 --out-dir /Volumes/server/data/Matthias/prj/rust/cynon/target/debug/build/cynon-f8228d0fe8773068 -C incremental=/Volumes/server/data/Matthias/prj/rust/cynon/target/debug/incremental -L dependency=/Volumes/server/data/Matthias/prj/rust/cynon/target/debug/deps --extern prost_build=/Volumes/server/data/Matthias/prj/rust/cynon/target/debug/deps/libprost_build-0de83129d3e4f190.rlib`
error: incremental compilation: could not create session directory lock file: Operation not supported (os error 45)

thread 'rustc' panicked at 'trying to get session directory from `IncrCompSession`: NotInitialized', compiler/rustc_session/src/session.rs:874:48
stack backtrace:
   0:        0x10981baa4 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hcfc48256a5ab8835
   1:        0x109882590 - core::fmt::write::haf3903118f694c48
   2:        0x10980d0a6 - std::io::Write::write_fmt::h7385463ac87804ed
   3:        0x10982046f - std::panicking::default_hook::{{closure}}::h91bd4c58cf71392b
   4:        0x10982013d - std::panicking::default_hook::h7bd29c87df967048
   5:        0x10eb0bc73 - rustc_driver::report_ice::h36110b17656cf994
   6:        0x109820b9e - std::panicking::rust_panic_with_hook::hae2b05f08a320721
   7:        0x1098206fb - std::panicking::begin_panic_handler::{{closure}}::h72d68d3a77e0b718
   8:        0x10981bf18 - std::sys_common::backtrace::__rust_end_short_backtrace::h7c5e286792f94edb
   9:        0x1098206ba - _rust_begin_unwind
  10:        0x1098a6e4b - std::panicking::begin_panic_fmt::h6826a3ebe3a95a51
  11:        0x113069e39 - rustc_session::session::Session::incr_comp_session_dir::h9e4df312765624b0
  12:        0x111fd4c20 - rustc_incremental::persist::fs::garbage_collect_session_directories::hc278153e85e5a69e
  13:        0x10ecd51e5 - rustc_session::utils::<impl rustc_session::session::Session>::time::hfb85c7a36ca241c4
  14:        0x10ed15384 - rustc_interface::passes::register_plugins::hcbb989c71900c709
  15:        0x10ed48eb8 - rustc_interface::queries::Queries::register_plugins::hbe269a7365ea56fb
  16:        0x10eb4a093 - rustc_interface::queries::<impl rustc_interface::interface::Compiler>::enter::heacbf49337f5fb9d
  17:        0x10eb27321 - rustc_span::with_source_map::h3927afb6bcf6a169
  18:        0x10eb457c9 - scoped_tls::ScopedKey<T>::set::h5608e7cc9397961b
  19:        0x10eb4d644 - std::sys_common::backtrace::__rust_begin_short_backtrace::h9b82f31f81e95856
  20:        0x10eae05bc - core::ops::function::FnOnce::call_once{{vtable.shim}}::h5baa2aca5ae17340
  21:        0x10982eb7d - std::sys::unix::thread::Thread::new::thread_start::he3e6719579180a65
  22:     0x7fff5a5cd2eb - __pthread_body
  23:     0x7fff5a5d0249 - __pthread_start

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.48.0 (7eac88abb 2020-11-16) running on x86_64-apple-darwin

note: compiler flags: -C embed-bitcode=no -C debuginfo=2 -C incremental --crate-type bin

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
end of query stack
error: aborting due to previous error

error: could not compile `cynon`

Caused by:
  process didn't exit successfully: `rustc --crate-name build_script_build --edition=2018 build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=f8228d0fe8773068 -C extra-filename=-f8228d0fe8773068 --out-dir /Volumes/server/data/Matthias/prj/rust/cynon/target/debug/build/cynon-f8228d0fe8773068 -C incremental=/Volumes/server/data/Matthias/prj/rust/cynon/target/debug/incremental -L dependency=/Volumes/server/data/Matthias/prj/rust/cynon/target/debug/deps --extern prost_build=/Volumes/server/data/Matthias/prj/rust/cynon/target/debug/deps/libprost_build-0de83129d3e4f190.rlib` (exit code: 101)

[jyn514]

@ms140569 that's a separate error - you're running rustc, not rustdoc, and it's an issue with incremental compilation. I'd open a new issue.

I'm going to close this since it seems like the code has been fixed and no one is running into rustdoc errors. Feel free to comment if you're still seeing an issue from rustdoc (and you can't reproduce with rustc directly).

[ms140569]

Cool, thanks! Sorry for mixing these issues ... Could you send me the issue number against rustc? Cheers, Matthias

[jyn514]

@ms140569 #76251 looks similar ( I searched label:A-incr-comp NFS).