parallel compiler: unconditional panic Resource temporarily unavailable when limiting max memory

[matthiaskrgr]

I use prlimit to limit the amount of memory that rustc can use to protect myself against OOMs.
It looks like with the parallel compiler, something might unconditionally check that threads * x amount of memory is available, even if rustc would never exhaust the given limit of memory it can take.
repro: I built with parrallel_compiler and debug assertions.
run:
prlimit and limit memory to around 3 gb, run rustc with ~100 threads, no actual file needed:
prlimit --noheadings --as=3076000000 --cpu=300 /home/matthias/.rustup/toolchains/local-debug-assertions/bin/rustc -Zthreads=100

thread 'main' panicked at /home/matthias/vcs/github/rust_debug_assertions/compiler/rustc_interface/src/util.rs:224:18:
called `Result::unwrap()` on an `Err` value: ThreadPoolBuildError { kind: IOError(Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }) }
stack backtrace:
   0:     0x7f07058ebb12 - std::backtrace_rs::backtrace::libunwind::trace::hc1a9dae7c306c6da
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:     0x7f07058ebb12 - std::backtrace_rs::backtrace::trace_unsynchronized::hc86005e3d9b0af0c
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x7f07058ebb12 - std::sys_common::backtrace::_print_fmt::h0c18d14606dddb80
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/sys_common/backtrace.rs:67:5
   3:     0x7f07058ebb12 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h14a630e4f7026ec0
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/sys_common/backtrace.rs:44:22
   4:     0x7f070593ddd7 - core::fmt::rt::Argument::fmt::h1dc8855cf0be02de
                               at /home/matthias/vcs/github/rust_debug_assertions/library/core/src/fmt/rt.rs:138:9
   5:     0x7f070593ddd7 - core::fmt::write::h35c6f281391d7802
                               at /home/matthias/vcs/github/rust_debug_assertions/library/core/src/fmt/mod.rs:1094:21
   6:     0x7f07058d60c5 - std::io::Write::write_fmt::h91c13c2d835d5458
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/io/mod.rs:1714:15
   7:     0x7f07058eb8e4 - std::sys_common::backtrace::_print::h752b660aa6060693
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/sys_common/backtrace.rs:47:5
   8:     0x7f07058eb8e4 - std::sys_common::backtrace::print::h88ee6c7530532e39
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/sys_common/backtrace.rs:34:9
   9:     0x7f07058a7cef - std::panicking::panic_hook_with_disk_dump::{{closure}}::hf580cec9dcf87a53
  10:     0x7f07058a79bb - std::panicking::panic_hook_with_disk_dump::h54ffa9cc3cfae1e6
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/panicking.rs:312:9
  11:     0x7f07076d6fe0 - <unknown>
  12:     0x7f07058a8587 - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h8cf126321bc7e708
                               at /home/matthias/vcs/github/rust_debug_assertions/library/alloc/src/boxed.rs:2021:9
  13:     0x7f07058a8587 - std::panicking::rust_panic_with_hook::h7898ccc4b8a3b270
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/panicking.rs:733:13
  14:     0x7f07058ebfb7 - std::panicking::begin_panic_handler::{{closure}}::h7bbb0077b7a9be20
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/panicking.rs:621:13
  15:     0x7f07058ebda6 - std::sys_common::backtrace::__rust_end_short_backtrace::hce388d46f5f45e0d
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/sys_common/backtrace.rs:170:18
  16:     0x7f07058a80f2 - rust_begin_unwind
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/panicking.rs:617:5
  17:     0x7f070587a4d3 - core::panicking::panic_fmt::h2aa9ff874ea77c22
                               at /home/matthias/vcs/github/rust_debug_assertions/library/core/src/panicking.rs:67:14
  18:     0x7f070587aa93 - core::result::unwrap_failed::hc532a68d0a2f9588
                               at /home/matthias/vcs/github/rust_debug_assertions/library/core/src/result.rs:1652:5
  19:     0x7f070773bf8e - <unknown>
  20:     0x7f07077101cc - <unknown>
  21:     0x7f07076ce829 - <unknown>
  22:     0x7f0707743294 - <unknown>
  23:     0x7f07076d8532 - <unknown>
  24:     0x55c3874f3167 - rustc_main[5fe54228ba4f79fc]::main
                               at /home/matthias/vcs/github/rust_debug_assertions/compiler/rustc/src/main.rs:73:5
  25:     0x55c3874f3133 - <fn() as core[1cc32ee8b476061a]::ops::function::FnOnce<()>>::call_once
                               at /home/matthias/vcs/github/rust_debug_assertions/library/core/src/ops/function.rs:250:5
  26:     0x55c3874f3133 - std[cf030e505d3d186e]::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/sys_common/backtrace.rs:154:18
  27:     0x55c3874f3149 - std[cf030e505d3d186e]::rt::lang_start::<()>::{closure#0}
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/rt.rs:166:18
  28:     0x7f07058a7e64 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h74c37da8eead49ab
                               at /home/matthias/vcs/github/rust_debug_assertions/library/core/src/ops/function.rs:284:13
  29:     0x7f07058a7e64 - std::panicking::try::do_call::hed9b9418bd805a43
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/panicking.rs:524:40
  30:     0x7f07058a7e64 - std::panicking::try::h76b073192252631f
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/panicking.rs:488:19
  31:     0x7f07058a7e64 - std::panic::catch_unwind::h631850fddd96c37d
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/panic.rs:142:14
  32:     0x7f07058a7e64 - std::rt::lang_start_internal::{{closure}}::h5f2bba15d5bf5c1c
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/rt.rs:148:48
  33:     0x7f07058a7e64 - std::panicking::try::do_call::hdfc3b55302e4eea2
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/panicking.rs:524:40
  34:     0x7f07058a7e64 - std::panicking::try::hec4ea318c529990f
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/panicking.rs:488:19
  35:     0x7f07058ea3cb - std::panic::catch_unwind::h75b0bbbf45878221
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/panic.rs:142:14
  36:     0x7f07058ea3cb - std::rt::lang_start_internal::h7fa11ae9b34cccc7
                               at /home/matthias/vcs/github/rust_debug_assertions/library/std/src/rt.rs:148:20
  37:     0x55c3874f319f - main
  38:     0x7f0705427cd0 - <unknown>
  39:     0x7f0705427d8a - __libc_start_main
  40:     0x55c3874f3045 - _start
  41:                0x0 - <unknown>

error: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: please attach the file at `/home/matthias/vcs/github/rust_debug_assertions/tests/ui/F/rustc-ice-2023-08-20T10:04:59.083020745Z-2968154.txt` to your bug report

note: compiler flags: -Z threads=100

query stack during panic:
end of query stack