Reduce interning overhead #11262
Merged
Reduce interning overhead #11262
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Building wheels and fs_util will be skipped. Delete if not intended. [ci skip-build-wheels]
… the GIL. [ci skip-build-wheels]
Eric-Arellano
approved these changes
Dec 3, 2020
# Building wheels and fs_util will be skipped. Delete if not intended. [ci skip-build-wheels]
jsirois
reviewed
May 30, 2021
| } | ||
| py.allow_threads(|| { | ||
| let (key, key_was_new) = { | ||
| let mut forward_keys = self.forward_keys.lock(); |
There was a problem hiding this comment.
I've observed the panic below in key_get in a SIGILL coredump. Looking here its seems clear this is "by design" since the forward keys are updated under a seperate lock from the reverse keys. With the following interleaving you get a panic:
- TH#1: key_insert(py, value): let (key, ley_was_new) = (key, true) [SUSPEND]
- TH#2: key_insert(py, value): let (key, ley_was_new) = (key, false) -> Ok(key)
- TH#<not 1>: key_get(key): panic!
Unless there is some higher level invariant expected to never allow this interleaving to happen this looks like a core-dump confirmed bug.
This in fact leads to a panic while processing panic via this stack:
$ coredumpctl debug 970710
PID: 970710 (pex)
UID: 1000 (jsirois)
GID: 1000 (jsirois)
Signal: 4 (ILL)
Timestamp: Sat 2021-05-29 20:48:50 MDT (7min ago)
Command Line: /home/jsirois/.cache/pants/named_caches/pex_root/venvs/short/9d94121e/bin/python3.7 -sE /tmp/process-executionAXm2tS/.cache/pex_root/venvs/80d44d2dea5dba12cfceb550342549b7200cdfda/d9ae00d5d551b738418a65d94b6a85ab0547f4b9/pex --no-header -vvs src/python/pants/core/goals/test_test.py
Executable: /usr/bin/python3.7
Control Group: /user.slice/user-1000.slice/user@1000.service/app.slice/vte-spawn-7617fd64-4c64-4761-b578-7923c3d09dca.scope
Unit: user@1000.service
User Unit: vte-spawn-7617fd64-4c64-4761-b578-7923c3d09dca.scope
Slice: user-1000.slice
Owner UID: 1000 (jsirois)
Boot ID: c8d9dda1aaf4402985aafe2be432a3db
Machine ID: c8b6b48c23a840ee862c8d2983461941
Hostname: gill
Storage: /var/lib/systemd/coredump/core.pex.1000.c8d9dda1aaf4402985aafe2be432a3db.970710.1622342930000000.zst (present)
Disk Size: 11.2M
Message: Process 970710 (pex) of user 1000 dumped core.
Stack trace of thread 970959:
#0 0x00007f83c0cc3fec n/a (/tmp/process-executionAXm2tS/src/python/pants/engine/internals/native_engine.so + 0x1b9afec)
GNU gdb (GDB) 10.2
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/python3.7...
(No debugging symbols found in /usr/bin/python3.7)
warning: core file may not match specified executable file.
[New LWP 970959]
[New LWP 970957]
[New LWP 970710]
[New LWP 970956]
[New LWP 970958]
[New LWP 971232]
[New LWP 971590]
[New LWP 970955]
[New LWP 970954]
[New LWP 971565]
[New LWP 970953]
[New LWP 971566]
[New LWP 970960]
[New LWP 971593]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Core was generated by `/home/jsirois/.cache/pants/named_caches/pex_root/venvs/short/9d94121e/bin/pytho'.
Program terminated with signal SIGILL, Illegal instruction.
#0 0x00007f83c0cc3fec in std::panicking::rust_panic_with_hook () at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/fmt/mod.rs:314
314 /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/fmt/mod.rs: No such file or directory.
[Current thread is 1 (Thread 0x7f83bd763640 (LWP 970959))]
(gdb) bt
#0 0x00007f83c0cc3fec in std::panicking::rust_panic_with_hook () at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/fmt/mod.rs:314
#1 0x00007f83c0cc3c07 in std::panicking::begin_panic_handler::{{closure}} () at library/std/src/panicking.rs:497
#2 0x00007f83c0cc16dc in std::sys_common::backtrace::__rust_end_short_backtrace<closure-0,!> () at library/std/src/sys_common/backtrace.rs:141
#3 0x00007f83c0cc3b69 in std::panicking::begin_panic_handler () at library/std/src/panicking.rs:493
#4 0x00007f83bf38b1db in std::panicking::begin_panic_fmt () at library/std/src/panicking.rs:435
#5 0x00007f83bf58c724 in engine::interning::{{impl}}::key_get::{{closure}} () at src/interning.rs:87
#6 0x00007f83bf406e90 in core::option::Option<engine::core::Value>::unwrap_or_else<engine::core::Value,closure-0> (self=..., f=...) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:427
#7 0x00007f83bf58c666 in engine::interning::Interns::key_get (self=0x7f83c1510eb0 <<engine::externs::INTERNS as core::ops::deref::Deref>::deref::__stability::LAZY+8>, k=0x7f839c0e1e58) at src/interning.rs:82
#8 0x00007f83bf722aae in engine::externs::val_for (key=0x7f839c0e1e58) at src/externs/mod.rs:90
#9 0x00007f83bf7283d5 in engine::externs::key_to_str (key=0x7f839c0e1e58) at src/externs/mod.rs:229
#10 0x00007f83bf5bc86c in engine::core::{{impl}}::fmt (self=0x7f839c0e1e58, f=0x7f83bd73f4d0) at src/core.rs:227
#11 0x00007f83bf5ae526 in core::fmt::{{impl}}::fmt<engine::core::Key> (self=0x7f83bd73f7a8, f=0x7f83bd73f4d0) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/fmt/mod.rs:2010
#12 0x00007f83c0ce41bf in core::fmt::write () at library/core/src/fmt/mod.rs:1092
#13 0x00007f83c0cc3db8 in core::fmt::Write::write_fmt<alloc::string::String> () at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/fmt/mod.rs:182
#14 std::panicking::begin_panic_handler::{{impl}}::fill::{{closure}} () at library/std/src/panicking.rs:459
#15 core::option::Option::get_or_insert_with<alloc::string::String,closure-0> () at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/option.rs:929
#16 std::panicking::begin_panic_handler::PanicPayload::fill () at library/std/src/panicking.rs:457
#17 std::panicking::begin_panic_handler::{{impl}}::get () at library/std/src/panicking.rs:475
#18 0x00007f83c0cc405c in std::panicking::rust_panic_with_hook () at library/std/src/panicking.rs:590
#19 0x00007f83c0cc3c07 in std::panicking::begin_panic_handler::{{closure}} () at library/std/src/panicking.rs:497
#20 0x00007f83c0cc16dc in std::sys_common::backtrace::__rust_end_short_backtrace<closure-0,!> () at library/std/src/sys_common/backtrace.rs:141
#21 0x00007f83c0cc3b69 in std::panicking::begin_panic_handler () at library/std/src/panicking.rs:493
#22 0x00007f83bf38b1db in std::panicking::begin_panic_fmt () at library/std/src/panicking.rs:435
#23 0x00007f83bf58c724 in engine::interning::{{impl}}::key_get::{{closure}} () at src/interning.rs:87
#24 0x00007f83bf406e90 in core::option::Option<engine::core::Value>::unwrap_or_else<engine::core::Value,closure-0> (self=..., f=...) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:427
#25 0x00007f83bf58c666 in engine::interning::Interns::key_get (self=0x7f83c1510eb0 <<engine::externs::INTERNS as core::ops::deref::Deref>::deref::__stability::LAZY+8>, k=0x7f839c0e1e58) at src/interning.rs:82
#26 0x00007f83bf722aae in engine::externs::val_for (key=0x7f839c0e1e58) at src/externs/mod.rs:90
#27 0x00007f83bf7283d5 in engine::externs::key_to_str (key=0x7f839c0e1e58) at src/externs/mod.rs:229
#28 0x00007f83bf5bc86c in engine::core::{{impl}}::fmt (self=0x7f839c0e1e58, f=0x7f83bd73f940) at src/core.rs:227
#29 0x00007f83bf5ae526 in core::fmt::{{impl}}::fmt<engine::core::Key> (self=0x7f83bd73fc18, f=0x7f83bd73f940) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/fmt/mod.rs:2010
#30 0x00007f83c0ce41bf in core::fmt::write () at library/core/src/fmt/mod.rs:1092
#31 0x00007f83c0cc3db8 in core::fmt::Write::write_fmt<alloc::string::String> () at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/fmt/mod.rs:182
#32 std::panicking::begin_panic_handler::{{impl}}::fill::{{closure}} () at library/std/src/panicking.rs:459
#33 core::option::Option::get_or_insert_with<alloc::string::String,closure-0> () at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/option.rs:929
#34 std::panicking::begin_panic_handler::PanicPayload::fill () at library/std/src/panicking.rs:457
#35 std::panicking::begin_panic_handler::{{impl}}::get () at library/std/src/panicking.rs:475
#36 0x00007f83c0cc405c in std::panicking::rust_panic_with_hook () at library/std/src/panicking.rs:590
#37 0x00007f83c0cc3c07 in std::panicking::begin_panic_handler::{{closure}} () at library/std/src/panicking.rs:497
#38 0x00007f83c0cc16dc in std::sys_common::backtrace::__rust_end_short_backtrace<closure-0,!> () at library/std/src/sys_common/backtrace.rs:141
#39 0x00007f83c0cc3b69 in std::panicking::begin_panic_handler () at library/std/src/panicking.rs:493
#40 0x00007f83bf38b1db in std::panicking::begin_panic_fmt () at library/std/src/panicking.rs:435
#41 0x00007f83bf58c724 in engine::interning::{{impl}}::key_get::{{closure}} () at src/interning.rs:87
#42 0x00007f83bf406e90 in core::option::Option<engine::core::Value>::unwrap_or_else<engine::core::Value,closure-0> (self=..., f=...) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:427
#43 0x00007f83bf58c666 in engine::interning::Interns::key_get (self=0x7f83c1510eb0 <<engine::externs::INTERNS as core::ops::deref::Deref>::deref::__stability::LAZY+8>, k=0x7f839c0e1e58) at src/interning.rs:82
#44 0x00007f83bf722aae in engine::externs::val_for (key=0x7f839c0e1e58) at src/externs/mod.rs:90
#45 0x00007f83bf532f9d in engine::nodes::{{impl}}::run_wrapped_node::__run_wrapped_node::{{closure}} () at src/nodes.rs:192
#46 0x00007f83bf4d8761 in core::future::from_generator::{{impl}}::poll<generator-0> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#47 0x00007f83bf43e935 in core::future::future::{{impl}}::poll<alloc::boxed::Box<Future, alloc::alloc::Global>> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/future.rs:119
#48 0x00007f83bf5215ac in engine::nodes::{{impl}}::select_product::{{closure}} () at src/nodes.rs:147
#49 0x00007f83bf4d0ce1 in core::future::from_generator::{{impl}}::poll<generator-0> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#50 0x00007f83bf4acd8a in futures_core::future::{{impl}}::try_poll<core::future::from_generator::GenFuture<generator-0>,engine::core::Value,engine::core::Failure> (self=..., cx=0x7f83bd761658)
at /home/jsirois/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-core-0.3.13/src/future.rs:84
#51 0x00007f83bf54da6c in futures_util::future::try_maybe_done::{{impl}}::poll<core::future::from_generator::GenFuture<generator-0>> (self=..., cx=0x7f83bd761658) at /home/jsirois/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.8/src/future/try_maybe_done.rs:72
#52 0x00007f83bf573c6a in futures_core::future::{{impl}}::try_poll<futures_util::future::try_maybe_done::TryMaybeDone<core::future::from_generator::GenFuture<generator-0>>,(),engine::core::Failure> (self=..., cx=0x7f83bd761658)
at /home/jsirois/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-core-0.3.13/src/future.rs:84
#53 0x00007f83bf5c3314 in futures_util::future::try_join_all::{{impl}}::poll<core::future::from_generator::GenFuture<generator-0>> (self=..., cx=0x7f83bd761658) at /home/jsirois/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.8/src/future/try_join_all.rs:111
#54 0x00007f83bf532a7b in engine::nodes::{{impl}}::run_wrapped_node::__run_wrapped_node::{{closure}} () at src/nodes.rs:174
#55 0x00007f83bf4d8761 in core::future::from_generator::{{impl}}::poll<generator-0> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#56 0x00007f83bf43e935 in core::future::future::{{impl}}::poll<alloc::boxed::Box<Future, alloc::alloc::Global>> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/future.rs:119
#57 0x00007f83bf52d8a0 in engine::nodes::{{impl}}::gen_get::{{closure}}::{{closure}}::{{closure}} () at src/nodes.rs:980
#58 0x00007f83bf4d1ac1 in core::future::from_generator::{{impl}}::poll<generator-2> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#59 0x00007f83bf4acd0a in futures_core::future::{{impl}}::try_poll<core::future::from_generator::GenFuture<generator-2>,engine::core::Value,engine::core::Failure> (self=..., cx=0x7f83bd761658)
at /home/jsirois/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-core-0.3.13/src/future.rs:84
#60 0x00007f83bf54df4c in futures_util::future::try_maybe_done::{{impl}}::poll<core::future::from_generator::GenFuture<generator-2>> (self=..., cx=0x7f83bd761658) at /home/jsirois/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.8/src/future/try_maybe_done.rs:72
#61 0x00007f83bf573b3a in futures_core::future::{{impl}}::try_poll<futures_util::future::try_maybe_done::TryMaybeDone<core::future::from_generator::GenFuture<generator-2>>,(),engine::core::Failure> (self=..., cx=0x7f83bd761658)
at /home/jsirois/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-core-0.3.13/src/future.rs:84
#62 0x00007f83bf5bfdf4 in futures_util::future::try_join_all::{{impl}}::poll<core::future::from_generator::GenFuture<generator-2>> (self=..., cx=0x7f83bd761658) at /home/jsirois/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.8/src/future/try_join_all.rs:111
#63 0x00007f83bf52e12e in engine::nodes::{{impl}}::gen_get::{{closure}} () at src/nodes.rs:986
#64 0x00007f83bf4d2fc1 in core::future::from_generator::{{impl}}::poll<generator-0> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#65 0x00007f83bf52e89d in engine::nodes::{{impl}}::generate::{{closure}} () at src/nodes.rs:1006
#66 0x00007f83bf4d2171 in core::future::from_generator::{{impl}}::poll<generator-0> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#67 0x00007f83bf53a828 in engine::nodes::{{impl}}::run_wrapped_node::__run_wrapped_node::{{closure}} () at src/nodes.rs:1074
#68 0x00007f83bf4da367 in core::future::from_generator::{{impl}}::poll<generator-0> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#69 0x00007f83bf43ecb5 in core::future::future::{{impl}}::poll<alloc::boxed::Box<Future, alloc::alloc::Global>> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/future.rs:119
#70 0x00007f83bf4185aa in futures_core::future::{{impl}}::try_poll<core::pin::Pin<alloc::boxed::Box<Future, alloc::alloc::Global>>,engine::nodes::PythonRuleOutput,engine::core::Failure> (self=..., cx=0x7f83bd761658)
at /home/jsirois/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-core-0.3.13/src/future.rs:84
#71 0x00007f83bf54f546 in futures_util::future::try_future::into_future::{{impl}}::poll<core::pin::Pin<alloc::boxed::Box<Future, alloc::alloc::Global>>> (self=..., cx=0x7f83bd761658)
at /home/jsirois/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.8/src/future/try_future/into_future.rs:31
#72 0x00007f83bf593474 in futures_util::future::future::map::{{impl}}::poll<futures_util::future::try_future::into_future::IntoFuture<core::pin::Pin<alloc::boxed::Box<Future, alloc::alloc::Global>>>,futures_util::fns::MapOkFn<closure-2>,core::result::Result<engine::nodes::NodeOutput, engine::core::Failure>> (self=..., cx=0x7f83bd761658) at /home/jsirois/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.8/src/future/future/map.rs:49
#73 0x00007f83bf4e3236 in futures_util::future::future::{{impl}}::poll<futures_util::future::try_future::into_future::IntoFuture<core::pin::Pin<alloc::boxed::Box<Future, alloc::alloc::Global>>>,futures_util::fns::MapOkFn<closure-2>> (self=..., cx=0x7f83bd761658)
at /home/jsirois/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.8/src/lib.rs:113
#74 0x00007f83bf82d5d6 in futures_util::future::try_future::{{impl}}::poll<core::pin::Pin<alloc::boxed::Box<Future, alloc::alloc::Global>>,closure-2> (self=..., cx=0x7f83bd761658) at /home/jsirois/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.8/src/lib.rs:113
#75 0x00007f83bf53ee82 in engine::nodes::{{impl}}::run::__run::{{closure}}::{{closure}} () at src/nodes.rs:1357
#76 0x00007f83bf4dcfc7 in core::future::from_generator::{{impl}}::poll<generator-1> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#77 0x00007f83bf558589 in workunit_store::with_workunit::{{closure}}::{{closure}}<core::future::from_generator::GenFuture<generator-1>,closure-2> () at /home/jsirois/dev/pantsbuild/jsirois-pants/src/rust/engine/workunit_store/src/lib.rs:828
#78 0x00007f83bf4df737 in core::future::from_generator::{{impl}}::poll<generator-0> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#79 0x00007f83bf73130f in tokio::task::task_local::{{impl}}::poll<core::option::Option<workunit_store::WorkunitStoreHandle>,core::future::from_generator::GenFuture<generator-0>> (self=..., cx=0x7f83bd761658)
at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/task/task_local.rs:216
#80 0x00007f83bf74bb55 in tokio::task::task_local::{{impl}}::scope::{{closure}}<core::option::Option<workunit_store::WorkunitStoreHandle>,core::future::from_generator::GenFuture<generator-0>> ()
at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/task/task_local.rs:122
#81 0x00007f83bf4db5b7 in core::future::from_generator::{{impl}}::poll<generator-0> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#82 0x00007f83bf55c833 in workunit_store::scope_task_workunit_store_handle::{{closure}}<core::future::from_generator::GenFuture<generator-0>> () at /home/jsirois/dev/pantsbuild/jsirois-pants/src/rust/engine/workunit_store/src/lib.rs:901
#83 0x00007f83bf4d3727 in core::future::from_generator::{{impl}}::poll<generator-0> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#84 0x00007f83bf5580d2 in workunit_store::with_workunit::{{closure}}<core::future::from_generator::GenFuture<generator-1>,closure-2> () at /home/jsirois/dev/pantsbuild/jsirois-pants/src/rust/engine/workunit_store/src/lib.rs:827
#85 0x00007f83bf4d4b87 in core::future::from_generator::{{impl}}::poll<generator-0> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#86 0x00007f83bf540d13 in engine::nodes::{{impl}}::run::__run::{{closure}} () at src/nodes.rs:1404
#87 0x00007f83bf4d3821 in core::future::from_generator::{{impl}}::poll<generator-0> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#88 0x00007f83bf43e745 in core::future::future::{{impl}}::poll<alloc::boxed::Box<Future, alloc::alloc::Global>> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/future.rs:119
#89 0x00007f83bf84e3a2 in graph::entry::{{impl}}::spawn_node_execution::{{closure}}<engine::nodes::NodeKey> () at /home/jsirois/dev/pantsbuild/jsirois-pants/src/rust/engine/graph/src/entry.rs:345
#90 0x00007f83bf4da561 in core::future::from_generator::{{impl}}::poll<generator-0> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#91 0x00007f83bf84f533 in graph::entry::{{impl}}::spawn_node_execution::{{closure}}::{{closure}}<engine::nodes::NodeKey> (cx=0x7f83bd761658) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/macros/select.rs:455
#92 0x00007f83bf584d58 in tokio::future::poll_fn::{{impl}}::poll<graph::entry::{{impl}}::spawn_node_execution::{{closure}}::util::Out<(), core::option::Option<core::result::Result<engine::nodes::NodeOutput, engine::core::Failure>>>,closure-0> (self=..., cx=0x7f83bd761658)
at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/future/poll_fn.rs:38
#93 0x00007f83bf84ec76 in graph::entry::{{impl}}::spawn_node_execution::{{closure}}<engine::nodes::NodeKey> () at /home/jsirois/dev/pantsbuild/jsirois-pants/src/rust/engine/graph/src/entry.rs:352
#94 0x00007f83bf4ddafb in core::future::from_generator::{{impl}}::poll<generator-1> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#95 0x00007f83bf7302b6 in tokio::task::task_local::{{impl}}::poll<core::option::Option<workunit_store::WorkunitStoreHandle>,core::future::from_generator::GenFuture<generator-1>> (self=..., cx=0x7f83bd761658)
at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/task/task_local.rs:216
#96 0x00007f83bf74e97d in tokio::task::task_local::{{impl}}::scope::{{closure}}<core::option::Option<workunit_store::WorkunitStoreHandle>,core::future::from_generator::GenFuture<generator-1>> ()
at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/task/task_local.rs:122
#97 0x00007f83bf4dde6b in core::future::from_generator::{{impl}}::poll<generator-0> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#98 0x00007f83bf55d5b5 in workunit_store::scope_task_workunit_store_handle::{{closure}}<core::future::from_generator::GenFuture<generator-1>> () at /home/jsirois/dev/pantsbuild/jsirois-pants/src/rust/engine/workunit_store/src/lib.rs:901
#99 0x00007f83bf4d680b in core::future::from_generator::{{impl}}::poll<generator-0> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#100 0x00007f83bf738a2c in task_executor::{{impl}}::future_with_correct_context::{{closure}}<core::future::from_generator::GenFuture<generator-1>> () at /home/jsirois/dev/pantsbuild/jsirois-pants/src/rust/engine/task_executor/src/lib.rs:186
#101 0x00007f83bf4dadcb in core::future::from_generator::{{impl}}::poll<generator-0> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#102 0x00007f83bf72fd21 in tokio::task::task_local::{{impl}}::poll<alloc::sync::Arc<stdio::Destination>,core::future::from_generator::GenFuture<generator-0>> (self=..., cx=0x7f83bd761658)
at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/task/task_local.rs:216
#103 0x00007f83bf74a7e4 in tokio::task::task_local::{{impl}}::scope::{{closure}}<alloc::sync::Arc<stdio::Destination>,core::future::from_generator::GenFuture<generator-0>> () at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/task/task_local.rs:122
#104 0x00007f83bf4d1fab in core::future::from_generator::{{impl}}::poll<generator-0> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#105 0x00007f83bf817eca in stdio::scope_task_destination::{{closure}}<core::future::from_generator::GenFuture<generator-0>> () at /home/jsirois/dev/pantsbuild/jsirois-pants/src/rust/engine/stdio/src/lib.rs:425
#106 0x00007f83bf4d445b in core::future::from_generator::{{impl}}::poll<generator-0> (self=..., cx=0x7f83bd761658) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
#107 0x00007f83bf7ee1cf in tokio::runtime::task::core::{{impl}}::poll::{{closure}}<core::future::from_generator::GenFuture<generator-0>> (ptr=0x7f83980166b8) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/task/core.rs:235
#108 0x00007f83bf5b7d2a in tokio::loom::std::unsafe_cell::UnsafeCell<tokio::runtime::task::core::Stage<core::future::from_generator::GenFuture<generator-0>>>::with_mut<tokio::runtime::task::core::Stage<core::future::from_generator::GenFuture<generator-0>>,core::task::poll::Poll<()>,closure-0> (self=0x7f83980166b8, f=...) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/loom/std/unsafe_cell.rs:14
#109 0x00007f83bf7ed087 in tokio::runtime::task::core::CoreStage<core::future::from_generator::GenFuture<generator-0>>::poll<core::future::from_generator::GenFuture<generator-0>> (self=0x7f83980166b8, cx=...)
at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/task/core.rs:225
#110 0x00007f83bf42868d in tokio::runtime::task::harness::poll_future<core::future::from_generator::GenFuture<generator-0>> (header=0x7f8398016680, core=0x7f83980166b8, snapshot=..., cx=...)
at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/task/harness.rs:411
#111 0x00007f83bf42a9bb in tokio::runtime::task::harness::Harness<core::future::from_generator::GenFuture<generator-0>, alloc::sync::Arc<tokio::runtime::thread_pool::worker::Worker>>::poll_inner<core::future::from_generator::GenFuture<generator-0>,alloc::sync::Arc<tokio::runtime::threa--Type <RET> for more, q to quit, c to continue without paging--
d_pool::worker::Worker>> (self=0x7f83bd7618d0) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/task/harness.rs:88
#112 0x00007f83bf43129b in tokio::runtime::task::harness::Harness<core::future::from_generator::GenFuture<generator-0>, alloc::sync::Arc<tokio::runtime::thread_pool::worker::Worker>>::poll<core::future::from_generator::GenFuture<generator-0>,alloc::sync::Arc<tokio::runtime::thread_pool::worker::Worker>> (self=...) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/task/harness.rs:58
#113 0x00007f83bf862520 in tokio::runtime::task::raw::poll<core::future::from_generator::GenFuture<generator-0>,alloc::sync::Arc<tokio::runtime::thread_pool::worker::Worker>> (ptr=...)
at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/task/raw.rs:104
#114 0x00007f83c09874ff in tokio::runtime::task::raw::RawTask::poll (self=...) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/task/raw.rs:66
#115 0x00007f83c08edc31 in tokio::runtime::task::Notified<alloc::sync::Arc<tokio::runtime::thread_pool::worker::Worker>>::run<alloc::sync::Arc<tokio::runtime::thread_pool::worker::Worker>> (self=...)
at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/task/mod.rs:171
#116 0x00007f83c09535b2 in tokio::runtime::thread_pool::worker::{{impl}}::run_task::{{closure}} () at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/thread_pool/worker.rs:349
#117 0x00007f83c09398cc in tokio::coop::with_budget::{{closure}}<core::result::Result<alloc::boxed::Box<tokio::runtime::thread_pool::worker::Core, alloc::alloc::Global>, ()>,closure-0> (cell=0x7f839c000c8a)
at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/coop.rs:106
#118 0x00007f83c0918718 in std::thread::local::LocalKey<core::cell::Cell<tokio::coop::Budget>>::try_with<core::cell::Cell<tokio::coop::Budget>,closure-0,core::result::Result<alloc::boxed::Box<tokio::runtime::thread_pool::worker::Core, alloc::alloc::Global>, ()>> (self=0x7f83c145d1d0,
f=...) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:272
#119 0x00007f83c0917e4e in std::thread::local::LocalKey<core::cell::Cell<tokio::coop::Budget>>::with<core::cell::Cell<tokio::coop::Budget>,closure-0,core::result::Result<alloc::boxed::Box<tokio::runtime::thread_pool::worker::Core, alloc::alloc::Global>, ()>> (self=0x7f83c145d1d0,
f=...) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:248
#120 0x00007f83c09533d3 in tokio::coop::with_budget<core::result::Result<alloc::boxed::Box<tokio::runtime::thread_pool::worker::Core, alloc::alloc::Global>, ()>,closure-0> (budget=..., f=...) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/coop.rs:99
#121 tokio::coop::budget<core::result::Result<alloc::boxed::Box<tokio::runtime::thread_pool::worker::Core, alloc::alloc::Global>, ()>,closure-0> (f=...) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/coop.rs:76
#122 tokio::runtime::thread_pool::worker::Context::run_task (self=0x7f83bd761f48, task=..., core=0x556694206c50) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/thread_pool/worker.rs:348
#123 0x00007f83c0952f8b in tokio::runtime::thread_pool::worker::Context::run (self=0x7f83bd761f48, core=0x556694206c50) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/thread_pool/worker.rs:325
#124 0x00007f83c0952c23 in tokio::runtime::thread_pool::worker::run::{{closure}} () at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/thread_pool/worker.rs:303
#125 0x00007f83c08e25ed in tokio::macros::scoped_tls::ScopedKey<tokio::runtime::thread_pool::worker::Context>::set<tokio::runtime::thread_pool::worker::Context,closure-0,()> (self=0x7f83c145d468 <tokio::runtime::thread_pool::worker::CURRENT>, t=0x7f83bd761f48, f=...)
at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/macros/scoped_tls.rs:61
#126 0x00007f83c0952b03 in tokio::runtime::thread_pool::worker::run (worker=...) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/thread_pool/worker.rs:300
#127 0x00007f83c09529cb in tokio::runtime::thread_pool::worker::{{impl}}::launch::{{closure}} () at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/thread_pool/worker.rs:279
#128 0x00007f83c08eedff in tokio::runtime::blocking::task::{{impl}}::poll<closure-0,()> (self=..., _cx=0x7f83bd762148) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/blocking/task.rs:42
#129 0x00007f83c08d5aa0 in tokio::runtime::task::core::{{impl}}::poll::{{closure}}<tokio::runtime::blocking::task::BlockingTask<closure-0>> (ptr=0x556693d3c470) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/task/core.rs:235
#130 0x00007f83c0979fce in tokio::loom::std::unsafe_cell::UnsafeCell<tokio::runtime::task::core::Stage<tokio::runtime::blocking::task::BlockingTask<closure-0>>>::with_mut<tokio::runtime::task::core::Stage<tokio::runtime::blocking::task::BlockingTask<closure-0>>,core::task::poll::Poll<()>,closure-0> (self=0x556693d3c470, f=...) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/loom/std/unsafe_cell.rs:14
#131 0x00007f83c08d4ee8 in tokio::runtime::task::core::CoreStage<tokio::runtime::blocking::task::BlockingTask<closure-0>>::poll<tokio::runtime::blocking::task::BlockingTask<closure-0>> (self=0x556693d3c470, cx=...)
at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/task/core.rs:225
#132 0x00007f83c08ed371 in tokio::runtime::task::harness::poll_future<tokio::runtime::blocking::task::BlockingTask<closure-0>> (header=0x556693d3c440, core=0x556693d3c470, snapshot=..., cx=...)
at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/task/harness.rs:411
#133 0x00007f83c08e6ed3 in tokio::runtime::task::harness::Harness<tokio::runtime::blocking::task::BlockingTask<closure-0>, tokio::runtime::blocking::schedule::NoopSchedule>::poll_inner<tokio::runtime::blocking::task::BlockingTask<closure-0>,tokio::runtime::blocking::schedule::NoopSchedule> (self=0x7f83bd7623c0) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/task/harness.rs:88
#134 0x00007f83c08e6a2c in tokio::runtime::task::harness::Harness<tokio::runtime::blocking::task::BlockingTask<closure-0>, tokio::runtime::blocking::schedule::NoopSchedule>::poll<tokio::runtime::blocking::task::BlockingTask<closure-0>,tokio::runtime::blocking::schedule::NoopSchedule>
(self=...) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/task/harness.rs:58
#135 0x00007f83c0987652 in tokio::runtime::task::raw::poll<tokio::runtime::blocking::task::BlockingTask<closure-0>,tokio::runtime::blocking::schedule::NoopSchedule> (ptr=...) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/task/raw.rs:104
#136 0x00007f83c09874ff in tokio::runtime::task::raw::RawTask::poll (self=...) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/task/raw.rs:66
#137 0x00007f83c08edca1 in tokio::runtime::task::Notified<tokio::runtime::blocking::schedule::NoopSchedule>::run<tokio::runtime::blocking::schedule::NoopSchedule> (self=...) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/task/mod.rs:171
#138 0x00007f83c093e0d8 in tokio::runtime::blocking::pool::Inner::run (self=0x5566940964a0, worker_thread_id=6) at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/blocking/pool.rs:278
#139 0x00007f83c093de69 in tokio::runtime::blocking::pool::{{impl}}::spawn_thread::{{closure}} () at /home/jsirois/.cargo/git/checkouts/tokio-849b427e4e7bd338/d541685/tokio/src/runtime/blocking/pool.rs:258
#140 0x00007f83c091b2cc in std::sys_common::backtrace::__rust_begin_short_backtrace<closure-0,()> (f=...) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:125
#141 0x00007f83c0966981 in std::thread::{{impl}}::spawn_unchecked::{{closure}}::{{closure}}<closure-0,()> () at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/mod.rs:474
#142 0x00007f83c08c88c1 in std::panic::{{impl}}::call_once<(),closure-0> (self=..., _args=()) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:344
#143 0x00007f83c095828c in std::panicking::try::do_call<std::panic::AssertUnwindSafe<closure-0>,()> (data=0x7f83bd762aa8) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:379
#144 0x00007f83c0963cad in __rust_try () from /tmp/process-executionAXm2tS/src/python/pants/engine/internals/native_engine.so
#145 0x00007f83c0958111 in std::panicking::try<(),std::panic::AssertUnwindSafe<closure-0>> (f=...) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:343
#146 0x00007f83c08c90e1 in std::panic::catch_unwind<std::panic::AssertUnwindSafe<closure-0>,()> (f=...) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:431
#147 0x00007f83c0966775 in std::thread::{{impl}}::spawn_unchecked::{{closure}}<closure-0,()> () at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/mod.rs:473
#148 0x00007f83c08fa16f in core::ops::function::FnOnce::call_once<closure-0,()> () at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227
#149 0x00007f83c0ccb42a in alloc::boxed::{{impl}}::call_once<(),FnOnce<()>,alloc::alloc::Global> () at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/alloc/src/boxed.rs:1546
#150 alloc::boxed::{{impl}}::call_once<(),alloc::boxed::Box<FnOnce<()>, alloc::alloc::Global>,alloc::alloc::Global> () at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/alloc/src/boxed.rs:1546
#151 std::sys::unix::thread::{{impl}}::new::thread_start () at library/std/src/sys/unix/thread.rs:71
#152 0x00007f83c347e259 in start_thread () from /usr/lib/libpthread.so.0
#153 0x00007f83c35945e3 in clone () from /usr/lib/libc.so.6
(gdb) bt -full
#0 0x00007f83c0cc3fec in std::panicking::rust_panic_with_hook () at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/fmt/mod.rs:314
No locals.
#1 0x00007f83c0cc3c07 in std::panicking::begin_panic_handler::{{closure}} () at library/std/src/panicking.rs:497
No locals.
#2 0x00007f83c0cc16dc in std::sys_common::backtrace::__rust_end_short_backtrace<closure-0,!> () at library/std/src/sys_common/backtrace.rs:141
No locals.
#3 0x00007f83c0cc3b69 in std::panicking::begin_panic_handler () at library/std/src/panicking.rs:493
No locals.
#4 0x00007f83bf38b1db in std::panicking::begin_panic_fmt () at library/std/src/panicking.rs:435
No locals.
#5 0x00007f83bf58c724 in engine::interning::{{impl}}::key_get::{{closure}} () at src/interning.rs:87
k = 0x7f839c0e1e58
#6 0x00007f83bf406e90 in core::option::Option<engine::core::Value>::unwrap_or_else<engine::core::Value,closure-0> (self=..., f=...) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:427
No locals.
#7 0x00007f83bf58c666 in engine::interning::Interns::key_get (self=0x7f83c1510eb0 <<engine::externs::INTERNS as core::ops::deref::Deref>::deref::__stability::LAZY+8>, k=0x7f839c0e1e58) at src/interning.rs:82
No locals.
#8 0x00007f83bf722aae in engine::externs::val_for (key=0x7f839c0e1e58) at src/externs/mod.rs:90
No locals.
#9 0x00007f83bf7283d5 in engine::externs::key_to_str (key=0x7f839c0e1e58) at src/externs/mod.rs:229
No locals.
#10 0x00007f83bf5bc86c in engine::core::{{impl}}::fmt (self=0x7f839c0e1e58, f=0x7f83bd73f4d0) at src/core.rs:227
No locals.
#11 0x00007f83bf5ae526 in core::fmt::{{impl}}::fmt<engine::core::Key> (self=0x7f83bd73f7a8, f=0x7f83bd73f4d0) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/fmt/mod.rs:2010
No locals.
#12 0x00007f83c0ce41bf in core::fmt::write () at library/core/src/fmt/mod.rs:1092
No locals.
#13 0x00007f83c0cc3db8 in core::fmt::Write::write_fmt<alloc::string::String> () at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/fmt/mod.rs:182
No locals.
#14 std::panicking::begin_panic_handler::{{impl}}::fill::{{closure}} () at library/std/src/panicking.rs:459
No locals.
#15 core::option::Option::get_or_insert_with<alloc::string::String,closure-0> () at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/option.rs:929
No locals.
#16 std::panicking::begin_panic_handler::PanicPayload::fill () at library/std/src/panicking.rs:457
No locals.
#17 std::panicking::begin_panic_handler::{{impl}}::get () at library/std/src/panicking.rs:475
No locals.
#18 0x00007f83c0cc405c in std::panicking::rust_panic_with_hook () at library/std/src/panicking.rs:590
No locals.
#19 0x00007f83c0cc3c07 in std::panicking::begin_panic_handler::{{closure}} () at library/std/src/panicking.rs:497
No locals.
#20 0x00007f83c0cc16dc in std::sys_common::backtrace::__rust_end_short_backtrace<closure-0,!> () at library/std/src/sys_common/backtrace.rs:141
No locals.
#21 0x00007f83c0cc3b69 in std::panicking::begin_panic_handler () at library/std/src/panicking.rs:493
No locals.
#22 0x00007f83bf38b1db in std::panicking::begin_panic_fmt () at library/std/src/panicking.rs:435
No locals.
#23 0x00007f83bf58c724 in engine::interning::{{impl}}::key_get::{{closure}} () at src/interning.rs:87
k = 0x7f839c0e1e58
#24 0x00007f83bf406e90 in core::option::Option<engine::core::Value>::unwrap_or_else<engine::core::Value,closure-0> (self=..., f=...) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:427
No locals.
#25 0x00007f83bf58c666 in engine::interning::Interns::key_get (self=0x7f83c1510eb0 <<engine::externs::INTERNS as core::ops::deref::Deref>::deref::__stability::LAZY+8>, k=0x7f839c0e1e58) at src/interning.rs:82
No locals.
#26 0x00007f83bf722aae in engine::externs::val_for (key=0x7f839c0e1e58) at src/externs/mod.rs:90
No locals.
#27 0x00007f83bf7283d5 in engine::externs::key_to_str (key=0x7f839c0e1e58) at src/externs/mod.rs:229
No locals.
#28 0x00007f83bf5bc86c in engine::core::{{impl}}::fmt (self=0x7f839c0e1e58, f=0x7f83bd73f940) at src/core.rs:227
No locals.
#29 0x00007f83bf5ae526 in core::fmt::{{impl}}::fmt<engine::core::Key> (self=0x7f83bd73fc18, f=0x7f83bd73f940) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/fmt/mod.rs:2010
No locals.
#30 0x00007f83c0ce41bf in core::fmt::write () at library/core/src/fmt/mod.rs:1092
No locals.
#31 0x00007f83c0cc3db8 in core::fmt::Write::write_fmt<alloc::string::String> () at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/fmt/mod.rs:182
No locals.
#32 std::panicking::begin_panic_handler::{{impl}}::fill::{{closure}} () at library/std/src/panicking.rs:459
No locals.
#33 core::option::Option::get_or_insert_with<alloc::string::String,closure-0> () at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/option.rs:929
No locals.
#34 std::panicking::begin_panic_handler::PanicPayload::fill () at library/std/src/panicking.rs:457
No locals.
#35 std::panicking::begin_panic_handler::{{impl}}::get () at library/std/src/panicking.rs:475
No locals.
#36 0x00007f83c0cc405c in std::panicking::rust_panic_with_hook () at library/std/src/panicking.rs:590
No locals.
#37 0x00007f83c0cc3c07 in std::panicking::begin_panic_handler::{{closure}} () at library/std/src/panicking.rs:497
No locals.
#38 0x00007f83c0cc16dc in std::sys_common::backtrace::__rust_end_short_backtrace<closure-0,!> () at library/std/src/sys_common/backtrace.rs:141
No locals.
#39 0x00007f83c0cc3b69 in std::panicking::begin_panic_handler () at library/std/src/panicking.rs:493
No locals.
#40 0x00007f83bf38b1db in std::panicking::begin_panic_fmt () at library/std/src/panicking.rs:435
No locals.
#41 0x00007f83bf58c724 in engine::interning::{{impl}}::key_get::{{closure}} () at src/interning.rs:87
k = 0x7f839c0e1e58
#42 0x00007f83bf406e90 in core::option::Option<engine::core::Value>::unwrap_or_else<engine::core::Value,closure-0> (self=..., f=...) at /home/jsirois/.rustup/toolchains/1.52.1-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:427
No locals.
(gdb) frame 41
#41 0x00007f83bf58c724 in engine::interning::{{impl}}::key_get::{{closure}} () at src/interning.rs:87
87 .unwrap_or_else(|| panic!("Previously memoized object disappeared for {:?}", k))
(gdb) print k
$1 = (*mut engine::core::Key) 0x7f839c0e1e58
(gdb) frame 23
#23 0x00007f83bf58c724 in engine::interning::{{impl}}::key_get::{{closure}} () at src/interning.rs:87
87 .unwrap_or_else(|| panic!("Previously memoized object disappeared for {:?}", k))
(gdb) print k
$2 = (*mut engine::core::Key) 0x7f839c0e1e58
The key to finding this was using a patched tokio 1.4.0: jsirois/tokio@d541685 to remove their panic catcher.
There was a problem hiding this comment.
...wow. Yea, of course.
Thanks a lot for digging in to find that!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Profiles of high-contention situations (including #11201) showed a non-trivial amount of time waiting on the
Internslock.Solution
In two commits, remove the need to separately intern python types, and simplify our GIL/
Internslocking strategy by implementing it internally to theInternsstruct.Result
Simpler code, and a slight (~5%) speedup for
./pants dependencies --transitive ::. #11201 remains open: likely we will want to make structural changes toTransitiveTargetsto avoid the redundant/contending computation.