Prevent compiler stack overflow for deeply recursive code #55617

oli-obk · 2018-11-02T15:28:22Z

I was unable to write a test that

runs in under 1s
overflows on my machine without this patch

The following reproduces the issue, but I don't think it's sensible to include a test that takes 30s to compile. We can now easily squash newly appearing overflows by the strategic insertion of calls to ensure_sufficient_stack.

// compile-pass

#![recursion_limit="1000000"]

macro_rules! chain {
    (EE $e:expr) => {$e.sin()};
    (RECURSE $i:ident $e:expr) => {chain!($i chain!($i chain!($i chain!($i $e))))};
    (Z $e:expr) => {chain!(RECURSE EE $e)};
    (Y $e:expr) => {chain!(RECURSE Z $e)};
    (X $e:expr) => {chain!(RECURSE Y $e)};
    (A $e:expr) => {chain!(RECURSE X $e)};
    (B $e:expr) => {chain!(RECURSE A $e)};
    (C $e:expr) => {chain!(RECURSE B $e)};
    // causes overflow on x86_64 linux
    // less than 1 second until overflow on test machine
    // after overflow has been fixed, takes 30s to compile :/
    (D $e:expr) => {chain!(RECURSE C $e)};
    (E $e:expr) => {chain!(RECURSE D $e)};
    (F $e:expr) => {chain!(RECURSE E $e)};
    // more than 10 seconds
    (G $e:expr) => {chain!(RECURSE F $e)};
    (H $e:expr) => {chain!(RECURSE G $e)};
    (I $e:expr) => {chain!(RECURSE H $e)};
    (J $e:expr) => {chain!(RECURSE I $e)};
    (K $e:expr) => {chain!(RECURSE J $e)};
    (L $e:expr) => {chain!(RECURSE L $e)};
}


fn main() {
    let x = chain!(D 42.0_f32);
}

fixes #55471
fixes #41884
fixes #40161
fixes #34844
fixes #32594

cc @alexcrichton @rust-lang/compiler

I looked at all code that checks the recursion limit and inserted stack growth calls where appropriate.

rust-highfive · 2018-11-02T15:28:34Z

r? @michaelwoerister

(rust_highfive has picked a reviewer for you, use r? to override)

src/librustc/middle/recursion_limit.rs

alexcrichton · 2018-11-02T15:45:23Z

src/librustc/Cargo.toml

@@ -34,6 +34,7 @@ byteorder = { version = "1.1", features = ["i128"]}
 chalk-engine = { version = "0.8.0", default-features=false }
 rustc_fs_util = { path = "../librustc_fs_util" }
 smallvec = { version = "0.6.5", features = ["union"] }
+stacker = "0.1.3"


I'm not sure stacker is necessarily rustc-ready right now, but that doesn't mean that it can't be! Some issues I can think of are:

It only has support for x86 platforms basically, and only Windows/Mac/Linux. It should be easy enough to "add support" for other platforms by basically doing nothing. Full support could be added over time as necessary

I don't think stacker does anything with guard pages, but ideally it'd also be sure to allocate guard pages for larger segments to protect agains accidental stack overflow

I'm not entirely sure how well panics and such work? It should be relatively easy to catch_unwind and resume_unwind though when necessary (just needs to be done)

These are all pretty minor, but I'd want to be sure to handle them before merging if possible!

src/librustc/ty/query/plumbing.rs

eddyb · 2018-11-03T20:14:05Z

src/librustc_driver/lib.rs

@@ -1460,6 +1460,8 @@ fn parse_crate_attrs<'a>(sess: &'a Session, input: &Input) -> PResult<'a, Vec<as
    }
 }

+const STACK_SIZE: usize = 4 * 1024 * 1024; // 4MB


Maybe make this dependent on whether stacker actually works?

How's that relevant? When stacker doesn't work, these values don't matter, because they don't do anything

I thought this was the default thread stack size, unrelated to stacker. My bad if that's not the case.

I'm gonna rename the constant to be clearer about this

Oh you're right, I was looking at the wrong value. But this change is just for crater as noted by @nagisa in #55617 (comment)

We'll bump it back after crater succeeds

michaelwoerister · 2018-11-05T12:11:58Z

Let's do a perf run.
@bors try

bors · 2018-11-05T12:12:05Z

🔒 Merge conflict

This pull request and the master branch diverged in a way that cannot be automatically merged. Please rebase on top of the latest master branch, and let the reviewer approve again.

How do I rebase?

Assuming self is your fork and upstream is this repository, you can resolve the conflict following these steps:

git checkout stacker (switch to your branch)
git fetch upstream master (retrieve the latest master)
git rebase upstream/master -p (rebase on top of it)
Follow the on-screen instruction to resolve conflicts (check git status if you got lost).
git push self stacker --force-with-lease (update this PR)

You may also read Git Rebasing to Resolve Conflicts by Drew Blessing for a short tutorial.

Please avoid the "Resolve conflicts" button on GitHub. It uses git merge instead of git rebase which makes the PR commit history more difficult to read.

Sometimes step 4 will complete without asking for resolution. This is usually due to difference between how Cargo.lock conflict is handled during merge and rebase. This is normal, and you should still perform step 5 to update this PR.

Error message

warning: Cannot merge binary files: src/Cargo.lock (HEAD vs. heads/homu-tmp)
Auto-merging src/librustc_typeck/check/mod.rs
Auto-merging src/librustc_traits/dropck_outlives.rs
Auto-merging src/librustc_mir/monomorphize/collector.rs
Auto-merging src/librustc_driver/lib.rs
Auto-merging src/librustc/traits/select.rs
Auto-merging src/librustc/traits/query/normalize.rs
Auto-merging src/librustc/traits/project.rs
Auto-merging src/librustc/hir/lowering.rs
Auto-merging src/Cargo.lock
CONFLICT (content): Merge conflict in src/Cargo.lock
Automatic merge failed; fix conflicts and then commit the result.

oli-obk · 2018-11-06T13:13:21Z

@bors try

bors · 2018-11-06T13:13:33Z

⌛ Trying commit 9f61d00 with merge 2b10b3d...

@alexcrichton

Prevent compiler stack overflow for deeply recursive code I was unable to write a test that 1. runs in under 1s 2. overflows on my machine without this patch The following reproduces the issue, but I don't think it's sensible to include a test that takes 30s to compile. We can now easily squash newly appearing overflows by the strategic insertion of calls to `ensure_sufficient_stack`. ```rust // compile-pass #![recursion_limit="1000000"] macro_rules! chain { (EE $e:expr) => {$e.sin()}; (RECURSE $i:ident $e:expr) => {chain!($i chain!($i chain!($i chain!($i $e))))}; (Z $e:expr) => {chain!(RECURSE EE $e)}; (Y $e:expr) => {chain!(RECURSE Z $e)}; (X $e:expr) => {chain!(RECURSE Y $e)}; (A $e:expr) => {chain!(RECURSE X $e)}; (B $e:expr) => {chain!(RECURSE A $e)}; (C $e:expr) => {chain!(RECURSE B $e)}; // causes overflow on x86_64 linux // less than 1 second until overflow on test machine // after overflow has been fixed, takes 30s to compile :/ (D $e:expr) => {chain!(RECURSE C $e)}; (E $e:expr) => {chain!(RECURSE D $e)}; (F $e:expr) => {chain!(RECURSE E $e)}; // more than 10 seconds (G $e:expr) => {chain!(RECURSE F $e)}; (H $e:expr) => {chain!(RECURSE G $e)}; (I $e:expr) => {chain!(RECURSE H $e)}; (J $e:expr) => {chain!(RECURSE I $e)}; (K $e:expr) => {chain!(RECURSE J $e)}; (L $e:expr) => {chain!(RECURSE L $e)}; } fn main() { let x = chain!(D 42.0_f32); } ``` fixes #55471 fixes #41884 fixes #40161 fixes #34844 fixes #32594 cc @alexcrichton @rust-lang/compiler I looked at all code that checks the recursion limit and inserted stack growth calls where appropriate.

bors · 2018-11-06T15:33:35Z

☀️ Test successful - status-travis
State: approved= try=True

oli-obk · 2018-11-06T15:47:42Z

@rust-timer build 2b10b3d

rust-timer · 2018-11-06T15:47:43Z

Success: Queued 2b10b3d with parent f90aab7, comparison URL.

rust-timer · 2018-11-06T18:04:30Z

Finished benchmarking try commit 2b10b3d

oli-obk · 2018-11-06T18:25:28Z

Improvements for ctfe stress tests (spurious?), regressions up to 3% for everything else except the clean-incremental part

michaelwoerister · 2018-11-07T09:51:53Z

Makes sense that the clean-incremental cases don't see a difference since they hardly execute any of the changed code. The other regressions are rather unfortunate though. Can we optimize this?

oli-obk · 2018-11-07T12:36:46Z

Well, this is an operation we now run on every single query (not every call, just every evaluation). There are loads of queries. It seems logical that this introduces some regression that we can't get rid of.

michaelwoerister · 2018-11-07T15:11:09Z

But

do we need to run the operation really for every query invocation?
can we make the operation less expensive, especially in the case where the stack doesn't have to be grown?

eddyb · 2018-11-07T15:58:03Z

I feel like the actual stack check shouldn't be noticeable compared to hashing and looking up a key in a hashmap. Maybe we're growing the stack more often than we need to?

nikic · 2018-11-07T17:15:02Z

Looking at the stacker implementation, a possible issue might be that we're sitting somewhere close to the stack limit and regularly go over it and below it again. This will allocate and deallocate a new stack every time. Maybe retaining the last allocation would help to reduce the performance impact?

nagisa · 2018-11-08T05:56:20Z

My initial proposal separately mentioned that we should only grow the stack and never bother redicing its size. I made that call with performance in mind. I didn't realise stacker was deallocating stack fragments, though now that I think about it, *that* is the obvious implementation for the stacker approach.

…

On Wed, Nov 7, 2018, 19:15 Nikita Popov ***@***.*** wrote: Looking at the stacker implementation, a possible issue might be that we're sitting somewhere close to the stack limit and regularly go over it and below it again. This will allocate and deallocate a new stack every time. Maybe retaining the last allocation would help to reduce the performance impact? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#55617 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AApc0ssfA1jZIb2KMI_ZkxHe12s2Eb0yks5usxU-gaJpZM4YL3aW> .

Dylan-DPC-zz · 2020-05-06T11:24:07Z

@bors retry (yield)

bors · 2020-05-06T14:38:59Z

⌛ Testing commit 935a05f with merge 1e4ad8ae1ae7c74ced296fd2e6380a81a5f6357c...

Dylan-DPC-zz · 2020-05-06T15:06:44Z

@bors retry yield

bors · 2020-05-06T15:55:26Z

⌛ Testing commit 935a05f with merge 698a0e16dbbb8fa45c5f0dd8c298ad8e98d14f80...

Dylan-DPC-zz · 2020-05-06T16:59:40Z

@bors retry yield

bors · 2020-05-06T20:33:48Z

⌛ Testing commit 935a05f with merge 18e1bdf136f4f0a269ac64e89c73020ad8b882a6...

Dylan-DPC-zz · 2020-05-06T20:38:51Z

@bors retry yield

bors · 2020-05-07T00:03:34Z

⌛ Testing commit 935a05f with merge 97f3eee...

bors · 2020-05-07T03:31:34Z

☀️ Test successful - checks-actions, checks-azure
Approved by: nagisa,oli-obk
Pushing 97f3eee to master...

Mark-Simulacrum · 2020-05-08T20:58:02Z

cc @XAMPPRocky I tagged this with relnotes-perf but it's not really perf so much as "hey this fixes a longstanding problem, would be cool to mention"

rustdoc's `main()` immediately spawns a thread, M, with a large stack (16MiB or 32MiB) on which it runs `main_args()`. `main_args()` does a small amount of options processing and then calls `setup_callbacks_and_run_in_default_thread_pool_with_globals()`, which spawns it own thread, and M is not used further. So, thread M seems unnecessary. However, it does serve a purpose: if the options processing in `main_args()` panics, that panic is caught when M is joined. So M can't simply be removed. However, `main_options()`, which is called by `main_args()`, has a `catch_fatal_errors()` call within it. We can move that call to `main()` and change it to the very similar `catch_with_exit_code()`. With that in place, M can be removed, and panics from options processing will still be caught appropriately. Even better, this makes rustdoc's `main()` match rustc's `main()`, which also uses `catch_with_exit_code()`. (Also note that the use of a 16MiB/32MiB stack was eliminated from rustc in rust-lang#55617.)

rust-highfive assigned michaelwoerister Nov 2, 2018

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Nov 2, 2018

This comment has been minimized.

Sign in to view

nagisa reviewed Nov 2, 2018

View reviewed changes

src/librustc/middle/recursion_limit.rs Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

alexcrichton reviewed Nov 2, 2018

View reviewed changes

oli-obk added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 2, 2018

oli-obk mentioned this pull request Nov 2, 2018

Support panicking and make unsupported platforms a nop rust-lang/stacker#5

Closed

eddyb reviewed Nov 3, 2018

View reviewed changes

src/librustc/ty/query/plumbing.rs Outdated Show resolved Hide resolved

eddyb reviewed Nov 3, 2018

View reviewed changes

oli-obk force-pushed the stacker branch from 7ecd20f to 9f61d00 Compare November 6, 2018 13:13

pnkfelix mentioned this pull request Nov 8, 2018

regression: stack overflow on macosx with xcode 6.4 #55471

Closed

bors mentioned this pull request May 7, 2020

Update cargo #71925

Merged

bors added the merged-by-bors This PR was explicitly merged by bors. label May 7, 2020

bors mentioned this pull request May 7, 2020

Implement new asm! syntax from RFC 2850 #69171

Merged

bors merged commit 97f3eee into rust-lang:master May 7, 2020

oli-obk deleted the stacker branch May 7, 2020 10:51

Mark-Simulacrum added the relnotes-perf Performance improvements that should be mentioned in the release notes. label May 8, 2020

This was referenced May 10, 2020

Encounter STATUS_STACK_BUFFER_OVERRUN when compiling diesel on windows 10 #72084

Closed

update stacker to 0.1.9 to unbreak build on OpenBSD #72079

Merged

LifeIsStrange mentioned this pull request May 18, 2020

stdlib: DeepRecursiveFunction JetBrains/kotlin#3398

Merged

ehuss mentioned this pull request May 29, 2020

submodules: Update RLS and Rustfmt #72423

Closed

Mark-Simulacrum mentioned this pull request Jun 2, 2020

thread 'rustc' has overflowed its stack #72933

Closed

nnethercote mentioned this pull request Aug 4, 2020

Clean up rustdoc's main() #75124

Merged

KevinCathcart mentioned this pull request Aug 13, 2022

There may be a problem with modestly deep recursion swc-project/swc#5470

Closed

Kobzol mentioned this pull request Oct 1, 2023

Fix default stack size in stdlib docs #116322

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Prevent compiler stack overflow for deeply recursive code #55617

Prevent compiler stack overflow for deeply recursive code #55617

oli-obk commented Nov 2, 2018 •

edited

Loading

rust-highfive commented Nov 2, 2018

This comment has been minimized.

This comment has been minimized.

alexcrichton Nov 2, 2018

eddyb Nov 3, 2018

oli-obk Nov 3, 2018

eddyb Nov 3, 2018

oli-obk Nov 4, 2018

oli-obk Nov 4, 2018

michaelwoerister commented Nov 5, 2018

bors commented Nov 5, 2018

oli-obk commented Nov 6, 2018

bors commented Nov 6, 2018

bors commented Nov 6, 2018

oli-obk commented Nov 6, 2018

rust-timer commented Nov 6, 2018

rust-timer commented Nov 6, 2018

oli-obk commented Nov 6, 2018

michaelwoerister commented Nov 7, 2018

oli-obk commented Nov 7, 2018

michaelwoerister commented Nov 7, 2018

eddyb commented Nov 7, 2018

nikic commented Nov 7, 2018

nagisa commented Nov 8, 2018 via email

Dylan-DPC-zz commented May 6, 2020

bors commented May 6, 2020

Dylan-DPC-zz commented May 6, 2020

bors commented May 6, 2020

Dylan-DPC-zz commented May 6, 2020

bors commented May 6, 2020

Dylan-DPC-zz commented May 6, 2020

bors commented May 7, 2020

bors commented May 7, 2020

Mark-Simulacrum commented May 8, 2020

Prevent compiler stack overflow for deeply recursive code #55617

Prevent compiler stack overflow for deeply recursive code #55617

Conversation

oli-obk commented Nov 2, 2018 • edited Loading

rust-highfive commented Nov 2, 2018

This comment has been minimized.

This comment has been minimized.

alexcrichton Nov 2, 2018

Choose a reason for hiding this comment

eddyb Nov 3, 2018

Choose a reason for hiding this comment

oli-obk Nov 3, 2018

Choose a reason for hiding this comment

eddyb Nov 3, 2018

Choose a reason for hiding this comment

oli-obk Nov 4, 2018

Choose a reason for hiding this comment

oli-obk Nov 4, 2018

Choose a reason for hiding this comment

michaelwoerister commented Nov 5, 2018

bors commented Nov 5, 2018

oli-obk commented Nov 6, 2018

bors commented Nov 6, 2018

bors commented Nov 6, 2018

oli-obk commented Nov 6, 2018

rust-timer commented Nov 6, 2018

rust-timer commented Nov 6, 2018

oli-obk commented Nov 6, 2018

michaelwoerister commented Nov 7, 2018

oli-obk commented Nov 7, 2018

michaelwoerister commented Nov 7, 2018

eddyb commented Nov 7, 2018

nikic commented Nov 7, 2018

nagisa commented Nov 8, 2018 via email

Dylan-DPC-zz commented May 6, 2020

bors commented May 6, 2020

Dylan-DPC-zz commented May 6, 2020

bors commented May 6, 2020

Dylan-DPC-zz commented May 6, 2020

bors commented May 6, 2020

Dylan-DPC-zz commented May 6, 2020

bors commented May 7, 2020

bors commented May 7, 2020

Mark-Simulacrum commented May 8, 2020

oli-obk commented Nov 2, 2018 •

edited

Loading