Stack Monitoring

[zephyrbot]

Reported by Anas Nashif:

On some system without hardware memory protection stack monitoring provides an alternative (but less secure) solution where some protection is deemed better then none.

The operating system shall be able to monitor stack usage and check for a stack overflow on a per executable object basis (Thread/ISR).

(Imported from Jira ZEP-1621)

[zephyrbot]

by Andy Ross:

Seems straightforward enough as a debug feature. Put a canary magic number at the bottom of the stack, flag a "check this thread" bit per-thread (somewhere safe and not at e.g. the bottom of the stack), and just loop through threads and check them at the end of each ISR (or maybe only some ISRs).

There's a non-trivial interrupt performance penalty, though. I don't know if any ideas for optimizing this have been discussed elsewhere...

[zephyrbot]

by Andrew Boie:

I started to look at this independently.

What I did was to stick a sentinel value at the very end of struct k_thread, since this would be the first value overwritten upon busting the stack.

I agree you could check upon every interrupt to see that the value is untouched at a small performance penalty. I think this would be useful.

I started to look into whether we could set an x86 memory watch point at runtime, such that if the sentinel ever gets touched we would immediately generate an exception, but could not get it to work in the short amount of time I spent looking at it:

https://gerrit.zephyrproject.org/r/10376

[zephyrbot]

by Andy Ross:

This one might want to be revisited in light of the recent MMU/MPU interest. It may be that our decision should be "do stack protection instead for everyone who wants this, don't bother with overflow detection".

That said, the simple canary idea (magic number at the bottom of a stack that gets checked from ISR bottoms) seems easy to me, and reasonably portable and supportable even if we do proper stack protection (or Andrew's idea about x86 debug registers) elsewhere.

[zephyrbot]

by Sharron LIU:

I label "Testable" for stack overflow detection. Need more test cases adding to "zephyr/tests/kernel/stackprot", since the app now skip checking to the "bottom" of the stack of a thread.
Regarding MPU/MMU, I noticed the related stories still in "New" state. Need re-visit testability for this. Thanks.

[zephyrbot]

by Sharron LIU:

Andrew Boie , https://gerrit.zephyrproject.org/r/#/c/13266
Could you review the test cases for stack overflow detection? It was blind coded, since this story not yet merged. Thanks.

[zephyrbot]

by Andrew Boie:

Sharron LIU sure I can use them to test the solution I am working on.

[zephyrbot]

by Andrew Boie:

This is now implemented and working.

Dependency is #130 which moves the struct k_thread out of the stack. I decided it was better to do this with a clean slate.

On all arches except ARC, I implement CONFIG_STACK_SENTINEL which puts a 4-byte sentinel value at the lowest stack address. It's presence is checked in 4 spots:

1. Upon thread self-termination (returning from entry function)
2. After servicing an interrupt that didn't require a context switch
3. Any context switch
4. Anytime a thread calls k_yield() without context switching

On ARC, instead we just use CONFIG_ARC_STACK_CHECKING.

I wrote a test case to exercise all of this.
Code currently lives in this branch:
https://github.com/andrewboie/zephyr/tree/stack-overflow

I'll do a PR once the k_thread changes go in.

[zephyrbot]

by Andrew Boie:

#164

[zephyrbot]

by Sharron LIU:

@andrew the test cases look fine, and I don't think I need submit my original one.

Just asking, is ISR stack monitoring supported?

[zephyrbot]

by Andrew Boie:

I didn't implement interrupt stack monitoring, no.
It wouldn't be terribly hard to add however.

[zephyrbot]

by Sharron LIU:

Verify stack monitoring with "tests/kernel/fatal", passed
@x86 (quark x86 core)
@arc (quark arc core)
@arm (qemu)
@Nios2 (qemu)
@riscv32 (qemu)

@xtensa build failed.

Closing this. Thanks.

[zephyrbot]

Related to ZEP-965