Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sanitycheck: CONFIG_TEST_USERSPACE / userspace tag cleanup #16466

Merged
merged 1 commit into from
May 30, 2019

Conversation

marc-hb
Copy link
Collaborator

@marc-hb marc-hb commented May 29, 2019

  • Delete CONFIG_TEST_USERSPACE=n no-ops because it's the default
    since commit 7b1ee5c

  • Some tests have a "userspace" tag pretending to TEST_USERSPACE but
    don't and vice versa: fix missing or spurious "userspace" tags in
    testcase.yaml files.

Tests have a spurious "userspace" tag when they PASS this command
cause none should pass:

./scripts/sanitycheck --tag=userspace -p qemu_x86
--extra-args=CONFIG_TEST_USERSPACE=n
--extra-args=CONFIG_USERSPACE=n | tee userspace.log

All tests run by this command must either fail to build or fail to run
with some userspace related error. Shortcut to look at all test
failures:

zephyr_failure_logs() {
awk '/see.*log/ {print

Extra close brace or missing open brace

$2}' "$
@"
}

Tests missing "userspace" tag FAIL to either build or to run with some
userspace related error when running this:

./scripts/sanitycheck --exclude=userspace -p qemu_x86
--extra-args=CONFIG_TEST_USERSPACE=n
--extra-args=CONFIG_USERSPACE=n | tee excludeuserspace.log

Note the detection methods above are not 100% perfect because some
flexible tests like tests/kernel/queue/src/main.c evade them with #ifdef
CONFIG_USERSPACE smarts. Considering they never break, it is purely the
test author's decision to include or not such flexible tests in the
"userspace" subset.

Signed-off-by: Marc Herbert marc.herbert@intel.com

- Delete CONFIG_TEST_USERSPACE=n no-ops because it's the default
since commit 7b1ee5c

- Some tests have a "userspace" tag pretending to TEST_USERSPACE but
don't and vice versa: fix missing or spurious "userspace" tags in
testcase.yaml files.

Tests have a _spurious_ "userspace" tag when they PASS this command
cause none should pass:

  ./scripts/sanitycheck --tag=userspace -p qemu_x86 \
      --extra-args=CONFIG_TEST_USERSPACE=n  \
      --extra-args=CONFIG_USERSPACE=n | tee userspace.log

All tests run by this command must either fail to build or fail to run
with some userspace related error. Shortcut to look at all test
failures:

 zephyr_failure_logs() {
     awk '/see.*log/ {print $2}' "$@"
 }

Tests _missing_ "userspace" tag FAIL to either build or to run with some
userspace related error when running this:

  ./scripts/sanitycheck --exclude=userspace -p qemu_x86 \
      --extra-args=CONFIG_TEST_USERSPACE=n  \
      --extra-args=CONFIG_USERSPACE=n | tee excludeuserspace.log

Note the detection methods above are not 100% perfect because some
flexible tests like tests/kernel/queue/src/main.c evade them with #ifdef
CONFIG_USERSPACE smarts. Considering they never break, it is purely the
test author's decision to include or not such flexible tests in the
"userspace" subset.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Copy link
Member

@jukkar jukkar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ioannisg ioannisg added the area: Sanitycheck Sanitycheck has been renamed to Twister label May 29, 2019
@nashif nashif added the Trivial Changes that can be reviewed by anyone, i.e. doc changes, minor build system tweaks, etc. label May 29, 2019
@nashif nashif merged commit 4afcc0f into zephyrproject-rtos:master May 30, 2019
@marc-hb marc-hb deleted the userspace-tag branch May 30, 2019 23:23
@henrikbrixandersen
Copy link
Member

@marc-hb This was already merged?

@marc-hb
Copy link
Collaborator Author

marc-hb commented Jun 3, 2019

I should have said "just FYI" but didn't want to spam the others.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: Sanitycheck Sanitycheck has been renamed to Twister Trivial Changes that can be reviewed by anyone, i.e. doc changes, minor build system tweaks, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants