Skip to content
This repository has been archived by the owner on Jun 20, 2024. It is now read-only.

Smoke tests run_all.sh is not robust #591

Closed
awh opened this issue Apr 22, 2015 · 1 comment · Fixed by #594
Closed

Smoke tests run_all.sh is not robust #591

awh opened this issue Apr 22, 2015 · 1 comment · Fixed by #594
Assignees
Labels
Milestone

Comments

@awh
Copy link
Contributor

awh commented Apr 22, 2015

  • In order to determine if all the tests have passed, one must read through the entire log output and check the individual test suites. The script should clearly indicate at the end whether all tests passed, and return an appropriate exit status for use by our CI system
  • Some of the shell commands used to check system state can cause the tests to end abruptly due to the use of set -e in config.sh. All failures should be reported through the unit test assert mechanism.
@awh awh added the chore label Apr 22, 2015
@awh awh self-assigned this Apr 22, 2015
@awh
Copy link
Contributor Author

awh commented Apr 22, 2015

There are two issues with the way we're using lehmannro/assert.sh:

  • end_suite calls assert_end via whiteley; this uses a sub-shell to capture output, so the overall status var set by assert_end is not visible to the outer script
  • Individual test scripts re-source config.sh, which re-sources assert.sh resetting the global test failure status var

@rade rade closed this as completed in #594 Apr 22, 2015
rade added a commit that referenced this issue Apr 22, 2015
Improved smoke test robustness

Fixes #591.
@rade rade modified the milestone: 0.11.0 May 12, 2015
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants