opa test exit value is the same for skipped as it is for failures

[kirk-patton]

I created a unit test prefixed with "todo_" and then ran it with OPA test. As expected, the test was skipped. I checked the exit code and noted it was "2". When I remove the "todo_" and run the test, I get an expected failure. But, the exit value is also "2".

Expected Behavior

I expected some other exit value other that "2".

Actual Behavior

I expected some other value so that it would be possible to allow skipped tests in our CICD pipeline by checking the exit value.

Steps to Reproduce the Problem

• create a todo_sometest{}
• opa test -r todo_sometest
• echo $?
• rename todo_sometest{} to sometest{}
• opa test -r sometest
• echo $?

Additional Info

[tsandall]

Thanks for filing this Kirk. I believe the test subcommand exits with status code 2 because of this line.

Changing the exit code to another value if there are no failures and at least one skip sounds reasonable to me. This would encourage people to make all of their tests pass but allow them to handle the exceptional case where some tests have to be skipped.

How does that sound?

[kirk-patton]

Perfect 👍

[kirk-patton]

I submitted a small PR #3774