"Upload Coverage Result To Coveralls" sometimes fails even when fail_on_error: false is set

[patbl]

I've had a bunch of build failures since starting to use the Coveralls orb. This is undesirable because it can block people from merging their changes for what we consider to be a non-essential part of the process. I thought that setting fail_on_error: false would prevent this, but it seems not to have.

Here's one of the errors I've encountered:

sha256sum: coveralls-linux.tar.gz: No such file or directory
coveralls-linux.tar.gz: FAILED open or read
sha256sum: WARNING: 1 listed file could not be read
tar (child): coveralls-linux.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
Parsing args
Dry run - 0
Reporting coverage
+ ./coveralls report --no-fail
/bin/bash: line 74: ./coveralls: No such file or directory

Exited with code exit status 127

It appears to be failing on this line:

orb/src/scripts/coveralls.sh

Line 75 in b48e556

./coveralls report $args

I think this could be solved by adding || true to the end of the command if fail_on_error is false.

I'm using coveralls/coveralls@2.2.1.

[afinetooth]

@patbl Thanks for reporting, and thanks so much for the potential fix.

We aim to get that out next week.

[afinetooth]

Hi @patbl. We just released v2.2.2 of the coveralls orb, which addresses this issue.

Thanks so much for your PR. We decided to make slightly more specific changes scoped to individual setup steps that will be parallel to code changes in our github-action. (I will leave a note on your PR.)

I tested fail_on_error in a number of local tests with success:

1. Failure of download
2. Failure to verify checksums
3. Failure to find binary named coveralls

Example:

# Failure to find binary named `coveralls`
# (Removed file after extraction)
root@8da46491130b:/app# export COVERALLS_FAIL_ON_ERROR="1"
root@8da46491130b:/app# ./test_coveralls_script.sh
coveralls-linux.tar.gz: OK
Coveralls binary not found after extraction.
root@8da46491130b:/app# echo $?
1

root@8da46491130b:/app# export COVERALLS_FAIL_ON_ERROR="false"
root@8da46491130b:/app# ./test_coveralls_script.sh
coveralls-linux.tar.gz: OK
Coveralls binary not found after extraction.
root@8da46491130b:/app# echo $?
0

We didn't have time right now, but we do intend to implement those tests (of "setup failures") as new steps in our CI/CD pipeline at some point.

I would say "Let me know if it doesn't work for you!" but I hope you won't encounter any more setup issues anytime soon.