let: count+=: Syntax Error on OS X 10.11.2

[schmunk42]

I tried to run this test-suite locally on and got this error:

/usr/local/libexec/bats-exec-suite: Zeile 20: let: count+=: Syntax Fehler: Operator erwartet.  (Fehlerverursachendes Zeichen ist \"+=\").

which hardly translates to

/usr/local/libexec/bats-exec-suite: line 20: let: count+=: Syntax Error: Operator expected.  (Erroneous  character is \"+=\").

My bash version

$ bash --version
GNU bash, Version 4.3.42(1)-release (x86_64-apple-darwin15.2.0)

Installed latest from master.

[mislav]

That's weird; for me bash 4.3.42 works with bats just as well as bash 3.2.

Can you try to reproduce the error with:

bash -c 'echo $BASH; let count+=$(echo 1); echo $count'

[schmunk42]

No, this looks good, I think.

$ bash -c 'echo $BASH; let count+=$(echo 1); echo $count'
/opt/local/bin/bash
1

I have an ENV var SHELL=/bin/bash though, which points to

/bin/bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin15)
Copyright (C) 2007 Free Software Foundation, Inc.

[mislav]

Can you isolate the smallest test file that fails with the error you described originally? Ideally, bats mytest.bats should fail. Then paste us that *.bats file. Thanks!

[schmunk42]

I ran the built-in test suite:

Kraftbuch:bats tobias$ bats -v
Bats 0.4.0
Kraftbuch:bats tobias$ bats test/
bats-exec-suite  -x /Users/tobias/Webserver/github.com/sstephenson/bats/test/bats.bats /Users/tobias/Webserver/github.com/sstephenson/bats/test/suite.bats bats-format-tap-stream
 ✓ no arguments prints usage instructions
 ✓ -v and --version print version number
 ✓ -h and --help print help
 ✓ invalid filename prints an error
 ✗ empty test file runs zero tests
   (in test file test/bats.bats, line 33)
     `[ "$output" = "1..0" ]' failed
 ✗ one passing test
   (in test file test/bats.bats, line 39)
     `[ "${lines[0]}" = "1..1" ]' failed
 ✗ summary passing tests
   (in test file test/bats.bats, line 46)
     `[ "${lines[1]}" = "1 test, 0 failures" ]' failed
 ✗ summary passing and skipping tests
   (in test file test/bats.bats, line 52)
     `[ "${lines[2]}" = "2 tests, 0 failures, 1 skipped" ]' failed
 ✗ summary passing and failing tests
   (in test file test/bats.bats, line 58)
     `[ "${lines[4]}" = "2 tests, 1 failure" ]' failed
 ✗ summary passing, failing and skipping tests
   (in test file test/bats.bats, line 64)
     `[ "${lines[5]}" = "3 tests, 1 failure, 1 skipped" ]' failed
 ✗ one failing test
   (in test file test/bats.bats, line 70)
     `[ "${lines[0]}" = '1..1' ]' failed
 ✗ one failing and one passing test
   (in test file test/bats.bats, line 79)
     `[ "${lines[0]}" = '1..2' ]' failed
 ✗ failing test with significant status
   (in test file test/bats.bats, line 89)
     `[ "${lines[3]}" = "#   \`eval \"( exit \${STATUS:-1} )\"' failed with status 2" ]' failed
 ✗ failing helper function logs the test case's line number
   (in test file test/bats.bats, line 95)
     `[ "${lines[1]}" = 'not ok 1 failing helper function' ]' failed
 ✓ test environments are isolated
 ✓ setup is run once before each test
 ✓ teardown is run once after each test, even if it fails
 ✗ setup failure
   (in test file test/bats.bats, line 125)
     `[ "${lines[1]}" = 'not ok 1 truth' ]' failed
 ✗ passing test with teardown failure
   (in test file test/bats.bats, line 133)
     `[ "${lines[1]}" = 'not ok 1 truth' ]' failed
 ✗ failing test with teardown failure
   (in test file test/bats.bats, line 141)
     `[ "${lines[1]}" =  'not ok 1 truth' ]' failed
 ✗ teardown failure with significant status
   (in test file test/bats.bats, line 149)
     `[ "${lines[3]}" = "#   \`eval \"( exit \${STATUS:-1} )\"' failed with status 2" ]' failed
 ✗ failing test file outside of BATS_CWD
   (in test file test/bats.bats, line 156)
     `[ "${lines[2]}" = "# (in test file $FIXTURE_ROOT/failing.bats, line 4)" ]' failed
 ✓ load sources scripts relative to the current test file
 ✓ load aborts if the specified script does not exist
 ✓ load sources scripts by absolute path
 ✓ load aborts if the script, specified by an absolute path, does not exist
 ✗ output is discarded for passing tests and printed for failing tests
   (in test file test/bats.bats, line 182)
     `[ "${lines[6]}"  = '# failure stdout 1' ]' failed
 ✗ -c prints the number of tests
   (in test file test/bats.bats, line 190)
     `[ "$output" = "0" ]' failed
 ✗ dash-e is not mangled on beginning of line
   (in test file test/bats.bats, line 200)
     `[ "${lines[1]}" = "ok 1 dash-e on beginning of line" ]' failed
 ✓ dos line endings are stripped before testing
 ✗ test file without trailing newline
   (in test file test/bats.bats, line 211)
     `[ "${lines[1]}" = "ok 1 truth" ]' failed
 ✗ skipped tests
   (in test file test/bats.bats, line 217)
     `[ "${lines[1]}" = "ok 1 # skip a skipped test" ]' failed
 ✓ extended syntax
 ✓ pretty and tap formats
 ✗ pretty formatter bails on invalid tap
   (in test file test/bats.bats, line 245)
     `[ "${lines[0]}" = "This isn't TAP!" ]' failed
 ✗ single-line tests
   (in test file test/bats.bats, line 252)
     `[ "${lines[1]}" =  'ok 1 empty' ]' failed
 ✗ testing IFS not modified by run
   (in test file test/bats.bats, line 263)
     `[ "${lines[1]}" = "ok 1 loop_func" ]' failed
 ✗ running a suite with no test files
   (in test file test/suite.bats, line 9)
     `[ "$output" = "1..0" ]' failed
 ✗ running a suite with one test file
   (in test file test/suite.bats, line 15)
     `[ "${lines[0]}" = "1..1" ]' failed
 ✗ counting tests in a suite
   (in test file test/suite.bats, line 22)
     `[ "$output" -eq 1 ]' failed with status 2
   /var/folders/4_/lsj7k0n88xjfmj0059bg4_lh0000gn/T/bats.29892.src: Zeile 22: [: bats-exec-test -c  /Users/tobias/Webserver/github.com/sstephenson/bats/test/fixtures/suite/single/test.bats cat
   1: Ganzzahliger Ausdruck erwartet.
 ✗ aggregated output of multiple tests in a suite
   (in test file test/suite.bats, line 32)
     `[ "${lines[0]}" = "1..3" ]' failed
 ✗ a failing test in a suite results in an error exit code
   (in test file test/suite.bats, line 41)
     `[ "${lines[0]}" = "1..3" ]' failed
 ✗ running an ad-hoc suite by specifying multiple test files
   (in test file test/suite.bats, line 48)
     `[ "${lines[0]}" = "1..3" ]' failed
 ✗ extended syntax in suite
   (in test file test/suite.bats, line 57)
     `[ "${lines[0]}" = "1..3" ]' failed

45 tests, 30 failures

Looking not that good, I'll investigate... let me know if you have a suggestion or need additional system info.

[loren-osborn]

I've been chasing down some permutations of this. It turns out that Apple has stuck to a patched version of bash based on 3.2.51 as this is the last version before bash cut over to GPLv3. While this is unfortunate, I think this is adequate reason to support this ancient bash version. (Bash can, of course, be easily upgraded on a per system basis with tools like MacPorts and brew, requiring a current bash version restricts out of the box bash support.) It appears bash has made many changes since 3.2.51, especially including behaviors with "set -e" that bats relies on heavily.

What is the viability of back-porting bats to support bash 3.2.51 as well as the current bash releases. I am not a particularly proficient shell script developer, but am attempting some of this myself. (First order of business is to get bats unit tests passing.)

Comments/Suggestions welcome

Updated: as mentioned above, it looks like Mac OS X has moved bash up to 3.2.57 now, but I can't check until I boot my machine in the morning... (I don't expect there are major changes between bash 3.2.51 and 3.2.57.)

[cuonglm]

@loren-osborn In bash, even the newer version, there's no reason for using let or ((...)) for arithmetic expansion. The line:

let count+="$(bats-exec-test -c "$filename")"

can always be rewritten to:

: "$(( count+="$(bats-exec-test -c "$filename") ))"

or safer:

_count=$(bats-exec-test -c "$filename")
: "$(( count+=_count ))

See this http://unix.stackexchange.com/a/149832/38906 for more details.

[loren-osborn]

@Gnouc, great... That's the kind of help I find very useful. Currently I'm fighting with libexec/bats-exec-test:292

    trap "bats_debug_trap \"\$BASH_SOURCE\"" debug

and the backtrack capture mechanism. For some reason by the time I'm getting into bats_exit_trap, $BATS_ERROR_STACK_TRACE Is getting wiped out. Any ideas?

Updated: I don't know if there's any difference with the trap, but $BATS_ERROR_STACK_TRACE is getting reset somehow... Probably by going out of scope, as I don't see it getting set explicitly.

[cuonglm]

@loren-osborn: I'm not sure, maybe the condition https://github.com/sstephenson/bats/blob/master/libexec/bats-exec-test#L216 is false?

[loren-osborn]

@Gnouc, Thanks... sorry... I did some more troubleshooting, and my debug output was getting discarded, leading me to false conclusions... I did confirm that the latest version of Mac OS X El Capítain includes 3.2.57(1), but that is still ancient next to 4.3

I have been doing research into this. I think these links have been rather helpful:

• Bash Hackers Wiki: Bash Changes
• BashFAQ/105: Why doesn't set -e (or set -o errexit, or trap ERR) do what I expected?
• BASH Debugger
• Freenode IRC: #bash

The deeper I dive into this rabbit hole, the more I'm starting to feel that set -e is a convenience we can't afford if we want consistent behavior between bash versions... especially 3.x vs 4.x. I'm even less concerned about set -e in bats itself, but really in the code from the *.bats files. We can always fix the code in bats to correct version compatibility issues, but user code is out of our control. Without reimplementing the bash parser, I doubt we can make the behavior of user code version agnostic.

The only think I can think of, and this is rather drastic, encouraging people to move to *.bats2 files that are executed with set -e turned off. That way, we can still support set -e in legacy scripts that rely on it, but new code can be made more portable by avoiding it.

What do you think? (@sstephenson, this is a rather big proposed change. Can you share your opinion?)

[schmunk42]

I wasn't able to track this down further...

As a workaround I was using a bats Docker image, like so:

docker run -v ${PWD}/test:/test dduportal/bats /test

to run tests locally (on my Mac).

[MisanthropicBit]

I get the same error. I'm not really sure if this is relevant, and I'm no bash expert, but it might provide a hint towards the source of the problem.

In my tests I have a test_base.bash script that is included in all my tests using load. If this file's extension is changed to .sh instead of .bash, I get the error:

/usr/local/Cellar/bats/0.4.0/libexec/bats-exec-suite: line 20: let: count+=: syntax error: operand expected (error token is "+=")

The error disappears if I change the extension back to .bash again. My bash version is:

GNU bash, version 4.3.42(1)-release (x86_64-apple-darwin15.0.0)

and my bats version is:

Bats 0.4.0

[manikanta-kondeti]

Is this solved? I am stuck with this.

Getting the following error : /usr/local/Cellar/bats/0.4.0/libexec/bats-exec-suite: line 20: let: count+=: syntax error: operand expected (error token is "=")

I am on Mac OS X 10.11.6 and bats version 0.4.0

[mstade]

I'm seeing the same issue as @manikanta-kondeti – anyone figured out a workaround? This worked just recently, but I updated to macOS Sierra 10.12.1 since then, and I think I might have installed bash via Homebrew because bash --version reports GNU bash, version 4.4.0(1)-release (x86_64-apple-darwin16.0.0). Is there a min/max version of Bash that bats is compatible with?

[mstade]

I took a stab at debugging this, and it turns out the error I got, which is the same as @manikanta-kondeti and @schmunk42, was kind of a red herring. I was using this bit of offending code in a script used in my tests via the load mechanism:

project_dirname=$(readlink -m ./)

This fails, because readlink on this system doesn't support -m. It has supported this flag in the past, but I must have changed my system such that I now have a different readlink installed. What I think happened is that at some point I ran brew doctor and it told me to not shadow coreutils so these are now available as prefixed versions, like greadlink. Once I changed this the tests just started working again! The syntax error is because the call to bats-exec-test -c on line 20 returns, well, nothing. I guess count+="" isn't quite valid.

Thing is, I found out about this only due to @loren-osborn's comment about set -e. I removed set -e in bats-exec-suite and hey presto – the readlink error was clear as day! None of the tests actually worked though, and there was lots of strange output.

I ended up using this function to canonicalize the path, removing the readlink dependency altogether. My tests now run quiet nicely though, even with set -e. Not sure if this helps, but I hope so!

[ivan-at-wiris]

PROBLEM:

I was following the tutorial:
http://kitchen.ci/docs/getting-started/writing-test
and got stuck with this error:

   /tmp/verifier/vendor/bats/libexec/bats-exec-suite: line 20: let: +=: syntax error: operand expected (error token is "+=")
   !!!!!! Command [/tmp/verifier/vendor/bats/bin/bats /tmp/verifier/suites/bats] exit code was 1

only when using the vbox official centos 6 box:
https://atlas.hashicorp.com/centos/boxes/6/versions/1610.01/providers/virtualbox.box
but not when using the vbox bento/centos-6.8 box, if all the *.bats test files are correct.

If any *.bats test file has incorrect syntax, then this error happens in any box, even in bento box.

FIX:

nano /tmp/verifier/vendor/bats/libexec/bats-exec-suite

...
count=0
for filename in "$@"; do
  #let count += "$(bats-exec-test -c "$filename")"
  _count=$(bats-exec-test -c "$filename")
  (( count += "$_count" ))
done
...

[andkirby]

I got the same error.

$ bash --version
GNU bash, version 4.2.46(1)-release (x86_64-redhat-linux-gnu)

[andkirby]

Is this project died? This issue have been opened since the beginning of this year (2016).
More than two years ago were commits. There are lots of push requests.

[ztombol]

@andkirby The maintainer is looking for help to keep the project going. See #150.

[andkirby]

I got this. It's clear now. thx!

[relsqui]

Based on @Gnouc's comment above, I'm using this one-liner as a workaround in order to get my tests running in Travis:

sudo sed -i "20s/.*/  _count=\$(bats-exec-test -c \"\$filename\"); : \"\$(( count+=_count ))\"/" /usr/local/libexec/bats-exec-suite

Ain't pretty, but it works. This assumes you've installed bats using its install.sh to /usr/local; if you installed through a package manager, change the path at the end to /usr/lib/bats/bats-exec-suite.

NB: This will definitely stop working when a change is made to that file, for example, fixing this bug. Don't use it anywhere important.

Edit: Here's a more robust version.