testutil: Should ensure consul executable is correct version or use embedded code

[stevehorsfield]

Client: 0.7.0
Server: 0.7.0

Description of the Issue (and unexpected/desired result)

Products such as consul-template use testutil to spin up Consul servers. While developing a patch I was hitting unexpected issues in tests. The cause was a local consul binary being out of date. Note that consul-template includes the full consul repo as a vendor source.

Reproduction steps

• Install consul binary 0.6.3 into the path
• Download consul-template and issue make test

Various failures are experienced.

Upgrade consul binary to 0.7.0 and issues are resolved.

Suggestion

It would be nice if the test util did one of the following:

• validated the version of the Consul binary it uses to match the configured version
• validated the version of the Consul binary it uses to a supported version
• used the local Consul code to spin up the server
• At least indicated the mismatch

[slackpad]

Hi @stevehorsfield I've been bitten by this kind of thing, too. We could probably leverage go-getter to pull in a known revision that you point at, with maybe an optional way to say "build my local Consul".

[slackpad]

Actually thinking more on this, we should probably just ensure the version is >= some expected version - that'll hit both cases and should work well and be a simple change.

[stevehorsfield]

That sounds like a good solution. I agree there's a lot of permutations to consider here. Thanks for the quick response.

[magiconair]

IMO, the test code should not depend on an external binary. We should spin up the consul server from within. I have that on my plate.

[magiconair]

#3037 is work towards solving that properly.