-
Notifications
You must be signed in to change notification settings - Fork 20
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BATs acceptance tests? #89
Comments
our "Makefile" tasks like |
Agree, more akin to the acceptance tests in Terraform provider will be ideal. i.e. the testing framework actually spawn a new test instance of the plugin, set it up with the Vault server, send the test steps (as cli text stream?), and check the results. Alas, what HashiCorp has provided leave us doing most of the heavy lifting right now. |
While I was looking for ways to test for the storage errors, I noticed that the Hashicorp plugins tend to use "BATs" for acceptance testing. That way they are able to spin up docker containers as needed (artifactory) and use the vault binary directly instead of interfacing at the "Backend" layer in code, which feels like more like unit testing (unit tests with a real artifactory server).
I am not saying we can't keep the go-based acceptance tests, but it feels kinda strange to have direct code access when doing acceptance tests. Acceptance Tests are supposed to be "black-box" or completely separate from the code. We can write them in golang, but they need to be in a separate directory (package).
Since the "normal" user interface is the vault binary, it feels like a shell based testing system would be easier.
Thoughts?
The text was updated successfully, but these errors were encountered: