-
Notifications
You must be signed in to change notification settings - Fork 347
Add integration tests that run against a real Redshift cluster #41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Current coverage is
|
3bc0fca to
4b203c7
Compare
|
Alright, the basic test is now passing (fixed the non-deterministic failure issue), so I'm now going to focus on enabling more tests. |
.travis.yml
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if I should purposely give these different names than the ones expected by the Amazon SDKs; this might be necessary in order to be able to test credential mechanisms (e.g. for writing a regression test for #32)
|
All tests are now enabled, but some seem to fail due to problems in the tests themselves (due to not using a HiveContext) or due to bugs. |
|
This is now ready for review. Let's not commit until #42 goes in and until I have time for one final pass. This isn't all of the tests that I want to add but it's a reasonable first cut; I'll add more tests and fix more potential bugs in smaller followups. |
.travis.yml
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: is there a reason to not test style and other fast stuff first?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wanted the tests to still be able to run even if style checks failed. I figured this wasn't a huge deal here compared to what we do in Spark because the test time is really fast.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: newline
|
This is awesome! LGTM. |
…ceed" This reverts commit 9fdd5b3.
|
This was rebased on top of #32, but I'm going to try to back out of that rebase so that we can write a proper regression test for that issue. |
|
Alright, I have now backed out all non-test-related changes and have addressed the review comments, so I'm going to merge this now and will continue to iterate in followups. |
This PR adds end-to-end integration tests that run against a real Redshift cluster.
I moved the build definition from
.sbtto.scalaformat in order to use SBT's standard IntegrationTests configuration.I configured the Travis build to store Redshift credentials using encrypted environment variables. To minimize security risks, Travis does not allow pull requests from third-party forks to run tests using these credentials.