Refactor vitess/lite Dockerfiles. #5592
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Anthony Yeh <enisoc@planetscale.com>
enisoc
requested review from
derekperkins,
dkhenry and
sougou
as code owners
ajeetj
added a commit
to planetscale/vitess
that referenced
this pull request
Dec 20, 2019
* Rebase test_master with latest Cluster code (#22) * pick latest fix from cluster_for_test Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Tabletmanager2 test cases in GO using cluster (#23) * tabletmanager lock unlock table test case Signed-off-by: Ajeet jain <ajeet@planetscale.com> * Vtctld Test cases in Go using cluster (#25) * converted vtctld_test.py to go Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Converted schema.py testcase (#26) * migrated one of testcase from schema.py to schema_test.go Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Tablet Manager test cases in Go using cluster (#27) * converted tabletmanager test cases to go Signed-off-by: Ajeet jain <ajeet@planetscale.com> * Converted keyspace test to go (#28) * ported testcase of keyspace_test.py Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Converted sharded test from py to go (#29) * Converted sharded test from py to go Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * initial commit for sharding test Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * added reconciliation assert Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * added insert and stream replication Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * added rest assertions including migrateservedtypes Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * completed remaining assertions and fixed non-workings Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * converted to v3 way of resharding Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * restructured the testcases Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * added testcase with bytes as sharding key type Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Rebase test_master with latest Cluster code (#22) * pick latest fix from cluster_for_test Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Tabletmanager2 test cases in GO using cluster (#23) * tabletmanager lock unlock table test case Signed-off-by: Ajeet jain <ajeet@planetscale.com> * Vtctld Test cases in Go using cluster (#25) * converted vtctld_test.py to go Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Converted schema.py testcase (#26) * migrated one of testcase from schema.py to schema_test.go Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Tablet Manager test cases in Go using cluster (#27) * converted tabletmanager test cases to go Signed-off-by: Ajeet jain <ajeet@planetscale.com> * Converted keyspace test to go (#28) * ported testcase of keyspace_test.py Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Converted sharded test from py to go (#29) * Converted sharded test from py to go Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Added testcase for mysqlctl process Signed-off-by: saurabh <saurabh.408@gmail.com> * removed unused variables Signed-off-by: saurabh <saurabh.408@gmail.com> * removing comment Signed-off-by: saurabh <saurabh.408@gmail.com> * addressed review comments Signed-off-by: saurabh <saurabh.408@gmail.com> * removed cleandir flag from vttablet teardown Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * added v3 sharding test Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * vtgate buffer testcase migrated in go Signed-off-by: saurabh <saurabh.408@gmail.com> * inital code for resharding Signed-off-by: Ajeet jain <ajeet@planetscale.com> * completed the cluster code Signed-off-by: Ajeet jain <ajeet@planetscale.com> * more progress Signed-off-by: Ajeet jain <ajeet@planetscale.com> * pull graceful shutdown code Signed-off-by: Ajeet jain <ajeet@planetscale.com> * added fe more cases in resharding_test.go Signed-off-by: Ajeet jain <ajeet@planetscale.com> * fixed merge issues Signed-off-by: Ajeet jain <ajeet@planetscale.com> * running mysql in parallel mode and fe more assertions Signed-off-by: Ajeet jain <ajeet@planetscale.com> * used rbr mysql mode Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * almost finished with reshading v2 version Signed-off-by: Ajeet jain <ajeet@planetscale.com> * Rebase test_master with latest Cluster code (#22) * pick latest fix from cluster_for_test Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Tabletmanager2 test cases in GO using cluster (#23) * tabletmanager lock unlock table test case Signed-off-by: Ajeet jain <ajeet@planetscale.com> * Vtctld Test cases in Go using cluster (#25) * converted vtctld_test.py to go Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Converted schema.py testcase (#26) * migrated one of testcase from schema.py to schema_test.go Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Tablet Manager test cases in Go using cluster (#27) * converted tabletmanager test cases to go Signed-off-by: Ajeet jain <ajeet@planetscale.com> * Converted keyspace test to go (#28) * ported testcase of keyspace_test.py Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Converted sharded test from py to go (#29) * Converted sharded test from py to go Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Added testcase for mysqlctl process Signed-off-by: saurabh <saurabh.408@gmail.com> * removed unused variables Signed-off-by: saurabh <saurabh.408@gmail.com> * removing comment Signed-off-by: saurabh <saurabh.408@gmail.com> * addressed review comments Signed-off-by: saurabh <saurabh.408@gmail.com> * removed cleandir flag from vttablet teardown Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * vtgate buffer testcase migrated in go Signed-off-by: saurabh <saurabh.408@gmail.com> * changed code to use v3 way of sharding Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * converted to v3 way Signed-off-by: Ajeet jain <ajeet@planetscale.com> * added multi split diff and multi sharding Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * converted to v3 way Signed-off-by: Ajeet jain <ajeet@planetscale.com> * TabletManager remaining test cases TabletManager remaining test cases in GO Signed-off-by: Ajeet jain <ajeet@planetscale.com> * moved sql start to non-blocking mode (#35) * moved sql start to non-blocking mode Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * added testcase for vtgate vars Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * implement pr suggestion Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * trying to work with throttler Signed-off-by: Ajeet jain <ajeet@planetscale.com> * use ks1 mysql ports for other tablets in ks2 Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * converted on int keys Signed-off-by: Ajeet jain <ajeet@planetscale.com> * Rebase test_master with latest Cluster code (#22) * pick latest fix from cluster_for_test Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Tabletmanager2 test cases in GO using cluster (#23) * tabletmanager lock unlock table test case Signed-off-by: Ajeet jain <ajeet@planetscale.com> Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Vtctld Test cases in Go using cluster (#25) * converted vtctld_test.py to go Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Converted schema.py testcase (#26) * migrated one of testcase from schema.py to schema_test.go Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Tablet Manager test cases in Go using cluster (#27) * converted tabletmanager test cases to go Signed-off-by: Ajeet jain <ajeet@planetscale.com> Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Converted keyspace test to go (#28) * ported testcase of keyspace_test.py Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Converted sharded test from py to go (#29) * Converted sharded test from py to go Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Added testcase for mysqlctl process Signed-off-by: saurabh <saurabh.408@gmail.com> Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * removed unused variables Signed-off-by: saurabh <saurabh.408@gmail.com> Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * removing comment Signed-off-by: saurabh <saurabh.408@gmail.com> Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * addressed review comments Signed-off-by: saurabh <saurabh.408@gmail.com> Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * removed cleandir flag from vttablet teardown Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * vtgate buffer testcase migrated in go Signed-off-by: saurabh <saurabh.408@gmail.com> Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * TabletManager remaining test cases TabletManager remaining test cases in GO Signed-off-by: Ajeet jain <ajeet@planetscale.com> Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * moved sql start to non-blocking mode (#35) * moved sql start to non-blocking mode Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * added testcase for vtgate vars Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * redirect process errors to log files (#38) Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * debug the test error in ci Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * fix initial sharding multi Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * fixed protected mysqlctl teardown Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * removed sleep time Signed-off-by: Ajeet jain <ajeet@planetscale.com> * refactor mysqlctl start and stop Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Fixed multisplitdiff command Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * added log statement for error condition Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Ignoring Python tests which are moved to GO (#40) Signed-off-by: Ajeet jain <ajeet@planetscale.com> * readme for go endtoend test cases Signed-off-by: Ajeet jain <ajeet@planetscale.com> * Update README.md * Fixed byte column test cases Signed-off-by: Ajeet jain <ajeet@planetscale.com> * lint fixes Signed-off-by: Ajeet jain <ajeet@planetscale.com> * updated comments Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * restructure of launch cluster Signed-off-by: Ajeet jain <ajeet@planetscale.com> * fix tear down code Signed-off-by: Ajeet jain <ajeet@planetscale.com> * Refactor code Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * fix the throttler test cases Signed-off-by: Ajeet jain <ajeet@planetscale.com> * fix the static checks Signed-off-by: Ajeet jain <ajeet@planetscale.com> * disable running python test cases in travis Signed-off-by: Ajeet jain <ajeet@planetscale.com> * fixed static checks Signed-off-by: Ajeet jain <ajeet@planetscale.com> * removed unrequired line to set RBR mode Signed-off-by: Ajeet jain <ajeet@planetscale.com> * Add matrix build for mysql Signed-off-by: Morgan Tocker <tocker@gmail.com> * Add additional flavors, move flaky test Signed-off-by: Morgan Tocker <tocker@gmail.com> * vindex: add required NeedVCursor function There's currently no way to know if a vindex requires the vcursor. We need this knowledge to prevent vreplication from choosing a vindex that requires a VCursor. This change requires all vindexes to report if they need the vcursor or not. Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com> * vrepl: choose best vindex instead of primary Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com> * few of the reparent test cases Signed-off-by: Ajeet jain <ajeet@planetscale.com> * vttablet-up.sh : Adds detection of mysqld launch failure. Given failure, tests for further for presence of apparmor. Exits with error and recommendation to disable. Signed-off-by: ChrisR <radsurfer@gmail.com> * completed all the reparent tests Signed-off-by: Ajeet jain <ajeet@planetscale.com> * Remove MariaDB 10.1 Re-enable unit race Change CI git checkout to be v2.0 Disable test caching Signed-off-by: Morgan Tocker <tocker@gmail.com> * Revert some of the MariaDB changes, fix unit_race Signed-off-by: Morgan Tocker <tocker@gmail.com> * Add debugging for macOS issue Signed-off-by: Morgan Tocker <tocker@gmail.com> * Add additional flaky test Signed-off-by: Morgan Tocker <tocker@gmail.com> * Add sleep for macOS Signed-off-by: Morgan Tocker <tocker@gmail.com> * Disable macOS local example for now (flaky) Signed-off-by: Morgan Tocker <tocker@gmail.com> * Disable unit_race again Signed-off-by: Morgan Tocker <tocker@gmail.com> * updated test cases for reparent Signed-off-by: Ajeet jain <ajeet@planetscale.com> * Resharding endtoend testcases in Go migrated from Python (#44) Signed-off-by: Ajeet jain <ajeet@planetscale.com> * fix vtgate schema testcases Signed-off-by: Ajeet jain <ajeet@planetscale.com> * Convert binlog.py to go version (#41) Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * fixed range based reparenting Signed-off-by: Ajeet jain <ajeet@planetscale.com> * removed unrequired comments Signed-off-by: Ajeet jain <ajeet@planetscale.com> * Modified apparmor detection to passive, added additional investigation info. Signed-off-by: ChrisR <radsurfer@gmail.com> * make-release-packages.sh: Disable shellcheck 1091 and Add README pointing to local install instructions Signed-off-by: Gary Edgar <gary@planetscale.com> * review comments addressed Signed-off-by: Ajeet jain <ajeet@planetscale.com> * Splitting github endtoend action into 2 files Signed-off-by: Ajeet jain <ajeet@planetscale.com> * vindex: NeedVCursor->NeedsVCursor Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com> * Set correct tablet type to fix failing test Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Fixing initial sharding multi Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * fix the assertions Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * fixed master-spare test case Signed-off-by: Ajeet jain <ajeet@planetscale.com> * Testing dockerized test in github workflow Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Vertical split testcase migrated in go (#46) Vertical split endtoend testcases in Go migrated from Python Signed-off-by: saurabh <saurabh.408@gmail.com> * Adjusted shard number for testcase Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Reverting changes for config.json Signed-off-by: Arindam Nayak <arindam.nayak@outlook.com> * Refactor vitess/lite Dockerfiles. (vitessio#5592) Signed-off-by: Anthony Yeh <enisoc@planetscale.com> Co-authored-by: Arindam Nayak <arindamnayak@users.noreply.github.com> Co-authored-by: saurabh408 <saurabh.408@gmail.com> Co-authored-by: Morgan Tocker <tocker@gmail.com> Co-authored-by: Sugu Sougoumarane <ssougou@gmail.com> Co-authored-by: ChrisR <radsurfer@gmail.com> Co-authored-by: Gary Edgar <gary@planetscale.com> Co-authored-by: Anthony Yeh <enisoc@planetscale.com> Co-authored-by: Deepthi Sigireddi <deepthi.sigireddi@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We no longer build off of vitess/base, which fixes the problem of automated vitess/lite builds sometimes having the incorrect binary versions. Note that we still have that problem for the docker/k8s family of images.
I also tried to pull common logic out of the various Dockerfiles into scripts and make rules to make it easier to keep them in sync with each other going forward.
Lastly, this adds XtraBackup to all flavors that support it.
cc @aquarapid