Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add capability to create consistent transactions #4310

Closed
wants to merge 37 commits into from

Commits on Jan 10, 2019

  1. Add capability create consistent transactions

    To make this possible, two things are added:
     - The capability to lock all tables on a tablet, to momenterily stop updates
     - Once the database is locked, we can create multiple consistent snapshot
       transactions that all share the same view of the data
     - Adds the capability to have replication move forward to a specific point
       in the transaction log
    
    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 10, 2019
    Configuration menu
    Copy the full SHA
    b6987f7 View commit details
    Browse the repository at this point in the history
  2. Always restart the engine when changing server type

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 10, 2019
    Configuration menu
    Copy the full SHA
    d7cb371 View commit details
    Browse the repository at this point in the history
  3. Restore tests removed by mistake

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 10, 2019
    Configuration menu
    Copy the full SHA
    1468d91 View commit details
    Browse the repository at this point in the history
  4. Fix unit test

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 10, 2019
    Configuration menu
    Copy the full SHA
    bee8e6e View commit details
    Browse the repository at this point in the history
  5. Move logic of state transitions into the tx engine.

    Instead of the tabletserver keeping track of when it's safe
    to close transactions to change serving type, the tx engine
    now keeps track of it's current state and what is needed for
    safe state changes.
    
    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 10, 2019
    Configuration menu
    Copy the full SHA
    bdb292e View commit details
    Browse the repository at this point in the history
  6. more better tests

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 10, 2019
    Configuration menu
    Copy the full SHA
    9612e40 View commit details
    Browse the repository at this point in the history
  7. Make the tablet server stop keeping track of the tx engine state

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 10, 2019
    Configuration menu
    Copy the full SHA
    dfa9126 View commit details
    Browse the repository at this point in the history
  8. Minor work on tx_engine

     * Handle the case when ReadOnly is requested during transition.
    
     * Instead of allowing the outside deciding when to forcefully
       shut down transactions, hide this behaviour behind a method,
       so state can properly be updated.
    
    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 10, 2019
    Configuration menu
    Copy the full SHA
    03de7e4 View commit details
    Browse the repository at this point in the history
  9. Minor refactorings

     * Remove unused splitColumns argument
     * Remove unused error argument
    
    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 10, 2019
    Configuration menu
    Copy the full SHA
    549c161 View commit details
    Browse the repository at this point in the history
  10. tabletserver.go now uses the TxEngineStateController interface instea…

    …d of the TxEngine class
    
    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 10, 2019
    Configuration menu
    Copy the full SHA
    aaa4dde View commit details
    Browse the repository at this point in the history
  11. Moved the beginRequests waitGroup into the tx_engine

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 10, 2019
    Configuration menu
    Copy the full SHA
    4bc7937 View commit details
    Browse the repository at this point in the history
  12. Better naming and comments

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 10, 2019
    Configuration menu
    Copy the full SHA
    6393b07 View commit details
    Browse the repository at this point in the history
  13. Split test file in two to get around test harness problem

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 10, 2019
    Configuration menu
    Copy the full SHA
    a57c799 View commit details
    Browse the repository at this point in the history
  14. Make sure to stop race condititions

    The particular race condition is:
    
     * Thread 1: tsv.Begin is called, which calls startRequest through execRequest.
     * Thread 1: tsv.mu is released once startRequest returns.
     * Thread 2: A call to setServingType is received.
     * Thread 2: goes in: there are no begin requests in the wait group. So, it closes the TxEngine, or changes its state.
     * Thread 1: TxEngine.Begin executes on a TxEngine. If the state has changed so that a transaction should not be allowed, an error will not be returned.
    
    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 10, 2019
    Configuration menu
    Copy the full SHA
    505458f View commit details
    Browse the repository at this point in the history
  15. Removed boolean flag that was not needed any more

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 10, 2019
    Configuration menu
    Copy the full SHA
    018ca33 View commit details
    Browse the repository at this point in the history
  16. Make all state transitions blocking

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 10, 2019
    Configuration menu
    Copy the full SHA
    c3db459 View commit details
    Browse the repository at this point in the history

Commits on Jan 11, 2019

  1. Don't touch the internals unnecessarily

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    5785cdd View commit details
    Browse the repository at this point in the history
  2. Prepare tests for read only transactions

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    1ee033c View commit details
    Browse the repository at this point in the history
  3. Moving code around to make it easier to read

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    e0e52e5 View commit details
    Browse the repository at this point in the history
  4. Reject write transactions when in read only mode

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    fabc520 View commit details
    Browse the repository at this point in the history
  5. Make sure the schema engine is initialized

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    f99a185 View commit details
    Browse the repository at this point in the history
  6. Check for null

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    caaad26 View commit details
    Browse the repository at this point in the history

Commits on Jan 14, 2019

  1. Merge remote-tracking branch 'upstream/master' into lock-tables

    * upstream/master:
      Skip ACL checks for dual pseudotable
      Makes sure the schema engine has started up before getting debug information.
      Support tables that have columns named desc
      exec_cases: add passing messages OnDup test
      exec_cases: add failing tests for msgs on dupe
      planbuilder: support ‘on duplicate key’ for msgs
    systay committed Jan 14, 2019
    Configuration menu
    Copy the full SHA
    1f5524d View commit details
    Browse the repository at this point in the history
  2. Revert "Split test file in two to get around test harness problem"

    This reverts commit a57c799.
    
    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 14, 2019
    Configuration menu
    Copy the full SHA
    611ec92 View commit details
    Browse the repository at this point in the history
  3. Merge remote-tracking branch 'upstream/master' into lock-tables

    * upstream/master:
      helm: release 1.0.5
      vreplication: address review comments
      tlstest_test: Go 1.12 / TLS 1.3 fix
      vreplication: vstream final tweaks
      vreplication: tests are done
      vreplication: handle singleton and mariadb issues
      vreplication: moved tests into vstreamer dir
      vreplication: more vstreamer tests
      vreplication: MariaDB test tweaks and more tests
      vreplication: tests: statements and DDL add column
      vreplication: make a singleton srvtopo.Server
      vreplication: got some basic tests working
      vreplication: tabletserver inits and other tweaks
      vreplication: vstreamer, plan and filtering
      vreplication: streamer planbuilder
      vreplication: vstreamer Engine
    
    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 14, 2019
    Configuration menu
    Copy the full SHA
    37c25bb View commit details
    Browse the repository at this point in the history
  4. Comments

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 14, 2019
    Configuration menu
    Copy the full SHA
    830917d View commit details
    Browse the repository at this point in the history

Commits on Jan 15, 2019

  1. Return tx_engine to the old behaviours of rolling back transactions

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 15, 2019
    Configuration menu
    Copy the full SHA
    05826a1 View commit details
    Browse the repository at this point in the history
  2. Minor refactoring

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 15, 2019
    Configuration menu
    Copy the full SHA
    3966d01 View commit details
    Browse the repository at this point in the history
  3. Merge remote-tracking branch 'upstream/master' into lock-tables

    * upstream/master:
      vtctl: run BackupShard on replica, rdonly or spare
      Expose `glog.MaxSize` as `-log_rotate_max_size` flag
      inline ok check; while-range instead of bare while
      test the signal handler
      reload db-credentials-file upon SIGHUP
      change ClusterAlias in _vt.local_metadata to be keyspace/shard instead of keyspace.shard
    
    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 15, 2019
    Configuration menu
    Copy the full SHA
    6626e4d View commit details
    Browse the repository at this point in the history
  4. Continue restoring the old behaviour back

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 15, 2019
    Configuration menu
    Copy the full SHA
    20ba0f5 View commit details
    Browse the repository at this point in the history

Commits on Jan 17, 2019

  1. temp fix for tx pool hang

    Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
    sougou committed Jan 17, 2019
    Configuration menu
    Copy the full SHA
    4fafca8 View commit details
    Browse the repository at this point in the history
  2. Make sure to not start 2PC when in read-only mode

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 17, 2019
    Configuration menu
    Copy the full SHA
    79f7894 View commit details
    Browse the repository at this point in the history
  3. Merge remote-tracking branch 'upstream/master' into lock-tables

    * upstream/master:
      Adds a test to check regressions in parser
      Adding in SQL commands to prepare an instance to join an existing shard
      Revert "Support tables that have columns named desc"
      Use a truly random Destination for DestinationAnyShard
    
    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 17, 2019
    Configuration menu
    Copy the full SHA
    fe00e51 View commit details
    Browse the repository at this point in the history

Commits on Jan 18, 2019

  1. Clean up test code

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 18, 2019
    Configuration menu
    Copy the full SHA
    d9882d8 View commit details
    Browse the repository at this point in the history
  2. Fix compilation error

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 18, 2019
    Configuration menu
    Copy the full SHA
    3c5b742 View commit details
    Browse the repository at this point in the history
  3. Fix data race

    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 18, 2019
    Configuration menu
    Copy the full SHA
    16a95a3 View commit details
    Browse the repository at this point in the history
  4. Test code clean up

    Trying to make it clearer what we are testing.
    
    Signed-off-by: Andres Taylor <antaylor@squareup.com>
    systay committed Jan 18, 2019
    Configuration menu
    Copy the full SHA
    21303bb View commit details
    Browse the repository at this point in the history