Commits on Jun 25, 2020

1. sql: add metadata router …

   This commit adds a DistSQL router that routes metadata to a given output
   stream based on the metadata's StreamIdx. This is used by flows which
   schedule DistSQL processors in order to coordinate work around the
   cluster.
   
   The motivation for this change is a refactoring to Restore which
   attempts to distribute the work of performing the restore across the
   cluster. RESTORE works by creating a pipeline of work with 2 stages. The
   first stage splits and scatters the ranges we are going to import. This
   shuffling means that a range could end up on a random node in the
   cluster. The second stage of the pipeline is to download the data from
   the backup file and restore the data, which is accomplished through an
   AddSSTable request. It is beneficial (as well as the motivation for this
   refactor) for the node which issues this request to also be the
   leaseholder of the range it is trying to import. This is to prevent a
   situation where many nodes are all waiting on one node that had the
   misfortune of being the recipient of many scatter ranges in a row.
   
   This router would allow restore to be implemented with 2 separate
   processors: one that splits and scatters the data, and one that imports
   the data. Using this router, the split and scatter processor could
   determine, on the fly, which processor is suitable to import the data.
   
   Release note: None

   

   pbardea committed Jun 25, 2020
   Configuration menu

   • View commit details
   • Copy full SHA for 80a4983
   • Browse repository at this point

   Copy the full SHA

   80a4983 View commit details

   Browse the repository at this point in the history