Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

opt: prune partial index columns and simplify partial index projections #58358

Merged
merged 6 commits into from
Jan 7, 2021
Merged

opt: prune partial index columns and simplify partial index projections #58358

merged 6 commits into from
Jan 7, 2021

Commits on Jan 7, 2021

  1. opt: do not derive prune columns for Upsert, Update, Delete 

    We no longer derive output prune columns for Upsert, Update, and Delete
ops in `DerivePruneCols`. There are no PruneCols rules for these
operators, so deriving their prune columns was only performing
unnecessary work. There are other rules that prune the fetch and return
columns for these operators. These rules do not rely on
`DerivePruneCols`.

Release note: None
    @mgartner
    mgartner committed Jan 7, 2021
    Configuration menu
    Copy the full SHA
    2b6f52b View commit details
    Browse the repository at this point in the history

  2. sql: remove logic to determine fetch cols in row updater 

    Previously, the `row.MakeUpdater` function had logic to determine the
fetch columns required for an update operation. This is not necessary
because the cost based optimizer already determines the necessary fetch
columns and plumbs them to `MakeUpdater` as the `requestedCols`
argument.

Release note: None
    @mgartner
    mgartner committed Jan 7, 2021
    Configuration menu
    Copy the full SHA
    07e9364 View commit details
    Browse the repository at this point in the history

  3. opt: safer access to partial index predicates in TableMeta 

    Previously, partial index predicate expressions in TableMeta were the
source-of-truth used within the optimizer to determine if an index is a
partial index. However, partial index predicates are not added to
TableMeta for all types of statements in optbuilder. Therefore, it was
not safe to assume this was a source-of-truth.

This commit unexports the map of partial index predicates in TableMeta.
Access to partial index predicates must now be done via
`TableMeta.PartialIndexPredicate`. This function checks the catalog to
determine if an index is a partial index, and panics if there is not a
corresponding predicate expression in the partial index predicate map.
This makes the function an actual a source-of-truth.

Release note: None
    @mgartner
    mgartner committed Jan 7, 2021
    Configuration menu
    Copy the full SHA
    c5e72ab View commit details
    Browse the repository at this point in the history

  4. opt: move addPartialIndexPredicatesForTable to optbuilder/partial_ind… 

    …ex.go

Release note: None
    @mgartner
    mgartner committed Jan 7, 2021
    Configuration menu
    Copy the full SHA
    62ab822 View commit details
    Browse the repository at this point in the history

  5. opt: prune update/upsert fetch columns not needed for partial indexes 

    Indexed columns of partial indexes are now only fetched for UPDATE and
UPSERT operations when needed. They are pruned in cases where it is
guaranteed that they are not needed to build old or new index entries.
For example, consider the table and UPDATE:

    CREATE TABLE t (
      a INT PRIMARY KEY,
      b INT,
      c INT,
      d INT,
      INDEX (b) WHERE c > 0,
      FAMILY (a), FAMILY (b), FAMILY (c), FAMILY (d)
    )

    UPDATE t SET d = d + 1 WHERE a = 1

The partial index is guaranteed not to change with this UPDATE because
neither its indexed columns nor the columns referenced in its predicate
are mutating. Therefore, the existing values of b do not need to be
fetched to maintain the state of the partial index. Furthermore, the
primary index does require the existing values of b because no columns
in b's family are mutating. So, b can be pruned from the UPDATE's fetch
columns.

Release note (performance improvement): Previously, indexed columns of
partial indexes were always fetched for UPDATEs and UPSERTs. Now they
are only fetched if they are required for maintaining the state of the
index. If an UPDATE or UPSERT mutates columns that are neither indexed by a
partial index nor referenced in a partial index predicate, they will no
longer be fetched (assuming that they are not needed to maintain the
state of other indexes, including the primary index).
    @mgartner
    mgartner committed Jan 7, 2021
    Configuration menu
    Copy the full SHA
    4e140b6 View commit details
    Browse the repository at this point in the history

  6. opt: normalize partial index PUT/DEL projections to false 

    The `SimplifyPartialIndexProjections` normalization rule has been added
that normalizes synthesized partial index PUT and DEL columns to False
when it is guaranteed that a mutation will not require changes to the
associated partial index. This normalization can lead to further
normalizations, such as pruning columns that the synthesized projections
relied on.

The motivation for this change is to allow fully disjoint updates to
different columns in the same row, when the columns are split across
different families. By pruning columns not needed to maintain a partial
index, we're not forced to scan all column families. This can ultimately
reduce contention during updates.

Release note (performance improvement): UPDATE operations on tables with
partial indexes no longer evaluate partial index predicate expressions
when it is guaranteed that the operation will not alter the state of the
partial index. In some cases, this can eliminate fetching the existing
value of columns that are referenced in partial index predicates.
    @mgartner
    mgartner committed Jan 7, 2021
    Configuration menu
    Copy the full SHA
    039fb1b View commit details
    Browse the repository at this point in the history