49475: opt: create library that determines how joins affect input rows r=andy-kimball a=DrewKimball

Previously, there was no simple way to determine whether all rows from
a join input will be included in its output, nor whether input rows will
be duplicated by the join.

This patch adds a library that constructs a Multiplicity struct for join
operators. The Multiplicity can be queried for information about how a
join will affect its input rows (e.g. duplicated, filtered and/or
null-extended). The existing SimplifyLeftJoinWithFilters rule has been
refactored to use this library. The Multiplicity library will also be
useful for future join elimination and limit pushdown rules.

Release note: None

49662: roachtest: don't run schema change workload on 19.2 releases r=spaskob a=spaskob

Fixes cockroachdb#47024.

Release note (bug fix):
The schema change workload is meant for testing the behavior of schema
changes on clusters with nodes with min version 19.2. It will deadlock
on earlier versions.

Co-authored-by: Drew Kimball <andrewekimball@gmail.com>
Co-authored-by: Spas Bojanov <spas@cockroachlabs.com>

49611: searchpath: add pg_extension to searchpath.Iter() before public r=rohany a=otan

Release note (sql change): When doing name resolution via search path,
the `pg_extension` schema (containing tables such as `geometry_columns`,
`geography_columns` and `spatial_ref_sys`) will have an attempted
resolution before the `public` schema. This mimics PostGIS behavior
where the aforementioned tables are in the public schema, and so
by default are discoverable tables with a new CLI session.

Co-authored-by: Oliver Tan <otan@cockroachlabs.com>

When this test was introduced, the cluster version override was added to
ensure that rangefeeds would be used. This was not necessary given the
other testing knob which this change adopts. The problem with overrriding
the version is that it can happen after the automatic version upgrade already
occurs leading to a fatal error due to trying to downgrade the cluster
version.

Fixes cockroachdb#49632.

Release note: None

Previously, there were two pairs of rules for join simplification; one
to handle joins with filters, and one to handle cross joins. Since the
filtersMatchAllLeftRows function can now handle cross joins, there is
no reason to have two sets of rules.

This patch folds the four join simplification rules into two rules:
SimplifyLeftJoin and SimplifyRightJoin.

Release note: None

49681: lease: fix flakey TestRangefeedUpdatesHandledProperlyInTheFaceOfRaces r=ajwerner a=ajwerner

When this test was introduced, the cluster version override was added to
ensure that rangefeeds would be used. This was not necessary given the
other testing knob which this change adopts. The problem with overrriding
the version is that it can happen after the automatic version upgrade already
occurs leading to a fatal error due to trying to downgrade the cluster
version.

Release note: None

Co-authored-by: Andrew Werner <ajwerner@cockroachlabs.com>

The work to remove the dependency has been done, so we're just waiting
until we can remove the code. Until then, make sure this use of Gossip
does not show up as a prominent caller to DeprecatedGossip any more,
for easier bookkeeping.

Release note: None

Visual regression testing is accomplished by `cypress-image-snapshot`
plugin.
The main issue solved with this change is how modules are loaded
from custom `node_modules` location (./opt/node_modules).
- for .ts files `baseUrl` and `typeRoots` are pointing to
`./opt/node_modules` to resolve packages and typings.
- Webpack configuration is extended with additional module
resolver.
- Plugin modules (which aren't processed by webpack) use
helper function which appends path to `opt/node_modules` to
required module name.

Release note: None

When a `bin/*.d` file references an `*.eg.go` file, that file was
required to exist or be buildable by the Makefile. Any commit which
removed a `*.eg.go` file would violate this requirement causing the
build to fail until the offending `bin/*.d` file was removed. In order
to prevent this badness, a catchall `%.eg.go` rule is added which will
force the target dependent on the `bin/%.d` file to be rebuilt.

Fixes cockroachdb#49676

Release note: None

Prior to this commit, filtering a histogram with a multi-span index
constraint could lead to incorrect results if the histogram column
was part of the exact prefix of the constraint. This was due to the
fact that the same value appeared in multiple spans, breaking the
assumption of the histogram code that values in spans were always
ordered and non-overlapping. For example, filtering a histogram on
column b with the constraint /b/c/: [/1/2 - /1/2] [/1/4 - /1/4] would
return an incorrect result, because the values in the matching
histogram bucket would be counted twice.

This commit fixes the problem by only considering a single span if
the column is part of the exact prefix.

Release note (performance improvement): Fixed a bug in the
histogram logic in the optimizer which was causing an over-estimate
for the cardinality of constrained index scans in some cases when
multiple columns of the index were constrained. This problem was
introduced early in the development for the 20.2 release so should
not have ever been part of a release. The fix improves the optimizer's
cardinality estimates so may result in better query plan selection.

Prior to this commit, the statistics_builder was incorrectly estimating
the distinct count of columns that were only slightly constrained as part
of an index constraint. For example, it was estimating based on
constraints such as /a/b: [/1 - /5/6] or /a/b: [ - /5/6] that the distinct
count of column b should be reduced by 2/3. However, in reality, we cannot
assume anything about the distinct count of column b based on those two
constraints.

This commit fixes the estimate by only reducing the distinct count for
columns that are part of the prefix of the constraint (columns for which
all the spans have the same start and end values) or the first column after.

Release note (performance improvement): Fixed the optimizer's distinct count
estimate for columns constrained by an index constraint, which was too low
in some cases. The fix improves the optimizer's cardinality estimates, which
can lead to better query plan selection.

Prior to this commit, it was possible that the estimated distinct count
of a multi-column statistic was larger than the product of the estimated
distinct counts of all of its individual columns. This could happen because
we reduce the distinct count of columns that are constrained by a predicate,
but we don't update the distinct counts of any multi-column stats that
contain that column. This is internally inconsistent and could lead to bad
cardinality estimates.

This commit fixes the problem by explicitly checking for such
inconsistencies and fixing them after the stats for each operator are
built or a column stat is calculated. If it is found that the estimated
distinct count is too high, it is reset to the product of the distinct
counts of its constituent columns.

This commit also adds a check that the distinct count of the multi-column
statistic is no smaller than the largest distinct count of its constituent
columns (although I haven't yet found a test case where this was a problem).

Release note (performance improvement): Fixed the optimizer's estimated
distinct count for a multi-column statistic when all of the columns in the
statistic are constrained by a filter predicate. The fix can lead to
improved cardinality estimates, leading to better query plan selection
in some cases.

49696: Makefile: fix handling of non-existent *.eg.go files r=petermattis a=petermattis

When a `bin/*.d` file references an `*.eg.go` file, that file was
required to exist or be buildable by the Makefile. Any commit which
removed a `*.eg.go` file would violate this requirement causing the
build to fail until the offending `bin/*.d` file was removed. In order
to prevent this badness, a catchall `%.eg.go` rule is added which will
force the target dependent on the `bin/%.d` file to be rebuilt.

Fixes cockroachdb#49676

Release note: None

Co-authored-by: Peter Mattis <petermattis@gmail.com>

This commit adds support for calculating selectivity from multi-column
statistics. It changes selectivityFromDistinctCounts to have the following
semantics:

selectivityFromDistinctCounts calculates the selectivity of a filter
by using estimated distinct counts of each constrained column before
and after the filter was applied. We can perform this calculation in
two different ways: (1) by treating the columns as completely independent,
or (2) by assuming they are correlated.

(1) Assuming independence between columns, we can calculate the selectivity
    by taking the product of selectivities of each constrained column. In
    the general case, this can be represented by the formula:
```
                     ┬-┬ ⎛ new_distinct(i) ⎞
      selectivity =  │ │ ⎜ --------------- ⎟
                     ┴ ┴ ⎝ old_distinct(i) ⎠
                    i in
                 {constrained
                   columns}
```
(2) If useMultiCol is true, we assume there is some correlation between
    columns. In this case, we calculate the selectivity using multi-column
    statistics.
```
                    ⎛ new_distinct({constrained columns}) ⎞
      selectivity = ⎜ ----------------------------------- ⎟
                    ⎝ old_distinct({constrained columns}) ⎠
```
    This formula looks simple, but the challenge is that it is difficult
    to determine the correct value for new_distinct({constrained columns})
    if each column is not constrained to a single value. For example, if
    new_distinct(x)=2 and new_distinct(y)=2, new_distinct({x,y}) could be 2,
    3 or 4. We estimate the new distinct count as follows, using the concept
    of "soft functional dependency (FD) strength":
```
      new_distinct({x,y}) = min_value + range * (1 - FD_strength_scaled)

    where

      min_value = max(new_distinct(x), new_distinct(y))
      max_value = new_distinct(x) * new_distinct(y)
      range     = max_value - min_value

                    ⎛ max(old_distinct(x),old_distinct(y)) ⎞
      FD_strength = ⎜ ------------------------------------ ⎟
                    ⎝         old_distinct({x,y})          ⎠

                        ⎛ max(old_distinct(x), old_distinct(y)) ⎞
      min_FD_strength = ⎜ ------------------------------------- ⎟
                        ⎝   old_distinct(x) * old_distinct(y)   ⎠

                           ⎛ FD_strength - min_FD_strength ⎞
      FD_strength_scaled = ⎜ ----------------------------- ⎟
                           ⎝      1 - min_FD_strength      ⎠
```
    Suppose that old_distinct(x)=100 and old_distinct(y)=10. If x and y are
    perfectly correlated, old_distinct({x,y})=100. Using the example from
    above, new_distinct(x)=2 and new_distinct(y)=2. Plugging in the values
    into the equation, we get:
```
      FD_strength_scaled  = 1
      new_distinct({x,y}) = 2 + (4 - 2) * (1 - 1) = 2
```
    If x and y are completely independent, however, old_distinct({x,y})=1000.
    In this case, we get:
```
      FD_strength_scaled  = 0
      new_distinct({x,y}) = 2 + (4 - 2) * (1 - 0) = 4
```
Note that even if useMultiCol is true and we calculate the selectivity
based on equation (2) above, we still want to take equation (1) into
account. This is because it is possible that there are two predicates that
each have selectivity s, but the multi-column selectivity is also s. In
order to ensure that the cost model considers the two predicates combined
to be more selective than either one individually, we must give some weight
to equation (1). Therefore, instead of equation (2) we actually return the
following selectivity:
```
  selectivity = (1 - w) * (eq. 1) + w * (eq. 2)
```
where w currently set to 0.9.

This selectivity will be used later to update the row count and the
distinct count for the unconstrained columns.

Fixes cockroachdb#34422

Release note (performance improvement): Added support for calculating the
selectivity of filter predicates in the optimizer using multi-column
statistics. This improves the cardinality estimates of the optimizer when
a query has filter predicates constraining multiple columns. As a result,
the optimizer may choose a better query plan in some cases.

This commit removes building muslc from the release process. It is
causing issues with the GEOS build, and we're not confident it is worth
spending the time to debug it.

* Delete mentions of muslc from publishing artifacts / provisional artifacts.
* Delete muslc from the Docker image.
* Remove Makefile items that are musl-c related.
* Unfortunately, I am not able to push the Docker image to
  cockroachdb/builder. I'm not sure that's important atm.

Release note (general change): Removed the publication of muslc
CockroachDB builds.

49560: geo/geoproj: initial framework to import the PROJ library r=petermattis,sumeerbhola a=otan


This commit introduces PROJ v4.9.3 to CockroachDB's compilation. We
cannot use a later version as we would like to exclude installing
sqlite3 as a pre-requisite for now.

To get this to work, `bin/execgen` must depend on the `LIBPROJ` target
to compile, and removed the c-deps ban on `pkg/geo/geoproj`.

Added some stub functions (non-finalized) that test whether
functionality works as expected.

Release note: None



Co-authored-by: Oliver Tan <otan@cockroachlabs.com>

49656: sql: fix panic when attempting to clone a table with a UDT r=rohany a=rohany

Due to how the type metadata was represented, there could have
been structs with unimplemented fields in the `TypeMeta`'s
`UserDefinedTypeName`. This would cause protobuf to panic when
attempting to perform reflection into `types.T` when cloning.

Release note: None

49699: Fixed code rot in builder README r=aaron-crl a=aaron-crl

Updated Dockerfile path in README.

Co-authored-by: Rohan Yadav <rohany@alumni.cmu.edu>
Co-authored-by: Aaron Blum <aaron@cockroachlabs.com>

Add created_by_type and created_by_id columns, along with the index over
these columns, to the system.jobs table.

Add sql migration code to migrate old definition to the new one.

See https://github.com/cockroachdb/cockroach/blob/master/docs/RFCS/20200414_scheduled_jobs.md

Release Notes: None

This change adds VolatilityLeakProof and changes a few hashing builtins to use
it (for consistency with postgres).

In postgres, `proleakproof` is a flag separate from `provolatile`, but all
instances of leak-proof operators are immutable.

We also change the volatility to a simple index since it no longer maps to the
postgres volatility directly. In the optimizer, we will need to have a bitfield
for which types of volatilities are contained in an expression so having them be
small consecutive integers is desirable.

Release note: None

This commit adds a new package for dealing with expressions within table
schemas, such as check constraints, computed columns, and partial index
predicates.

The utility provided previously by the following functions have been moved
and refactored into this new package:

  dequalifyColumnRefs - Now provided by DequalifyColumnRefs.

  makeCheckConstraint - Now provided by CheckConstraintBuilder.

  validateComputedColumn - Now provided by ComputedColumnValidator.

  validateIndexPredicate - Now provided by IndexPredicateValidator.

In addition, several helper functions were refactored into the new
package, including generateMaybeDuplicateNameForCheckConstraint,
generateNameForCheckConstraint, iterColDescriptorsInExpr, and
replaceVars. dummyColumnItems were also moved into the new package.

Release note: None

49685: build: remove muslc from the building process r=otan a=otan

This commit removes building muslc from the release process. It is
causing issues with the GEOS build, and we're not confident it is worth
spending the time to debug it.

* Delete mentions of muslc from publishing artifacts / provisional artifacts.
* Delete muslc from the Docker image.
* Remove Makefile items that are musl-c related.
* Unfortunately, I am not able to push the Docker image to
  cockroachdb/builder. I'm not sure that's important atm.

Release note (general change): Removed the publication of muslc
CockroachDB builds.

Co-authored-by: Oliver Tan <otan@cockroachlabs.com>

49134: opt: add support for calculating selectivity from multi-column stats r=rytaft a=rytaft

**opt: fix bug with histograms and multi-span index constraints**

Prior to this commit, filtering a histogram with a multi-span index
constraint could lead to incorrect results if the histogram column
was part of the exact prefix of the constraint. This was due to the
fact that the same value appeared in multiple spans, breaking the
assumption of the histogram code that values in spans were always
ordered and non-overlapping. For example, filtering a histogram on
column b with the constraint `/b/c/: [/1/2 - /1/2] [/1/4 - /1/4]`
would return an incorrect result, because the values in the matching
histogram bucket would be counted twice.

This commit fixes the problem by only considering a single span if
the column is part of the exact prefix.

Release note (performance improvement): Fixed a bug in the
histogram logic in the optimizer which was causing an over-estimate
for the cardinality of constrained index scans in some cases when
multiple columns of the index were constrained. This problem was
introduced early in the development for the 20.2 release so should
not have ever been part of a release. The fix improves the optimizer's
cardinality estimates so may result in better query plan selection.

**opt: fix distinct count estimates for index constraint columns**

Prior to this commit, the statistics_builder was incorrectly estimating
the distinct count of columns that were only slightly constrained as part
of an index constraint. For example, it was estimating based on
constraints such as `/a/b: [/1 - /5/6]` or `/a/b: [ - /5/6]` that the distinct
count of column b should be reduced by 2/3. However, in reality, we cannot
assume anything about the distinct count of column b based on those two
constraints.

This commit fixes the estimate by only reducing the distinct count for
columns that are part of the prefix of the constraint (columns for which
all the spans have the same start and end values) or the first column after.

Release note (performance improvement): Fixed the optimizer's distinct count
estimate for columns constrained by an index constraint, which was too low
in some cases. The fix improves the optimizer's cardinality estimates, which
can lead to better query plan selection.

**opt: ensure multi-col stats are consistent with single-col stats**

Prior to this commit, it was possible that the estimated distinct count
of a multi-column statistic was larger than the product of the estimated
distinct counts of all of its individual columns. This could happen because
we reduce the distinct count of columns that are constrained by a predicate,
but we don't update the distinct counts of any multi-column stats that
contain that column. This is internally inconsistent and could lead to bad
cardinality estimates.

This commit fixes the problem by explicitly checking for such
inconsistencies and fixing them after the stats for each operator are
built or a column stat is calculated. If it is found that the estimated
distinct count is too high, it is reset to the product of the distinct
counts of its constituent columns.

This commit also adds a check that the distinct count of the multi-column
statistic is no smaller than the largest distinct count of its constituent
columns (although I haven't yet found a test case where this was a problem).

Release note (performance improvement): Fixed the optimizer's estimated
distinct count for a multi-column statistic when all of the columns in the
statistic are constrained by a filter predicate. The fix can lead to
improved cardinality estimates, leading to better query plan selection
in some cases.

**opt: add support for calculating selectivity from multi-column stats**

This commit adds support for calculating selectivity from multi-column
statistics. It changes `selectivityFromDistinctCounts` to have the following
semantics:

`selectivityFromDistinctCounts` calculates the selectivity of a filter
by using estimated distinct counts of each constrained column before
and after the filter was applied. We can perform this calculation in
two different ways: (1) by treating the columns as completely independent,
or (2) by assuming they are correlated.

(1) Assuming independence between columns, we can calculate the selectivity
    by taking the product of selectivities of each constrained column. In
    the general case, this can be represented by the formula:

```
                     ┬-┬ ⎛ new_distinct(i) ⎞
      selectivity =  │ │ ⎜ --------------- ⎟
                     ┴ ┴ ⎝ old_distinct(i) ⎠
                    i in
                 {constrained
                   columns}
```

(2) If `useMultiCol` is true, we assume there is some correlation between
    columns. In this case, we calculate the selectivity using multi-column
    statistics.

```
                    ⎛ new_distinct({constrained columns}) ⎞
      selectivity = ⎜ ----------------------------------- ⎟
                    ⎝ old_distinct({constrained columns}) ⎠
```

    This formula looks simple, but the challenge is that it is difficult
    to determine the correct value for new_distinct({constrained columns})
    if each column is not constrained to a single value. For example, if
    new_distinct(x)=2 and new_distinct(y)=2, new_distinct({x,y}) could be 2,
    3 or 4. We estimate the new distinct count as follows, using the concept
    of "soft functional dependency (FD) strength":

```
      new_distinct({x,y}) = min_value + range * (1 - FD_strength_scaled)

    where

      min_value = max(new_distinct(x), new_distinct(y))
      max_value = new_distinct(x) * new_distinct(y)
      range     = max_value - min_value

                    ⎛ max(old_distinct(x),old_distinct(y)) ⎞
      FD_strength = ⎜ ------------------------------------ ⎟
                    ⎝         old_distinct({x,y})          ⎠

                        ⎛ max(old_distinct(x), old_distinct(y)) ⎞
      min_FD_strength = ⎜ ------------------------------------- ⎟
                        ⎝   old_distinct(x) * old_distinct(y)   ⎠

                           ⎛ FD_strength - min_FD_strength ⎞
      FD_strength_scaled = ⎜ ----------------------------- ⎟
                           ⎝      1 - min_FD_strength      ⎠
```

    Suppose that old_distinct(x)=100 and old_distinct(y)=10. If x and y are
    perfectly correlated, old_distinct({x,y})=100. Using the example from
    above, new_distinct(x)=2 and new_distinct(y)=2. Plugging in the values
    into the equation, we get:

```
      FD_strength_scaled  = 1
      new_distinct({x,y}) = 2 + (4 - 2) * (1 - 1) = 2
```

    If x and y are completely independent, however, old_distinct({x,y})=1000.
    In this case, we get:

```
      FD_strength_scaled  = 0
      new_distinct({x,y}) = 2 + (4 - 2) * (1 - 0) = 4
```

Note that even if `useMultiCol` is true and we calculate the selectivity
based on equation (2) above, we still want to take equation (1) into
account. This is because it is possible that there are two predicates that
each have selectivity s, but the multi-column selectivity is also s. In
order to ensure that the cost model considers the two predicates combined
to be more selective than either one individually, we must give some weight
to equation (1). Therefore, instead of equation (2) we actually return the
following selectivity:

```
  selectivity = (1 - w) * (eq. 1) + w * (eq. 2)
```

where w currently set to 0.9.

This selectivity is used update the row count and the distinct count for the
unconstrained columns.

Fixes cockroachdb#34422

Release note (performance improvement): Added support for calculating the
selectivity of filter predicates in the optimizer using multi-column
statistics. This improves the cardinality estimates of the optimizer when
a query has filter predicates constraining multiple columns. As a result,
the optimizer may choose a better query plan in some cases.

Co-authored-by: Rebecca Taft <becca@cockroachlabs.com>

a41e4a *: Update level_checker to account for sublevels
b14003 internal/manifest: Relax L0 CheckOrdering invariants for flush splits
9249ee db: add max memtable and max batch size checks
bb2f85 vfs: return nil from RemoveAll for nonexistent paths
eb5d41 db: use compaction picker's scores directly in (*db).Metrics
65313e cmd/pebble: sample read amp throughout compaction benchmarks

Release note: None

Use the new storage/FS.RemoveAll method to clean up temporary
directories.

Release note: none

49444: sqlbase: Add created_by columns to system.jobs r=miretskiy a=miretskiy

Informs cockroachdb#49346

Add created_by_type and created_by_id columns, along with the index over
these columns, to the system.jobs table.

Add sql migration code to migrate old definition to the new one.

See https://github.com/cockroachdb/cockroach/blob/master/docs/RFCS/20200414_scheduled_jobs.md

Release Notes: None

49680: sql: add LeakProof volatility r=RaduBerinde a=RaduBerinde

This change adds VolatilityLeakProof and changes a few hashing builtins to use
it (for consistency with postgres).

In postgres, `proleakproof` is a flag separate from `provolatile`, but all
instances of leak-proof operators are immutable.

We also change the volatility to a simple index since it no longer maps to the
postgres volatility directly. In the optimizer, we will need to have a bitfield
for which types of volatilities are contained in an expression so having them be
small consecutive integers is desirable.

Release note: None

![image](https://user-images.githubusercontent.com/16544120/83211087-13631d00-a111-11ea-9a10-cae758a4cfbf.png)


49713: vendor: bump pebble to a41e4ac9 r=jbowens a=jbowens

a41e4a *: Update level_checker to account for sublevels
b14003 internal/manifest: Relax L0 CheckOrdering invariants for flush splits
9249ee db: add max memtable and max batch size checks
bb2f85 vfs: return nil from RemoveAll for nonexistent paths
eb5d41 db: use compaction picker's scores directly in (*db).Metrics
65313e cmd/pebble: sample read amp throughout compaction benchmarks


Co-authored-by: Yevgeniy Miretskiy <yevgeniy@cockroachlabs.com>
Co-authored-by: Radu Berinde <radu@cockroachlabs.com>
Co-authored-by: Jackson Owens <jackson@cockroachlabs.com>

Noticed while working on multi-tenancy and confirmed that this
was an issue manually.

```
root@127.0.0.1:51841/movr> set experimental_enable_enums=true;
SET

Time: 259µs

root@127.0.0.1:51841/movr> select count(*) from crdb_internal.ranges;
  count
---------
     62
(1 row)

Time: 2.251ms

root@127.0.0.1:51841/movr> CREATE TYPE t AS ENUM ();
CREATE TYPE

Time: 1.698ms

root@127.0.0.1:51841/movr> select count(*) from crdb_internal.ranges;
  count
---------
     63
(1 row)

Time: 1.815ms
```

I tried to write a logic test that did essentially this, but that
ended up being pretty frustrating to get right and the end result
seemed too fragile to be worth it.

49667: sql: don't split ranges on user-defined types or schemas r=nvanbenschoten a=nvanbenschoten

Noticed while working on multi-tenancy and confirmed that this
was an issue manually.

```
root@127.0.0.1:51841/movr> set experimental_enable_enums=true;
SET

Time: 259µs

root@127.0.0.1:51841/movr> select count(*) from crdb_internal.ranges;
  count
---------
     62
(1 row)

Time: 2.251ms

root@127.0.0.1:51841/movr> CREATE TYPE t AS ENUM ();
CREATE TYPE

Time: 1.698ms

root@127.0.0.1:51841/movr> select count(*) from crdb_internal.ranges;
  count
---------
     63
(1 row)

Time: 1.815ms
```

I tried to write a logic test that did essentially this, but that
ended up being pretty frustrating to get right and the end result
seemed too fragile to be worth it.

Co-authored-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>

Release note (sql change): Previously, using NULL (or tuples
containing NULLs) as the left-hand-side of an IN operator would not
typecheck unless the NULLs were explicitly casted. Now, the casting is
not required.

For GROUP BY to work, there needs to be a way of encoding / decoding
table keys for each column type. As such, define EncodeTableKey and
DecodeTableKey for geo types to be their underlying protobuf
representation (we currently have no better way!). Note that this
does not yet allow us to support PKs for geospatial (we could, but we
need some more iteration beforehand).

I've shifted the array descriptor markers around as they seem to be new
in 20.2 as well -- so they should be stable in the upcoming alpha
release.

Release note: None

49419: colexec: add binary overloads on datum-backed types r=yuzefovich a=yuzefovich

**tree, colexec: preliminary steps for datum-backed binary overloads**

This commit does a few things that are preliminary to adding support of
binary operations on datum-backed vectors:
1. it exports `fn` field of `tree.BinaryExpr` - we will be using this
function to evaluate the binary operation on datum-backed vectors
2. it renames `decimalOverloadScratch` to `overloadHelper` and plumbs it
from the planning stage for projection and selection operators. This
struct will be used to supply the resolved `BinaryExpr.Fn` function (and
possibly other information) later
3. it cleans up `vectorize_types` logic test a bit since we now support
all types.

Release note: None

**colexec: add support for binary operations on datum-backed types**

This commit adds the support for 4 binary operations when at least one
of the arguments is datum-backed vector and the result is datum-backed
vector as well. We reuse the same function that row engine uses when
evaluating such expressions, and the function is plumbed during the
planning via `overloadHelper` struct. Native (i.e. not datum-backed)
arguments are converted to datums at runtime.

Release note: None

Co-authored-by: Yahor Yuzefovich <yahor@cockroachlabs.com>

Fixes cockroachdb#49716

Release note (performance improvement): Remove compaction-debt based
sstable ingestion backpressure which was artificially slowing down
IMPORTs and RESTOREs on Pebble and not providing any utility on
RocksDB. Removed the private
"rocksdb.ingest_backpressure.pending_compaction_threshold" cluster
setting.

Remove the Dummy import plague on the template files by using the
`goimports` infrastructure to make the .eg.go files get their imports
at generation time.

Release note: None

instead of repeated strings.ReplaceAll

Release note: None

Fixes cockroachdb#49197.

This PR adds the Postgres behavior of creating an implicit array type
for a user defined type when it is created. The implicit array type will
track the user defined type.

Release note: None

49678: sql: create implicit array types for UDTs r=otan,jordanlewis a=rohany

Fixes cockroachdb#49197.

This PR adds the Postgres behavior of creating an implicit array type
for a user defined type when it is created. The implicit array type will
track the user defined type.

Release note: None

Co-authored-by: Rohan Yadav <rohany@alumni.cmu.edu>

Sometimes we see timeouts or other issues contacting the schema registry.
However it would be nice if transient errors could be handled gracefully
without bubbling up to the user.

Release note (enterprise change): Changefeeds now retry after encountering transient errors contacting the Confluent Schema Registry.

49726: storage: remove compaction-debt based ingestion backpressure r=petermattis a=petermattis

Fixes cockroachdb#49716

Release note: Remove compaction-debt based sstable ingestion
backpressure which was artificially slowing down IMPORTs and RESTOREs on
Pebble and not providing any utility on RocksDB. Removed the private
"rocksdb.ingest_backpressure.pending_compaction_threshold" cluster
setting.

Co-authored-by: Peter Mattis <petermattis@gmail.com>

48759: changefeedccl: retry posting to confluent registry up to  10 times r=dt a=dt

Sometimes we see timeouts or other issues contacting the schema registry.
However it would be nice if transient errors could be handled gracefully
without bubbling up to the user.

Release note (enterprise change): Changefeeds now retry after encountering transient errors contacting the Confluent Schema Registry.

Co-authored-by: David Taylor <tinystatemachine@gmail.com>

49727: execgen: some small improvements r=jordanlewis a=jordanlewis

- Use `imports` to avoid having to precisely specify template imports
- Use `strings.Replacer` to streamline some code

Co-authored-by: Jordan Lewis <jordanthelewis@gmail.com>

49687: opt: fold join simplification rules r=andy-kimball a=DrewKimball

Previously, there were two pairs of rules for join simplification; one
to handle joins with filters, and one to handle cross joins. Since the
filtersMatchAllLeftRows function can now handle cross joins, there is
no reason to have two sets of rules.

This patch folds the four join simplification rules into two rules:
SimplifyLeftJoin and SimplifyRightJoin.

Release note: None

Co-authored-by: Drew Kimball <andrewekimball@gmail.com>

49723: sql: allow NULL IN <subquery> to typecheck r=jordanlewis a=rafiss

fixes cockroachdb#49651 

Release note (sql change): Previously, using NULL (or tuples
containing NULLs) as the left-hand-side of an IN operator would not
typecheck unless the NULLs were explicitly casted. Now, the casting is
not required.

Co-authored-by: Rafi Shamim <rafi@cockroachlabs.com>

I think concurrent calls of DrainMeta and Next are causing some flakes.
This commit temporarily removes short-circuiting in the hash joiner when
build side is empty.

Release note: None

49719: colexec: do not short-circuit hash joiner temporarily r=yuzefovich a=yuzefovich

I think concurrent calls of DrainMeta and Next are causing some flakes.
This commit temporarily removes short-circuiting in the hash joiner when
build side is empty.

Fixes: cockroachdb#49715.

Release note: None

Co-authored-by: Yahor Yuzefovich <yahor@cockroachlabs.com>

Add screenshot testing for main pages in Admin UI.
These tests are quite flaky and might fail because
nondeterministic state of the server during tests run.

Release note: None

Previously, error messages received on request for
non-table data on Databases page is not handled and not
displayed for users.

Now, default error message is shown in response on errors.

Release note (admin ui change): Show default error message
about restricted permissions for non-admin users on
Databases page.

The `fmtsafe` linter added in cockroachdb#48040 cockroachdb#48048 was actually
malfunctioning because it was not stripping the vendor prefix
properly.

This patch fixes it.

Fixing the linter uncovered a range of defects throughout the
remainder of the code, some benign and some outright bugs.

Examples:

```
  return pgerror.Wrapf(err, "while running %s", stmt)
```
(The second argument should be the pgcode, not the error string!)

```
 return errors.Errorf(`no consistency checks are defined for %s` + gen.Meta().Name)
```
(Incompatible use of `%s` and string concatenation.)

```
  errors.New(fmt.Sprintf("foo %s", blah))
```
(Should use `Newf()` instead to capture more data in telemetry.)

Release note: None

49714: colflow: use storage/fs.FS.RemoveAll r=jbowens a=jbowens

Use the new storage/FS.RemoveAll method to clean up temporary
directories.

Release note: none

Co-authored-by: Jackson Owens <jackson@cockroachlabs.com>

49660: *: fix the `fmtsafe` linter and fix the fallout r=irfansharif a=knz

Found while working on  cockroachdb#49447.

The `fmtsafe` linter added in cockroachdb#48040 cockroachdb#48048 was actually
malfunctioning because it was not stripping the vendor prefix
properly.

This patch fixes it.

Fixing the linter uncovered a range of defects throughout the
remainder of the code, some benign and some outright bugs.

Examples:

```
  return pgerror.Wrapf(err, "while running %s", stmt)
```
(The second argument should be the pgcode, not the error string!)

```
 return errors.Errorf(`no consistency checks are defined for %s` + gen.Meta().Name)
```
(Incompatible use of `%s` and string concatenation.)

```
  errors.New(fmt.Sprintf("foo %s", blah))
```
(Should use `Newf()` instead to capture more data in telemetry.)

Release note: None

Co-authored-by: Raphael 'kena' Poss <knz@thaumogen.net>

We are able to accelerate certain spatial predicates by determining
earlier on whether their bounding boxes intersect. This PR applies that
change.

It is worth noting that I have not applied this to geogfn yet, as the
bounding box for geogfns should be 3D. That's a biggish change that I
will leave for a follow up PR.

Release note: None

There was a teasing maxResults field already, completely unused.

Release note: None

Release note: None

The code was looking at the wrong error.

Release note: None

Release note: None

Seems to be breaking Upload Binaries (not sure why) because execgen
depends on geo. The dependencies should make it work (not sure why it
doesn't), but it doesn't seem important to solve this atm.

Release note: None

And also restrict SpanResolver's interface to returning only
ReplicaDescriptors, not also NodeDescriptors. This is all towards
reducing the physical planning's dependency on gossip - node descriptors
come from gossip.
This patch doesn't yet eliminate the dependency - leaseholder oracles
still use gossip to translate store ids to node ids and to order
replicas, but it's a step in the right direction.

Touches cockroachdb#48432

Release note: None

Release note: None

... in range cache eviction functions. And from there, upwards.

Release note: None

Fixes cockroachdb#48405, cockroachdb#49049

Release note (sql change): Implemented the geometry based builtins `ST_Summary`.

It seems like I ran into a proto bug, opened
gogo/protobuf#693.

Release note: None

Previously, the optimizer could not fold two grouping operators into
a single equivalent grouping operator.

This patch adds a rule that effects this transformation under the
following conditions:
1. All of the outer aggregates are aggregating on the output columns
   of the inner aggregates.
2. Every inner-outer aggregation pair can be replaced with an
   equivalent single aggregate.
3. The inner grouping columns functionally determine the outer
   grouping columns.
4. Both grouping operators are unordered.

As an example, the following query pairs are equivalent:
```
SELECT sum(t) FROM (SELECT sum(b) FROM ab GROUP BY a) AS g(t);
SELECT sum(b) FROM ab;

SELECT max(t) FROM (SELECT max(b) FROM ab GROUP BY a) AS g(t);
SELECT max(b) FROM ab;

SELECT sum_int(t) FROM (SELECT count(b) FROM ab GROUP BY a) AS g(t);
SELECT count(b) FROM ab;
```
This situation is rare in direct SQL queries, but can arise when
composing views and queries.

Release note (sql change): The optimizer can now fold two grouping
operators together when they are aggregating over functions like sum.

CMake is broken for the value of `PTHREAD_MUTEX_RECURSIVE` on certain
operating systems. To rectify this, we've patched PROJ at version 4.9.3
at `cc4a178` with a fix. We'll be using the CRDB fork of PROJ for the
meantime to fix this for now.

Release note: None

49738: geo: implement ST_Summary for Geography/Geometry r=otan a=hueypark

Fixes cockroachdb#48405, cockroachdb#49049

Release note (sql change): Implemented the geometry based builtins `ST_Summary`.

Co-authored-by: Jaewan Park <jaewan.huey.park@gmail.com>

49755: c-deps/proj: bump PROJ to compile on more systems r=knz a=otan

CMake is broken for the value of `PTHREAD_MUTEX_RECURSIVE` on certain
operating systems. To rectify this, we've patched PROJ at version 4.9.3
at `cc4a178` with a fix. We'll be using the CRDB fork of PROJ for the
meantime to fix this for now.

fixes cockroachdb#49749

Release note: None

49756: types: fix bug where calling String() on a UDT array would panic r=otan a=rohany

It seems like I ran into a proto bug, opened
gogo/protobuf#693.

Release note: None

49757: geo: remove geoproj dependency for now r=otan a=otan

Seems to be breaking Upload Binaries (not sure why) because execgen
depends on geo. The dependencies should make it work (not sure why it
doesn't), but it doesn't seem important to solve this atm (planning to hide
it in some sort of dependency injection hack later).

Release note: None

Co-authored-by: Oliver Tan <otan@cockroachlabs.com>
Co-authored-by: Rohan Yadav <rohany@alumni.cmu.edu>

This commit removes two private settings that could disable planning of
multiple join readers (i.e would disable distribution) and disable
planning merge joiners (i.e. would simply prohibit the use of merge
joiners). It also retires two settings that have been removed after 20.1
was cut.

Release note: None

A function dealing with inserting multiple descriptors
(insertRangeDescriptorsLocked) was short-circuiting if any descriptor
was found to be stale, which was causing trouble for the caller which
had to call it twice.

Release note: None

Release note: None

Release note: None

The eviction interface was weird - evictions took both a key and a
descriptor; they performed a lookup using the key and then they compared
with the descriptor. This was seemingly done to cover two cases -
evictions by key and evictions by descriptor. Everything was complicated
by the "inverted" argument which modified the key in a subtle way. This
patch splits the interface into an explicit eviction by key, which
doesn't take a descriptor, and one by descriptor which doesn't take a
key. And the inverted guy goes away, as it turns out that it's not
needed any more.

Release note: None

The EvictionToken was quite hard to read because it interfaced with the
RangeDescriptorCache through function pointers bound which were binding
arguments for arguments to the RangeDescriptorCache's eviction and
inserting methods. Prior commits have removed most of those arguments,
so this indirection doesn't serve any purpose any more. Now the
EvictionToken logic is readable.

Release note: None

Remove unnecessary protection against nil return.

Release note: None

49290: cli: limit flag for debug range-data / debug keys r=andreimatei a=andreimatei

There was a teasing maxResults field already, completely unused.

Release note: None

49661: sql: remove two private settings r=yuzefovich a=yuzefovich

This commit removes two private settings that could disable planning of
multiple join readers (i.e would disable distribution) and disable
planning merge joiners (i.e. would simply prohibit the use of merge
joiners). It also retires two settings that have been removed after 20.1
was cut.

Release note: None

49675: sql: stop dealing with node addresses in DistSQL planning r=andreimatei a=andreimatei

And also restrict SpanResolver's interface to returning only
ReplicaDescriptors, not also NodeDescriptors. This is all towards
reducing the physical planning's dependency on gossip - node descriptors
come from gossip.
This patch doesn't yet eliminate the dependency - leaseholder oracles
still use gossip to translate store ids to node ids and to order
replicas, but it's a step in the right direction.

Touches cockroachdb#48432

Release note: None

Co-authored-by: Andrei Matei <andrei@cockroachlabs.com>
Co-authored-by: Yahor Yuzefovich <yahor@cockroachlabs.com>

49641: pgwire/pgtest: make the tests work both on pg and crdb r=rafiss,rohany a=knz

There was some accumulated incompatibilities in the test files.
This patch fixes it, and in the process of doing so discovers
two new bugs (cockroachdb#49639 and cockroachdb#49640).

Summary of changes to the DSL:

- the new `only` directive skips an entire test file if the db
  does not match (used for the crdb-specific portal bug test file)

- the new flags `crdb_only` and `noncrdb_only` skip over a
  test directive if the db does not match.

- the new flags `ignore_table_oids` and `ignore_type_oids` replace
  the corresponding OIDs in the RowDescription message by 0
  prior to comparing with the expected value.

Release note: None

Co-authored-by: Raphael 'kena' Poss <knz@thaumogen.net>

This patch lifts the concern about consistently resolving spans from a
single range to the same node up to the spanResolver, from one of the
replica oracles. Regardless of what oracle is used (i.e. the
lease-holder aware one, or the follower-reads one), making a single node
in charge of all the spans that fall within one range seems like a good
idea. Currently only the leaseholder oracle was doing this.
This makes SpanResolver.ReplicaInfo() idempotent.

Release note: None

We were sleeping to deal with range cache clobbering, but we no longer
have these clobbering problems since cache updates are now resistant to
stale information overwiting newer one.

Release note: None

49657: sql: create schemaexpr package r=mgartner a=mgartner

This commit adds a new package for dealing with expressions within table
schemas, such as check constraints, computed columns, and partial index
predicates.

The utility provided previously by the following functions have been moved
and refactored into this new package:

  dequalifyColumnRefs - Now provided by DequalifyColumnRefs.

  makeCheckConstraint - Now provided by CheckConstraintBuilder.

  validateComputedColumn - Now provided by ComputedColumnValidator.

  validateIndexPredicate - Now provided by IndexPredicateValidator.

In addition, several helper functions were refactored into the new
package, including generateMaybeDuplicateNameForCheckConstraint,
generateNameForCheckConstraint, iterColDescriptorsInExpr, and
replaceVars. dummyColumnItems were also moved into the new package.

Release note: None


Co-authored-by: Marcus Gartner <marcus@cockroachlabs.com>

…ontainer

Additionally,
- added a randomized correctness test that compares the results of the indexed
  and numbered containers.
- added benchmark cases to the joinReader benchmark that limit memory. None of
  the workloads have repeated reads of the same right row and all access the
  right rows in monotonically increasing order so the difference between the
  two containers is due to the numbered container avoiding the overhead of
  populating the cache.
- reduced the number of allocations in newNumberedDiskRowIterator.
  - the accesses slices share the same underlying slice.
  - a row copy, when there is a miss and the row is not added to the cache, is
    eliminated. When a copy is needed and we have evicted a row from the cache,
    the copying reuses that evicted row.
  - allocations of the map, map elements are reused.

Fixes cockroachdb#48118

Release note: None

4887c526 vfs: return nil from MemFS.RemoveAll if parent doesn't exist

Prerequistite for cockroachdb#49717.

Release note: None

49276: roachtest/jobs: better failed test reporting r=spaskob a=spaskob

When `jobs/mixed-versions` test fails it is useful to report
on what mixed version configuration it failed. We also test
for failed jobs in between each node upgrade to catch failures
as quickly as possible.

Release note: none.

Co-authored-by: Spas Bojanov <spas@cockroachlabs.com>

49593: build: Upgrade base image to deployment dockerfile r=bobvawter a=bobvawter

This change updates the deployment base image from Debian 9.8 to 9.12.

Fixes: cockroachdb#41390

Release note (build change): Release Docker images are now built on
Debian 9.12.

Co-authored-by: Bob Vawter <bob@vawter.org>

49648: kvclient: assorted cleanups to the range desc cache r=andreimatei a=andreimatei

See individual commits.

49669: rowcontainer,rowexec: switch joinReader to use DiskBackedNumberedRowC… r=sumeerbhola a=sumeerbhola

…ontainer

Additionally,
- added a randomized correctness test that compares the results of the indexed
  and numbered containers.
- added benchmark cases to the joinReader benchmark that limit memory. None of the
  workloads have repeated reads of the same right row and all access the right
  rows in monotonically increasing order so the difference between the two
  containers is due to the numbered container avoiding the overhead of populating
  the cache.
- reduced the number of slice allocations in newNumberedDiskRowIterator.

Fixes cockroachdb#48118

Release note: None

49769: sql/physicalplan: small refactor r=andreimatei a=andreimatei

This patch lifts the concern about consistently resolving spans from a
single range to the same node up to the spanResolver, from one of the
replica oracles. Regardless of what oracle is used (i.e. the
lease-holder aware one, or the follower-reads one), making a single node
in charge of all the spans that fall within one range seems like a good
idea. Currently only the leaseholder oracle was doing this.
This makes SpanResolver.ReplicaInfo() idempotent.

Co-authored-by: Andrei Matei <andrei@cockroachlabs.com>
Co-authored-by: sumeerbhola <sumeer@cockroachlabs.com>

This commit adds a VolatilitySet logical property that will replace
CanHaveSideEffects. For now, the property is not used for anything. The property
reflects the volatility of any functions; future changes will incorporate the
volatility of casts and scalar operators.

The reason for why we need a set (rather than the "max" volatility) is that we
will need to reoptimize a cached expression if it contains stable operators
(regardless of whether it also contains volatile operators).

Release note: None

49725: opt: introduce VolatilitySet logical property r=RaduBerinde a=RaduBerinde

This commit adds a `VolatilitySet` logical property that will replace
`CanHaveSideEffects`. For now, the property is not used for anything. The property
reflects the volatility of any functions; future changes will incorporate the
volatility of casts and scalar operators.

The reason for why we need a set (rather than the "max" volatility) is that we
will need to reoptimize a cached expression if it contains stable operators
(regardless of whether it also contains volatile operators).

Release note: None

Co-authored-by: Radu Berinde <radu@cockroachlabs.com>

A DistSender test seems to have been left over from a different time - a
time when presumably the responsibilities of the DistSender and the
transport were split differently than today. This test was no longer
making a lot of sense because some subtests where generating SendErrors
in the transport, which is not a thing.

Release note: None

RangeKeyMismatchError.MismatchedRange is always set, even though it's
nullable. Client code was inconsistently protecting against it being
nil. This patch makes the field non-nullable, to clarify the situation.

Release note: None

49627: opt: add rule to fold two grouping operators into one r=DrewKimball a=DrewKimball

Previously, the optimizer could not fold two grouping operators into
a single equivalent grouping operator.

This patch adds a rule that effects this transformation under the
following conditions:
1. All of the outer aggregates are aggregating on the output columns
   of the inner aggregates.
2. Every inner-outer aggregation pair can be replaced with an
   equivalent single aggregate.
3. The inner grouping columns functionally determine the outer
   grouping columns.
4. Both grouping operators are unordered.

As an example, the following query pairs are equivalent:
```
SELECT sum(t) FROM (SELECT sum(b) FROM ab GROUP BY a) AS g(t);
SELECT sum(b) FROM ab;

SELECT max(t) FROM (SELECT max(b) FROM ab GROUP BY a) AS g(t);
SELECT max(b) FROM ab;

SELECT sum_int(t) FROM (SELECT count(b) FROM ab GROUP BY a) AS g(t);
SELECT count(b) FROM ab;
```
This situation is rare in direct SQL queries, but can arise when
composing views and queries.

Release note (sql change): The optimizer can now fold two grouping
operators together when they are aggregating over functions like sum.

Co-authored-by: Drew Kimball <drewk@cockroachlabs.com>

This change adds volatility information for unary, comparison, and binary
operators. A subset of comparison operators are leak-proof; a small set of
operators (mostly those involving the current timezone) are stable; and the rest
are immutable.

Release note: None

49711: encoding: define EncodeTableKey / DecodeTableKey for Geo types r=rohany a=otan

For GROUP BY to work, there needs to be a way of encoding / decoding
table keys for each column type. As such, define EncodeTableKey and
DecodeTableKey for geo types to be their underlying protobuf
representation (we currently have no better way!). Note that this
does not yet allow us to support PKs for geospatial (we could, but we
need some more iteration beforehand).

I've shifted the array descriptor markers around as they seem to be new
in 20.2 as well -- so they should be stable in the upcoming alpha
release.

Release note: None



49754: geo/geomfn: accelerate spatial predicates with bounding box checks r=sumeerbhola a=otan

We are able to accelerate certain spatial predicates by determining
earlier on whether their bounding boxes intersect. This PR applies that
change.

It is worth noting that I have not applied this to geogfn yet, as the
bounding box for geogfns should be 3D. That's a biggish change that I
will leave for a follow up PR.

Release note: None

Co-authored-by: Oliver Tan <otan@cockroachlabs.com>

With this commit, serialized partial index predicates are now stored on
index descriptors. Predicates are dequalified so that database and table
names are not included in column references.

Release note: None

49772: vendor: bump Pebble to 4887c526300055e1c30635c53fd16b3fe9d9e132 r=jbowens a=jbowens

4887c526 vfs: return nil from MemFS.RemoveAll if parent doesn't exist

Prerequistite for cockroachdb#49717.

Release note: None

Co-authored-by: Jackson Owens <jackson@cockroachlabs.com>

GEOSRelatePattern seems to avoid the checks for wildcards, making
ST_ContainsProperly faster.

Release note: None

49759: geo/geomfn: use GEOSRelatePattern for ST_ContainsProperly r=sumeerbhola a=otan

GEOSRelatePattern seems to avoid the checks for wildcards, making
ST_ContainsProperly faster.

Release note: None

Co-authored-by: Oliver Tan <otan@cockroachlabs.com>

Previously, a join under a grouping operator could not be eliminated
even when doing so would not affect the output of the grouping operator.

This patch introduces two rules that match grouping operators (Groupby,
ScalarGroupBy and DistinctOn) that take a LeftJoin or InnerJoin as input.
The join is removed if the following conditions are met:

1. Only columns from the left (or right)  input are being used.

2. It can be statically proven that removal of the join will not effect
   the output of the grouping operator.
```
CREATE TABLE xy (x INT PRIMARY KEY, y INT);
CREATE TABLE uv (u INT PRIMARY KEY, v INT);

SELECT x, sum(y) FROM xy LEFT JOIN uv ON x=u GROUP BY x;
=>
SELECT x, sum(y) FROM xy GROUP BY x;
```

Fixes cockroachdb#49141

Release note (sql change): The optimizer can now eliminate an unnecessary
join that is the input to a GroupBy operator.

Implemented ST_Buffer. Had to write our own parsing logic to make it
compatible with C.

Also ran clang-format and fixed up a few bad infos.

Release note (sql change): Implement ST_Buffer for geometry and string
variants.

49722: geo/geomfn: implement ST_Buffer r=summerbhola a=otan

Implemented ST_Buffer. Had to write our own parsing logic to make it
compatible with C.

Also ran clang-format and fixed up a few bad infos.

Resolves cockroachdb#48890
Resolves cockroachdb#48891
Resolves cockroachdb#48802
Resolves cockroachdb#48803
Resolves cockroachdb#48804

Release note (sql change): Implement ST_Buffer for geometry and string
variants.

Co-authored-by: Oliver Tan <otan@cockroachlabs.com>

This PR adds the `crdb_internal.create_type_statments` table, which
contains create statements for user defined types. This will be used
when generating create statments for `cockroach dump` in cockroachdb#47765.

This PR additionally refactors and deduplicates some code that iterates
over all user defined type descriptors in other virtual tables.

Release note (sql change): Add the
`crdb_internal.create_type_statements` virtual table. It holds create
statements for user defined types.

49683: opt: add rule to eliminate a do-nothing join under a GroupBy r=DrewKimball a=DrewKimball

Previously, a join under a grouping operator could not be eliminated
even when doing so would not effect the output of the grouping operator.

This patch introduces two rules that match grouping operators (Groupby,
ScalarGroupBy and DistinctOn) that take a LeftJoin or InnerJoin as input.
The join is removed if the following conditions are met:

1. Only columns from the left (or right)  input are being used.

2. It can be statically proven that removal of the join will not effect
   the output of the grouping operator.
```
CREATE TABLE xy (x INT PRIMARY KEY, y INT);
CREATE TABLE uv (u INT PRIMARY KEY, v INT);

SELECT x, sum(y) FROM xy LEFT JOIN uv ON x=u GROUP BY x;
=>
SELECT x, sum(y) FROM xy GROUP BY x;
```

Fixes cockroachdb#49141

Release note (sql change): The optimizer can now eliminate an unnecessary
join that is the input to a GroupBy operator.

Co-authored-by: Drew Kimball <andrewekimball@gmail.com>

When performing an incremental backup with revision history, we want to
include all spans that were public at any point during the latest
interval under consideration (the time between the last backup and when
you are performing the incremental backup).

However, consider a table that was dropped before the interval started.
The table's descriptor may still be visible (in the DROPPED state). We
should not be interested in the spans for this database. So, when going
through the list of revisions to table descriptors, we should make sure
that the table in question was not DROPPED at some point during this
interval.

To see why this is needed, consider the following scenario (all backups
are assumed to be taken with revision_history):
- Create table mydb.a
- Create table mydb.b
- Drop table mydb.a
- Take a backup of mydb (full)
- Take an incremental backup (inc) of mydb
- Create table mydb.c
- Take another incremental backup (inc2) of mydb

The backup "inc" and "inc2" should not be considered as backing up table
"mydb.a", since it has been dropped at that point. Note that since "inc"
does not see any descriptor changes, only "mydb.b" is included in its
backup. However, previously, "inc2" would see a table descriptor for
"mydb.a" (even though it is dropped) and include it in the set of spans
included in "inc2". This is an issue since "inc" did not include this
span, and thus there is a gap in the coverage for this dropped table.

Release note (bug fix): There was a bug where when performing
incremental backups with revision history on a database (or full
cluster) and a table in the database you were backing up was dropped and
then other tables were lated create the backup would return an error.
This is now fixed.

Fixes cockroachdb#48971
Fixes cockroachdb#48972

This PR adds following builtin functions
* ST_LineInterpolatePoint{{geometry, float8}}
* ST_LineInterpolatePoints{{geometry, float8, bool}}

which works for LineString only, allows us to determine one or more
interpolated points in the LineString which at an integral multiple
of given fraction of LineString's total length.

Release note (sql change): This PR implement adds the following built-in functions.
* ST_LineInterpolatePoint{{geometry, float8}}
* ST_LineInterpolatePoints{{geometry, float8, bool}}

49742: geo/geomfn: implement ST_LineInterpolatePoint(s) r=otan a=abhishek20123g

Fixes cockroachdb#48971
Fixes cockroachdb#48972

This PR adds following builtin functions
* ST_LineInterpolatePoint{{geometry, float8}}
* ST_LineInterpolatePoints{{geometry, float8, bool}}

which works for LineString only, allows us to determine one or more
interpolated points in the LineString which at an integral multiple
of given fraction of LineString's total length.

Release note (sql change): This PR implement adds the following built-in functions.
* ST_LineInterpolatePoint{{geometry, float8}}
* ST_LineInterpolatePoints{{geometry, float8, bool}}

Co-authored-by: abhishek20123g <abhishek20123g@gmail.com>