Skip to content

Commit

Permalink
Merge #49136 #49213 #49285 #49336
Browse files Browse the repository at this point in the history 
49136: opt: allow join filter FDs to be used in determining a left join key r=rytaft a=DrewKimball

Previously, the functional dependencies from a join's "on" filters were
not used in determining the key of the output of a left join.

This patch allows a key that only has columns from the left input of the
LeftJoin to be used if one can be proven to exist.

The columns of the left input of the LeftJoin form a strict key over the
result of the left join when the following conditions are met:

 1. The original left input had a strict key.

 2. The columns of the left input form a strict key over the result of an
    InnerJoin between the left and right inputs on the given filters.
    (Or, in other words, the left columns form a strict key over the
     filtered cartesian product of the join inputs).

If the above conditions do not hold, a key over the unfiltered cartesian
product will simply be used, if there is one.

Release note: None

49213: sql: populate the pg_enum catalog table r=rohany a=rohany

Fixes #48359.

Release note (sql change): Populate the catalog table
`pg_catalog.pg_enum`.

49285: vendor: bump pebble to af0d71572c7bd24deb066d09eee646b6cb7460a4 r=petermattis a=petermattis

* internal/private: ratchet visible sequence number too
* internal/rate: add Limiter.Delay{,N}
* db: refactor tableCache.estimateDiskUsage into withReader
* internal/cache: optimize common case of Cache.Delete()
* internal/cache: remove weak cache handles
* internal/randvar: optimize Uniform.IncMax
* internal/{manual,rawalloc}: remove empty assembly files
* internal/replay: assign ingested tables correct sequence number
* db: wait for pending writes during ingestion
* db: replace tableCacheShard LRU with CLOCK-Pro
* internal/randvar: reduce exclusive locks in randvar
* cmd/pebble: miscellaneous ycsb benchmark improvements
* internal/rate: change Reserve{,N} to return a value

Fixes #49154

Release note: None

49336: roachtest: bump latest patch version number to 19.2.7 r=jlinder a=jlinder

Before: The latest 19.2 release was 19.2.6.

Why: 19.2.7 is released

Now: The latest noted release is 19.2.7.

Release note: None

Co-authored-by: Drew Kimball <andrewekimball@gmail.com>
Co-authored-by: Rohan Yadav <rohany@alumni.cmu.edu>
Co-authored-by: Peter Mattis <petermattis@gmail.com>
Co-authored-by: James H. Linder <jamesl@cockroachlabs.com>
  • Loading branch information
@DrewKimball @rohany
@petermattis @jlinder
5 people committed May 20, 2020
5 parents 0437d44 + 35b791e + 7ff0e07 + 389b94f + b01659a commit 1bd13e7
Show file tree
Hide file tree
Showing 26 changed files with 986 additions and 570 deletions.
4 changes: 2 additions & 2 deletions Gopkg.lock
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion pkg/cmd/roachtest/test_runner.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1143,7 +1143,7 @@ func PredecessorVersion(buildVersion version.Version) (string, error) {
// map.
verMap := map[string]string{
"20.2": "20.1.0",
"20.1": "19.2.6",
"20.1": "19.2.7",
"19.2": "19.1.5",
"19.1": "2.1.9",
"2.2": "2.1.9",
Expand Down
22 changes: 12 additions & 10 deletions pkg/sql/logictest/testdata/logic_test/pg_catalog
View file
Original file line number Diff line number Diff line change
Expand Up @@ -916,13 +916,22 @@ JOIN pg_constraint ON objid=pg_constraint.oid AND refobjid=pg_constraint.conindi
contype
f

## pg_catalog.pg_type

# Create some user defined types for testing.
## pg_catalog.pg_enum
statement ok
CREATE TYPE newtype1 AS ENUM ('v1', 'v2');
CREATE TYPE newtype2 AS ENUM ('v3', 'v4')

query OORT colnames
SELECT * FROM pg_enum
----
oid enumtypid enumsortorder enumlabel
1043025669 100063 0 v1
1043025861 100063 1 v2
1865129000 100064 0 v3
1865129192 100064 1 v4

## pg_catalog.pg_type

query OTOOIBT colnames
SELECT oid, typname, typnamespace, typowner, typlen, typbyval, typtype
FROM pg_catalog.pg_type
Expand Down Expand Up @@ -1626,13 +1635,6 @@ SELECT objoid, classoid, description FROM pg_catalog.pg_shdescription
----
objoid classoid description

## pg_catalog.pg_enum

query OORT colnames
SELECT * FROM pg_catalog.pg_enum
----
oid enumtypid enumsortorder enumlabel

## pg_catalog.pg_event_trigger

query TTOOTT colnames
Expand Down
10 changes: 8 additions & 2 deletions pkg/sql/opt/memo/logical_props_builder.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1965,10 +1965,16 @@ func (h *joinPropsHelper) setFuncDeps(rel *props.Relational) {
addOuterColsToFuncDep(rel.OuterCols, &rel.FuncDeps)

case opt.LeftJoinOp, opt.LeftJoinApplyOp:
rel.FuncDeps.MakeLeftOuter(h.rightProps.OutputCols, notNullInputCols)
rel.FuncDeps.MakeLeftOuter(
&h.leftProps.FuncDeps, &h.filtersFD,
h.leftProps.OutputCols, h.rightProps.OutputCols, notNullInputCols,
)

case opt.RightJoinOp:
rel.FuncDeps.MakeLeftOuter(h.leftProps.OutputCols, notNullInputCols)
rel.FuncDeps.MakeLeftOuter(
&h.rightProps.FuncDeps, &h.filtersFD,
h.rightProps.OutputCols, h.leftProps.OutputCols, notNullInputCols,
)

case opt.FullJoinOp:
rel.FuncDeps.MakeFullOuter(h.leftProps.OutputCols, h.rightProps.OutputCols, notNullInputCols)
Expand Down
8 changes: 4 additions & 4 deletions pkg/sql/opt/memo/testdata/logprops/join
View file
Original file line number Diff line number Diff line change
Expand Up @@ -317,8 +317,8 @@ SELECT *, rowid FROM xysd RIGHT JOIN uv ON x=u
----
right-join (hash)
├── columns: x:1(int) y:2(int) s:3(string) d:4(decimal) u:5(int) v:6(int!null) rowid:7(int!null)
├── key: (1,7)
├── fd: (1)-->(2-4), (3,4)~~>(1,2), (7)-->(5,6)
├── key: (7)
├── fd: (1)-->(2-4), (3,4)~~>(1,2), (7)-->(1-6)
├── prune: (2-4,6,7)
├── reject-nulls: (1-4)
├── interesting orderings: (+1) (-3,+4,+1) (+7)
Expand Down Expand Up @@ -1372,8 +1372,8 @@ right-join (merge)
├── columns: x:1(int) y:2(int) s:3(string) d:4(decimal) u:5(int) v:6(int!null) rowid:7(int!null)
├── left ordering: +1
├── right ordering: +5
├── key: (1,7)
├── fd: (1)-->(2-4), (3,4)~~>(1,2), (7)-->(5,6)
├── key: (7)
├── fd: (1)-->(2-4), (3,4)~~>(1,2), (7)-->(1-6)
├── scan xysd
│ ├── columns: x:1(int!null) y:2(int) s:3(string) d:4(decimal!null)
│ ├── key: (1)
Expand Down
24 changes: 12 additions & 12 deletions pkg/sql/opt/memo/testdata/logprops/upsert
View file
Original file line number Diff line number Diff line change
Expand Up @@ -269,8 +269,8 @@ project
│ │ │ │ ├── left-join (hash)
│ │ │ │ │ ├── columns: x:5(int!null) y:6(int) column8:8(int) column9:9(int) a:14(int) b:15(int) c:16(int) rowid:17(int)
│ │ │ │ │ ├── side-effects
│ │ │ │ │ ├── key: (5,17)
│ │ │ │ │ ├── fd: (5)-->(6,8), (6)-->(9), (8)~~>(5,6,9), (17)-->(14-16), (14)-->(15-17), (15,16)~~>(14,17)
│ │ │ │ │ ├── key: (5)
│ │ │ │ │ ├── fd: (5)-->(6,8,14-17), (6)-->(9), (8)~~>(5,6,9), (17)-->(14-16), (14)-->(15-17), (15,16)~~>(14,17)
│ │ │ │ │ ├── prune: (15-17)
│ │ │ │ │ ├── reject-nulls: (14-17)
│ │ │ │ │ ├── interesting orderings: (+17) (+14) (+15,+16,+17)
Expand All @@ -297,8 +297,8 @@ project
│ │ │ │ │ │ │ ├── left-join (hash)
│ │ │ │ │ │ │ │ ├── columns: x:5(int!null) y:6(int) column8:8(int) column9:9(int) a:10(int) b:11(int) c:12(int) rowid:13(int)
│ │ │ │ │ │ │ │ ├── side-effects
│ │ │ │ │ │ │ │ ├── key: (5,13)
│ │ │ │ │ │ │ │ ├── fd: (5)-->(6,8), (6)-->(9), (13)-->(10-12), (10)-->(11-13), (11,12)~~>(10,13)
│ │ │ │ │ │ │ │ ├── key: (5)
│ │ │ │ │ │ │ │ ├── fd: (5)-->(6,8,10-13), (6)-->(9), (13)-->(10-12), (10)-->(11-13), (11,12)~~>(10,13)
│ │ │ │ │ │ │ │ ├── prune: (5,6,9-12)
│ │ │ │ │ │ │ │ ├── reject-nulls: (10-13)
│ │ │ │ │ │ │ │ ├── interesting orderings: (+5) (+6) (+13) (+10) (+11,+12,+13)
Expand Down Expand Up @@ -585,32 +585,32 @@ upsert abc
└── project
├── columns: upsert_a:17(int) upsert_b:18(int!null) upsert_c:19(int!null) upsert_rowid:20(int) y:6(int!null) column8:8(int!null) column9:9(int) column10:10(int!null) a:11(int) b:12(int) c:13(int) rowid:14(int) b_new:15(int!null) column16:16(int!null)
├── side-effects
├── key: (6,14)
├── fd: ()-->(8,10,15,16), (6)-->(9), (14)-->(11-13), (11)-->(12-14), (12,13)~~>(11,14), (6,11,14)-->(17), (14)-->(18), (14)-->(19), (9,14)-->(20)
├── key: (6)
├── fd: ()-->(8,10,15,16), (6)-->(9,11-14), (14)-->(11-13), (11)-->(12-14), (12,13)~~>(11,14), (6,11,14)-->(17), (14)-->(18), (14)-->(19), (9,14)-->(20)
├── prune: (6,8-20)
├── reject-nulls: (11-14)
├── interesting orderings: (+14) (+11) (+12,+13,+14)
├── project
│ ├── columns: column16:16(int!null) y:6(int!null) column8:8(int!null) column9:9(int) column10:10(int!null) a:11(int) b:12(int) c:13(int) rowid:14(int) b_new:15(int!null)
│ ├── side-effects
│ ├── key: (6,14)
│ ├── fd: ()-->(8,10,15,16), (6)-->(9), (14)-->(11-13), (11)-->(12-14), (12,13)~~>(11,14)
│ ├── key: (6)
│ ├── fd: ()-->(8,10,15,16), (6)-->(9,11-14), (14)-->(11-13), (11)-->(12-14), (12,13)~~>(11,14)
│ ├── prune: (6,8-16)
│ ├── reject-nulls: (11-14)
│ ├── interesting orderings: (+14) (+11) (+12,+13,+14)
│ ├── project
│ │ ├── columns: b_new:15(int!null) y:6(int!null) column8:8(int!null) column9:9(int) column10:10(int!null) a:11(int) b:12(int) c:13(int) rowid:14(int)
│ │ ├── side-effects
│ │ ├── key: (6,14)
│ │ ├── fd: ()-->(8,10,15), (6)-->(9), (14)-->(11-13), (11)-->(12-14), (12,13)~~>(11,14)
│ │ ├── key: (6)
│ │ ├── fd: ()-->(8,10,15), (6)-->(9,11-14), (14)-->(11-13), (11)-->(12-14), (12,13)~~>(11,14)
│ │ ├── prune: (6,8-15)
│ │ ├── reject-nulls: (11-14)
│ │ ├── interesting orderings: (+14) (+11) (+12,+13,+14)
│ │ ├── left-join (hash)
│ │ │ ├── columns: y:6(int!null) column8:8(int!null) column9:9(int) column10:10(int!null) a:11(int) b:12(int) c:13(int) rowid:14(int)
│ │ │ ├── side-effects
│ │ │ ├── key: (6,14)
│ │ │ ├── fd: ()-->(8,10), (6)-->(9), (14)-->(11-13), (11)-->(12-14), (12,13)~~>(11,14)
│ │ │ ├── key: (6)
│ │ │ ├── fd: ()-->(8,10), (6)-->(9,11-14), (14)-->(11-13), (11)-->(12-14), (12,13)~~>(11,14)
│ │ │ ├── prune: (12-14)
│ │ │ ├── reject-nulls: (11-14)
│ │ │ ├── interesting orderings: (+14) (+11) (+12,+13,+14)
Expand Down
8 changes: 4 additions & 4 deletions pkg/sql/opt/memo/testdata/stats/join
View file
Original file line number Diff line number Diff line change
Expand Up @@ -138,8 +138,8 @@ SELECT *, rowid FROM xysd RIGHT JOIN uv ON x=u
right-join (hash)
├── columns: x:1(int) y:2(int) s:3(string) d:4(decimal) u:5(int) v:6(int!null) rowid:7(int!null)
├── stats: [rows=10000, distinct(1)=500, null(1)=0]
├── key: (1,7)
├── fd: (1)-->(2-4), (3,4)~~>(1,2), (7)-->(5,6)
├── key: (7)
├── fd: (1)-->(2-4), (3,4)~~>(1,2), (7)-->(1-6)
├── scan xysd
│ ├── columns: x:1(int!null) y:2(int) s:3(string) d:4(decimal!null)
│ ├── stats: [rows=5000, distinct(1)=5000, null(1)=0]
Expand Down Expand Up @@ -719,8 +719,8 @@ SELECT *, rowid FROM xysd RIGHT JOIN uv ON x=u
right-join (hash)
├── columns: x:1(int) y:2(int) s:3(string) d:4(decimal) u:5(int) v:6(int!null) rowid:7(int!null)
├── stats: [rows=10000, distinct(1)=500, null(1)=0, distinct(2)=400, null(2)=5000, distinct(3)=499.999999, null(3)=100, distinct(5)=500, null(5)=5000, distinct(2,3)=4323.45892, null(2,3)=50, distinct(3,5)=10000, null(3,5)=50, distinct(1,2,7)=10000, null(1,2,7)=0]
├── key: (1,7)
├── fd: (1)-->(2-4), (3,4)~~>(1,2), (7)-->(5,6)
├── key: (7)
├── fd: (1)-->(2-4), (3,4)~~>(1,2), (7)-->(1-6)
├── scan xysd
│ ├── columns: x:1(int!null) y:2(int) s:3(string) d:4(decimal!null)
│ ├── stats: [rows=5000, distinct(1)=5000, null(1)=0, distinct(2)=400, null(2)=2500, distinct(3)=500, null(3)=50, distinct(1,2)=5000, null(1,2)=0, distinct(2,3)=5000, null(2,3)=25]
Expand Down
Loading

0 comments on commit 1bd13e7

Please sign in to comment.