Merge #55654

55654: xform: reorganize rules and tests for consistency r=RaduBerinde a=mgartner

#### xform: organize rules by the type of expressions they match

This commit organizes exploration rules by the type of expressions they
match. `GenerateZigzagJoins` and `GenerateInvertedZigzagJoins` have been
moved to `select.opt`. All other rules were correctly organized.

Release note: None

#### xform: reorganize tests to match the organization of the rules

This commit moves around (and makes some minor necessary changes) to
exploration tests so that tests for rules in `foo.opt` can be found in
`testdata/foo`. Most of the changes were related to zigzag join rules
which moved from `join` to `select`, and some tests for
`GenerateConstrainedScans` that have lived in `scan` for years.

The goal of this change is to make it easier to find tests for a
specific rule and to set a precedent for future tests.

Release note: None


Co-authored-by: Marcus Gartner <marcus@cockroachlabs.com>

pkg/sql/opt/norm/testdata/rules/combo

@@ -297,15 +297,15 @@ PruneJoinRightCols
 GenerateIndexScans (no changes)
 --------------------------------------------------------------------------------
 --------------------------------------------------------------------------------
-GenerateZigzagJoins (no changes)
---------------------------------------------------------------------------------
---------------------------------------------------------------------------------
 GeneratePartialIndexScans (no changes)
 --------------------------------------------------------------------------------
 --------------------------------------------------------------------------------
 GenerateConstrainedScans (no changes)
 --------------------------------------------------------------------------------
 --------------------------------------------------------------------------------
+GenerateZigzagJoins (no changes)
+--------------------------------------------------------------------------------
+--------------------------------------------------------------------------------
 GenerateIndexScans (no changes)
 --------------------------------------------------------------------------------
 ================================================================================
@@ -583,9 +583,6 @@ GenerateIndexScans
               ├── s:4 = 'foo' [outer=(4), constraints=(/4: [/'foo' - /'foo']; tight), fd=()-->(4)]
               └── f:3 > 100.0 [outer=(3), constraints=(/3: [/100.00000000000001 - ]; tight)]
 --------------------------------------------------------------------------------
-GenerateZigzagJoins (no changes)
---------------------------------------------------------------------------------
---------------------------------------------------------------------------------
 GeneratePartialIndexScans (no changes)
 --------------------------------------------------------------------------------
 ================================================================================
@@ -610,6 +607,9 @@ GenerateConstrainedScans
   -           ├── s:4 = 'foo' [outer=(4), constraints=(/4: [/'foo' - /'foo']; tight), fd=()-->(4)]
   -           └── f:3 > 100.0 [outer=(3), constraints=(/3: [/100.00000000000001 - ]; tight)]
   +      └── fd: ()-->(4), (1)-->(3), (3)-->(1)
+--------------------------------------------------------------------------------
+GenerateZigzagJoins (no changes)
+--------------------------------------------------------------------------------
 ================================================================================
 Final best expression
   Cost: 14.10
@@ -2872,15 +2872,15 @@ GenerateIndexScans (no changes)
 GenerateIndexScans (no changes)
 --------------------------------------------------------------------------------
 --------------------------------------------------------------------------------
-GenerateZigzagJoins (no changes)
---------------------------------------------------------------------------------
---------------------------------------------------------------------------------
 GeneratePartialIndexScans (no changes)
 --------------------------------------------------------------------------------
 --------------------------------------------------------------------------------
 GenerateConstrainedScans (no changes)
 --------------------------------------------------------------------------------
 --------------------------------------------------------------------------------
+GenerateZigzagJoins (no changes)
+--------------------------------------------------------------------------------
+--------------------------------------------------------------------------------
 ReorderJoins (no changes)
 --------------------------------------------------------------------------------
 --------------------------------------------------------------------------------

pkg/sql/opt/testutils/opttester/testdata/explore-trace

@@ -38,7 +38,7 @@ New expression 1 of 1:
              └── const: 1 [type=int]
 
 ================================================================================
-GenerateZigzagJoins
+GeneratePartialIndexScans
 ================================================================================
 Source expression:
   select
@@ -57,7 +57,7 @@ Source expression:
 No new expressions.
 
 ================================================================================
-GeneratePartialIndexScans
+GenerateConstrainedScans
 ================================================================================
 Source expression:
   select
@@ -73,10 +73,15 @@ Source expression:
              ├── variable: b:2 [type=int]
              └── const: 1 [type=int]
 
-No new expressions.
+New expression 1 of 1:
+  scan ab@secondary
+   ├── columns: a:1(int!null) b:2(int!null)
+   ├── constraint: /2/1: [/1 - /1]
+   ├── key: (1)
+   └── fd: ()-->(2)
 
 ================================================================================
-GenerateConstrainedScans
+GenerateZigzagJoins
 ================================================================================
 Source expression:
   select
@@ -92,12 +97,7 @@ Source expression:
              ├── variable: b:2 [type=int]
              └── const: 1 [type=int]
 
-New expression 1 of 1:
-  scan ab@secondary
-   ├── columns: a:1(int!null) b:2(int!null)
-   ├── constraint: /2/1: [/1 - /1]
-   ├── key: (1)
-   └── fd: ()-->(2)
+No new expressions.
 ----
 ----
 

pkg/sql/opt/testutils/opttester/testdata/opt-steps

@@ -143,9 +143,6 @@ GenerateIndexScans (higher cost)
               ├── variable: b:2 [type=int]
               └── const: 1 [type=int]
 --------------------------------------------------------------------------------
-GenerateZigzagJoins (no changes)
---------------------------------------------------------------------------------
---------------------------------------------------------------------------------
 GeneratePartialIndexScans (no changes)
 --------------------------------------------------------------------------------
 ================================================================================
@@ -167,6 +164,9 @@ GenerateConstrainedScans
   -           ├── variable: b:2 [type=int]
   -           └── const: 1 [type=int]
   + └── fd: ()-->(2)
+--------------------------------------------------------------------------------
+GenerateZigzagJoins (no changes)
+--------------------------------------------------------------------------------
 ================================================================================
 Final best expression
   Cost: 14.41
@@ -714,9 +714,6 @@ GenerateIndexScans (higher cost)
               ├── variable: k:1 [type=int]
               └── const: 1 [type=int]
 --------------------------------------------------------------------------------
-GenerateZigzagJoins (no changes)
---------------------------------------------------------------------------------
---------------------------------------------------------------------------------
 GeneratePartialIndexScans (no changes)
 --------------------------------------------------------------------------------
 --------------------------------------------------------------------------------
@@ -746,6 +743,9 @@ FoldComparison
   -           └── const: 1 [type=int]
   + ├── constraint: /2/1/3: [/true/1 - /true/1]
   + └── fd: ()-->(1)
+--------------------------------------------------------------------------------
+GenerateZigzagJoins (no changes)
+--------------------------------------------------------------------------------
 ================================================================================
 Final best expression
   Cost: 14.51

pkg/sql/opt/xform/rules/join.opt

@@ -234,48 +234,6 @@
     $private
 )
 
-# GenerateZigzagJoins creates ZigzagJoin operators for all index pairs (of the
-# Scan table) where the prefix column(s) of both indexes is/are fixed to
-# constant values in the filters. See comments in GenerateZigzagJoin and
-# distsqlrun/zigzagjoiner.go for more details on when a zigzag join can be
-# planned.
-#
-# Zigzag joins are prohibited when the source Scan operator has been configured
-# with a row-level locking mode. This is mostly out of convenience so that these
-# row-level locking modes don't need to added to the ZigzagJoin operator. There
-# doesn't seem to be a strong reason to support this, but if one comes up, it
-# should be possible to lift this restriction.
-[GenerateZigzagJoins, Explore]
-(Select
-    (Scan $scan:*) & (IsCanonicalScan $scan) & ^(IsLocking $scan)
-    $filters:*
-)
-=>
-(GenerateZigzagJoins $scan $filters)
-
-# GenerateInvertedIndexZigzagJoins creates ZigzagJoin operators for inverted
-# indexes that can be constrained with two or more distinct constant values.
-# Inverted indexes contain one row for each path-to-leaf in a JSON value, so one
-# row in the primary index could generate multiple inverted index keys. This
-# property can be exploited by zigzag joining on the same inverted index, fixed
-# at any two of the JSON paths we are querying for.
-#
-# Zigzag joins are prohibited when the source Scan operator has been configured
-# with a row-level locking mode. This is mostly out of convenience so that these
-# row-level locking modes don't need to added to the ZigzagJoin operator. There
-# doesn't seem to be a strong reason to support this, but if one comes up, it
-# should be possible to lift this restriction.
-[GenerateInvertedIndexZigzagJoins, Explore]
-(Select
-    (Scan $scan:*) &
-        (IsCanonicalScan $scan) &
-        ^(IsLocking $scan) &
-        (HasInvertedIndexes $scan)
-    $filters:*
-)
-=>
-(GenerateInvertedIndexZigzagJoins $scan $filters)
-
 # GenerateLookupJoinWithFilter creates a LookupJoin alternative for a Join which
 # has a Select->Scan combination as its right input. The filter can get merged
 # with the ON condition (this is correct for inner, left, and semi/anti join).

pkg/sql/opt/xform/rules/select.opt

@@ -53,6 +53,48 @@
 =>
 (GenerateInvertedIndexScans $scanPrivate $filters)
 
+# GenerateZigzagJoins creates ZigzagJoin operators for all index pairs (of the
+# Scan table) where the prefix column(s) of both indexes is/are fixed to
+# constant values in the filters. See comments in GenerateZigzagJoin and
+# distsqlrun/zigzagjoiner.go for more details on when a zigzag join can be
+# planned.
+#
+# Zigzag joins are prohibited when the source Scan operator has been configured
+# with a row-level locking mode. This is mostly out of convenience so that these
+# row-level locking modes don't need to added to the ZigzagJoin operator. There
+# doesn't seem to be a strong reason to support this, but if one comes up, it
+# should be possible to lift this restriction.
+[GenerateZigzagJoins, Explore]
+(Select
+    (Scan $scan:*) & (IsCanonicalScan $scan) & ^(IsLocking $scan)
+    $filters:*
+)
+=>
+(GenerateZigzagJoins $scan $filters)
+
+# GenerateInvertedIndexZigzagJoins creates ZigzagJoin operators for inverted
+# indexes that can be constrained with two or more distinct constant values.
+# Inverted indexes contain one row for each path-to-leaf in a JSON value, so one
+# row in the primary index could generate multiple inverted index keys. This
+# property can be exploited by zigzag joining on the same inverted index, fixed
+# at any two of the JSON paths we are querying for.
+#
+# Zigzag joins are prohibited when the source Scan operator has been configured
+# with a row-level locking mode. This is mostly out of convenience so that these
+# row-level locking modes don't need to added to the ZigzagJoin operator. There
+# doesn't seem to be a strong reason to support this, but if one comes up, it
+# should be possible to lift this restriction.
+[GenerateInvertedIndexZigzagJoins, Explore]
+(Select
+    (Scan $scan:*) &
+        (IsCanonicalScan $scan) &
+        ^(IsLocking $scan) &
+        (HasInvertedIndexes $scan)
+    $filters:*
+)
+=>
+(GenerateInvertedIndexZigzagJoins $scan $filters)
+
 # SplitDisjunction splits disjunctions (Or expressions) into a Union of two
 # Select expressions, the first containing the left sub-expression of the Or
 # expression and the second containing the right sub-expression. All other