schemachanger: refactor subzone config elements

[annrpom]

config/zonepb: add subzoneSpan-related methods

This patch adds subzoneSpan-related methods that aid in configuring
subzone zcs in the declarative schemachanger.

Epic: none

Release note: None

---

schemachanger: extend Catalog interface to support subzone writes

This patch extends the Catalog interface to support getting and
writing a zone config. This is necessary for us to be able to get
the parent zone config, apply all subzone changes, and then write
to kv in immediateState.

Epic: none

Release note: None

---

schemachanger: refactor subzone config elements

This patch refactors how subzoneSpans are handled in the declarative schema changer. We now filter out subzoneSpans that do not relate to the index/partition zone config element it is nested within. This makes discarding these subzone elements feasible.

In order to support zone config changes where the target's subzone spans -- along with other targets' subzone spans -- are modified, we make those changes a side-effect. That is:

...
ALTER PARTITION default OF TABLE db.person CONFIGURE ZONE USING gc.ttlseconds = 1;
-> PartitionZoneConfig{default}

ALTER PARTITION australia OF TABLE db.person CONFIGURE ZONE USING gc.ttlseconds = 2;
-> PartitionZoneConfig{default}
and
-> PartitionZoneConfig{australia}

Informs: #133158
Epic: none

Release note: None

---

schemachanger: refactor add/drop zc elements

This patch separates marking zc elements as dropped from
the add direction in the builder as subzone configs will have
a mixed state of drop subzone + add side effects.

Epic: none

Release note: None

[cockroach-teamcity]

This change is 

[annrpom]

TODO

• sort spans correctly in updatesubzoneconfig
• gather all subzone spans related to an idx/partition (forgot that it's not a 1:1 relation)

[annrpom]

woof ok -- so this is almost there. the problem now is from a situation like

cockroach/pkg/ccl/spanconfigccl/spanconfigsqltranslatorccl/testdata/partitions

Lines 2 to 39 in f2f059d

	CREATE DATABASE db;
	CREATE TABLE db.person (
	name STRING,
	country STRING,
	birth_date DATE,
	PRIMARY KEY (country, birth_date, name)
	)
	PARTITION BY LIST (country) (
	PARTITION australia
	VALUES IN ('AU', 'NZ')
	PARTITION BY RANGE (birth_date)
	(
	PARTITION old_au VALUES FROM (minvalue) TO ('1995-01-01'),
	PARTITION yung_au VALUES FROM ('1995-01-01') TO (maxvalue)
	),
	PARTITION north_america
	VALUES IN ('US', 'CA')
	PARTITION BY RANGE (birth_date)
	(
	PARTITION old_na VALUES FROM (minvalue) TO ('1995-01-01'),
	PARTITION yung_na VALUES FROM ('1995-01-01') TO (maxvalue)
	),
	PARTITION default
	VALUES IN (default)
	);
	----
	translate database=db table=person
	----
	/Table/10{6-7} range default
	exec-sql
	ALTER PARTITION default OF TABLE db.person CONFIGURE ZONE USING gc.ttlseconds = 1;
	----
	exec-sql
	ALTER PARTITION australia OF TABLE db.person CONFIGURE ZONE USING gc.ttlseconds = 2;
	----

where just updating australia's subzone spans is not going to work since default's subzone spans will need to get reconfigured to cover the keyspace(?) that australia doesnt cover; we need to add support to configure pre-existing subzone configs' subzone spans as a side effect

so:

ALTER PARTITION default OF TABLE db.person CONFIGURE ZONE USING gc.ttlseconds = 1; will have

• PartitionZoneConfig{default}

ALTER PARTITION australia OF TABLE db.person CONFIGURE ZONE USING gc.ttlseconds = 2; will have

• PartitionZoneConfig{australia}
  and
• PartitionZoneConfig{default} (just to update its subzone spans accordingly)

or something of the sort

[blathers-crl]

Your pull request contains more than 1000 changes. It is strongly encouraged to split big PRs into smaller chunks.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}

[annrpom]

arg i tried to do an optimization https://gist.github.com/annrpom/8a5012562c8e0682b8f995e6e3eb46c8

and it's ok, but not in transactions for some reason -- i left a TODO and put ^ gist up as a future task

[fqazi]

Great work @annrpom, this is coming along nicely.

Reviewed 6 of 14 files at r1, 23 of 23 files at r2, all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @annrpom)

---

pkg/sql/schemachanger/scbuild/internal/scbuildstmt/zone_config_helpers.go line 1220 at r2 (raw file):

// for a lookup of which subzone spans a particular subzone is referred to by.
func getAffectedSubzoneSpansWithIdx(
	subzonesSlice int, newSubzoneSpans []zonepb.SubzoneSpan,

Nit: maybe call subzoneSlices something like numSubZones (or something more descriptive)

---

pkg/sql/schemachanger/scexec/exec_immediate_mutation.go line 239 at r2 (raw file):

	for _, zcToUpdate := range s.modifiedZoneConfigs {
		if zcToUpdate.isSubzoneConfig {
			// TODO(before merge): unsatisfied with this; would it be better to

Yeah it might be cleaner to eliminate the overload

[annrpom]

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @fqazi)

---

pkg/sql/schemachanger/scbuild/internal/scbuildstmt/zone_config_helpers.go line 1220 at r2 (raw file):

Previously, fqazi (Faizan Qazi) wrote…

Nit: maybe call subzoneSlices something like numSubZones (or something more descriptive)

done -- the naming of the method was also a remnant of something i tried doing that didn't pan out; so, i changed that as well

---

pkg/sql/schemachanger/scexec/exec_immediate_mutation.go line 239 at r2 (raw file):

Previously, fqazi (Faizan Qazi) wrote…

Yeah it might be cleaner to eliminate the overload

done oki

[fqazi]

This look great @annrpom. Some very minor comments, but all of this much neater.

Reviewed 29 of 29 files at r4, 29 of 29 files at r5, 7 of 7 files at r6, all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @annrpom)

---

pkg/sql/schemachanger/scbuild/internal/scbuildstmt/configure_zone.go line 91 at r6 (raw file):

			return
		}
		toDrop, sideEffects := zco.getZoneConfigElemForDrop(b)

Nit: Use a more informative name here for sideEffects

---

pkg/sql/schemachanger/scbuild/internal/scbuildstmt/partition_zone_config.go line 104 at r4 (raw file):

		zc, err = pzo.getInheritedDefaultZoneConfig(b)
	} else {
		//zc, err = pzo.tableZoneConfigObj.getZoneConfig(b, false /* inheritDefaultRange */)

Nit: Clean up

---

pkg/config/zonepb/zone.go line 1186 at r4 (raw file):

// DeleteSubzoneSpans deletes all given SubzoneSpan from z.
func (z *ZoneConfig) DeleteSubzoneSpans(spansToDelete []SubzoneSpan) {
	toDeleteMap := make(map[string]bool, len(spansToDelete))

Nit: You could just make map[string]struct{}

[annrpom]

as discussed (offline), I added 1 more change: we now gather all subzone config changes on a table zone config before writing it to kv. this prevents the span config watcher from observing partial states of subzone configurations (in particular: partial states of subzone span changes), which could lead to a panic in the span config translator

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @fqazi)

---

pkg/config/zonepb/zone.go line 1186 at r4 (raw file):

Previously, fqazi (Faizan Qazi) wrote…

Nit: You could just make map[string]struct{}

done oop yeah save mem; ty

---

pkg/sql/schemachanger/scbuild/internal/scbuildstmt/configure_zone.go line 91 at r6 (raw file):

Previously, fqazi (Faizan Qazi) wrote…

Nit: Use a more informative name here for sideEffects

done -- added a blurb as well

---

pkg/sql/schemachanger/scbuild/internal/scbuildstmt/partition_zone_config.go line 104 at r4 (raw file):

Previously, fqazi (Faizan Qazi) wrote…

Nit: Clean up

done oop ty

[annrpom]

TFTR! ('-')7

bors r+

[craig]

Build succeeded:

• windows_build
• examples_orms
• linux_amd64_build
• local_roachtest
• check_generated_code
• linux_arm64_build
• macos_arm64_build
• local_roachtest_fips
• acceptance
• macos_amd64_build
• lint
• linux_amd64_fips_build
• docker_image_amd64
• unit_tests