kvserver: Update snapshot strategy for disaggregated ingests

[itsbilal]

The kvBatchSnapshotStrategy needs to be updated to recognize where the sending/recipient nodes are both using the same shared.Storage, and if so, use the new skip-shared iteration mode in db.ScanInternal to produce a list of shared sstable metadatas as well as some flat non-shared sstables that sit on top of them.

On the recipient side, these shared sstable metadata structs can be ingested into Pebble using the new IngestAndExcise command (see cockroachdb/pebble#2520 , which this issue has a dependency on).

Jira issue: CRDB-27800

[itsbilal]

As mentioned in cockroachdb/pebble#2538, we should call the new IngestAndExcise even if there are no shared files, to avoid the need to write a range tombstone.

[sumeerbhola]

Do we have a list of all the users of Pebble snapshots and how they will be affected by excise removing things under them? Copying some text from cockroachdb/pebble#2424 where we started listing these and also discussing whether the use of Pebble snapshots was necessary.

Pebble snapshots will only be valid if we have not excised a keyspan from the LSM, which we will be doing when dropping a CockroachDB range. So the current uses of Pebble snapshots will need to ensure that the CockroachDB range is still present.

• Sending range snapshots will be cheaper since most of the data is sitting in shared sstables, and does not need to be iterated over, hence using a Pebble snapshot may be unnecessary (more in db: efficient skipping of points deleted by RANGEDEL in the same file pebble#2424 (comment),
• ... mvccGCQueue: does it need a consistent state using a snapshot, or can afford to use multiple iterators when scanning a range.
• snapshots related to the TSDB:

  cockroach/pkg/kv/kvserver/ts_maintenance_queue.go

  Lines 151 to 156 in ee9831d

  	snap := repl.store.TODOEngine().NewSnapshot()
  	now := repl.store.Clock().Now()
  	defer snap.Close()
  	if err := q.tsData.MaintainTimeSeries(
  	ctx, snap, desc.StartKey, desc.EndKey, q.db, q.mem, TimeSeriesMaintenanceMemoryBudget, now,
  	); err != nil {

  . That use of a snapshot looks wholly unnecessary. (*ts.DB).MaintainTimeSeries passes it into (*ts.DB).findTimeSeries, which uses it once to create an iterator to find the names of the time series.

[itsbilal]

@sumeerbhola Thanks for linking that! Looks like the TS maintenance queue no longer uses a snapshot. The use of a snapshot to send a KV snapshot is likely okay for the purposes of this issue as we are guaranteed that we won't be clearing the replica until that replication is complete.

The only other use of a snapshot I can see is the GC queue. I couldn't find anything else from an audit of the code.