sync: Improve sync checksum algorithm and endpoints (#17567)

* Initial commit:

Add a `Table_Checksum` class that will contain the logic around checksumming.

Partially completed the query building and querying the database. There's still a lot of work to go through.

* Update the query to be actually built.

Return the results.

[not verified]

* Don't `GROUP BY` on non-granular queries.

[not verified]

* Update table configuration to be more precise and deterministic.

[not verified]

* Another mid-sentence commit. Still WIP.

The full flow for checksumming shold work now.

Reworked some of the old methods to call the new functionality

Removed old code.

[not verified]

* [not verified] Add checksum_fields to term_relationships and term_taxonomy

* [not verified] switched definitions and updated to post_modified_gmt

* [not verified] Addition of initial PHPDocs to be expanded. Update of comments to terminate in .

* [not verified] Fix logic so range edges are included in bucket sums. We now get the same crc sum regardless of # of buckets used. Created a helper function to identify # of buckets to use for a table.
Updated checksum_all to include all tables.
Updated checksum_all to be dynamic buckets to allow for large site calculations.

* [not verified] Update checksum helper functions for posts and postmeta to use dynamic buckets to protect large sites.

* Fix linting issues, hopefully?

* Some smaller changes

[not verified]

* UPdate checksum references to use tables not objects.

* [not verified] Add support for parent type filtering

* [not verified] Initial implementation of Integration Tests. Testing requires core functionality so these aren't unit tests. phpcs fixes

* [not verified] small phpcs updates

* reformatted for earlier versions

* [not verified] Add more advanced filtering and implement parent table relation when trying to calculate checksums

* get_range_edges Integration Tests

* [not verified] Resolve undifined index warning.

* Update Tests to use filter_values vs filter_sql

* [not verified] Generic consistency check on checksum calculation

* [not verified] Smoke Test that checksum is calculated for all tables based on default configs.
Add 1=1 to WHERE clause so we have legit sql if no filters

* Min-Max to helper functions and calculate_buckets.

* Resolve discrepancy where WP_Error is returned before attempting array_sum.

* Mark test_all_checksums_match  as skipped as it is not a valid End-to-End test.

* Fix Comments filtering. Add more proper field prefixing when building filters to make sure field clashes are avoided

* [not verified] checksum_disable setting, default and is_checksum_enabled function. Re-work calculate_buckets to limit usage. Integrate exceptions, errors when disabled.

* Add range query support, add fixes

* PHPCS

* PHPCS linting

* [not verified] More improvements. Disble checksum check due to fatal on WPCOM

* PHPCS lint cleanup

* PHPCS and restore is_checksum_enabled

* Overhaul meta endpoint as we need a object to key mapping per 'object'

* 2 more linting complaints about unused varaibles in tests.

* Remove a placeholder value for the keys

* =Disable Sync from triggering on checksum endpoints.

* Clear out unneeded changes to composer.json

Co-authored-by: Matthew Denton <matt@mdbitz.com>

Committed via a GitHub action: https://github.com/automattic/jetpack/runs/428367064

src/class-defaults.php

@@ -1140,6 +1140,13 @@ public static function is_multi_network() {
 	 */
 	public static $default_network_disable = 0;
 
+	/**
+	 * Default for disabling checksums.
+	 *
+	 * @var int Bool-ish. Default 0.
+	 */
+	public static $default_checksum_disable = 0;
+
 	/**
 	 * Should Sync use cron?
 	 *