Skip to content

Conversation

@simolus3
Copy link
Contributor

#331 introduced SqliteInputValue and the CustomSqliteValue class to represent values that may have to be synced differently depending on compatibility options. The original idea was that each sync stream and SQL bucket or event descriptor would apply their compatibility options when mapping the row, so thatCustomSqliteValues are temporary instances living between the replicator and sync rules.

That approach is flawed in two ways:

  1. It was based on the assumption that different bucket descriptors may have different compatibility options, which is no longer the case (there's one global compatibility context for sync rules). So applying the mapping at the level of the individual stream or sync rule can be inefficient.
  2. Worse, because we may be trying to store rows to recover from TOAST values in Postgres, we would attempt to serialize custom JS instances, which obviously fails.

To fix this, this moves the responsibility of applying the compatibility context to the batch implementation (which has all the necessary data because the compatibility context is known via the sync rules instance). That also means that we'll store mapped representations instead of the original values.

@changeset-bot
Copy link

changeset-bot bot commented Aug 28, 2025

⚠️ No Changeset found

Latest commit: e2e00d3

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@simolus3 simolus3 requested a review from rkistner August 28, 2025 08:13
@simolus3 simolus3 merged commit d5c4691 into main Aug 28, 2025
21 checks passed
@simolus3 simolus3 deleted the dont-store-input-values branch August 28, 2025 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants