Test and release per-stream state handling in Postgres Source

[grishick]

Once platform is ready with per-stream state, we need to manually test

• adding/removing tables to Postgres Source
• connections created with older connector version of Postgres Source (current version in Cloud)
• file any issues that we find

Context:

• Partial reset CDC postgres source acceptance tests: CDC partial reset acceptance test #14551
• Calculate diff to reset streams: Remove withRefreshedCatalog param from updateConnection endpoint #14477
• Per-Stream State Developer Community Announcement
• Per-Stream State Tech Spec

[grishick]

Hey team! Please add your planning poker estimate with ZenHub @edgao @ryankfu @subodh1810

[ryankfu]

UPDATE: this ticket is mostly taken care of by the OSS platform team with acceptance tests from Lake and Benoit. I do need to provide an update to them on the create a connection with an older version of the Postgres source but I believe their tests currently cover that (not fully verified)

I'm going to instead shift this ticket to address the documentation of the logic that was discovered as a result of spectating Benoit and Lake set up a per-stream non-CDC sync locally. I will update the ticket with a living google doc and tag the databases and oss-platform team once it's ready for review. Furthermore, I'm adding in a diagram of how our syncs work prior to per-stream changes and after per-stream through Whimsical

[ryankfu]

OSS platform team merged the per-stream changes today https://github.com/airbytehq/airbyte-cloud/pull/2151 which enables per-stream changes onto Airbyte Cloud that is useful for testing

Additionally, will be documenting any issues with setup and testing within Postgres Source GA doc. Due note that the document is not visible to non-Airbyters while the team continues to work out any refinements in the setup and information sharing process

Platform OSS filed issues related to per-stream state handling:

• Validate only on incremental #14966 - this addresses the issue where previously the validation logic also applied to full-sync streams when that particular mode does not create a state within per-stream

[ryankfu]

Document covering the per-stream changes here with overview of the complications, what is per-stream and protocol changes necessary to get per-stream operational