[source-postgres] CdcPostgresSourceLegacyCtidTest isn't testing against postgres 13 #35267
Labels
team/db-dw-sources
Backlog for Database and Data Warehouse Sources team
type/bug
Something isn't working
https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-postgres/src/test/java/io/airbyte/integrations/source/postgres/CdcPostgresSourceLegacyCtidTest.java
After we refactored test infra this does not actually test running on legacy postgres versions. We should update that.
I've tried to update that but encountered failures among unit tests: during the initial scan, lsn state in the last state message is different than the previous state message; this only happens on postgres 13 but not 16.
In the test https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-postgres/src/test/java/io/airbyte/integrations/source/postgres/CdcPostgresSourceTest.java#L199 expects state message stays the same in the same batch. However in postgres 13, the last state message will mutate. See example below:
Previous state message:
"[\"db_dlpenhggyr\",{\"server\":\"db_dlpenhggyr\"}]": "{\"transaction_id\":null,\"lsn\":24834904,\"txId\":678,\"ts_usec\":1707870101598505}"
Final state message generated by
PostgresCdcStateHandler
:"[\"db_dlpenhggyr\",{\"server\":\"db_dlpenhggyr\"}]": "{\"transaction_id\":null,\"lsn_proc\":24867752,\"lsn_commit\":24867752,\"lsn\":24867752,\"txId\":678,\"ts_usec\":1707870101598505}"
Some research would be required to understand why the behavior is different, and determine if we have an underlying bug/data loss potential in this case.
The text was updated successfully, but these errors were encountered: