-
Notifications
You must be signed in to change notification settings - Fork 4.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
#23949 Remove check() from read() method for sources Cockroach DB and Postgres #24000
Conversation
/test connector=connectors/source-postgres
Build FailedTest summary info:
|
Affected Connector ReportNOTE
|
Connector | Version | Changelog | Publish |
---|---|---|---|
source-alloydb |
2.0.2 |
✅ | ✅ |
source-alloydb-strict-encrypt |
2.0.2 |
🔵 (ignored) |
🔵 (ignored) |
source-cockroachdb |
0.1.21 |
✅ | ✅ |
source-cockroachdb-strict-encrypt |
0.1.21 |
🔵 (ignored) |
🔵 (ignored) |
source-postgres-strict-encrypt |
2.0.3 |
🔵 (ignored) |
🔵 (ignored) |
- See "Actionable Items" below for how to resolve warnings and errors.
✅ Destinations (0)
Connector | Version | Changelog | Publish |
---|
- See "Actionable Items" below for how to resolve warnings and errors.
✅ Other Modules (0)
Actionable Items
(click to expand)
Category | Status | Actionable Item |
---|---|---|
Version | ❌ mismatch |
The version of the connector is different from its normal variant. Please bump the version of the connector. |
⚠ doc not found |
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug. |
|
Changelog | ⚠ doc not found |
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug. |
❌ changelog missing |
There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog. | |
Publish | ⚠ not in seed |
The connector is not in the seed file (e.g. source_definitions.yaml ), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug. |
❌ diff seed version |
The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the /publish command to publish the latest version. |
/test connector=connectors/source-postgres
Build FailedTest summary info:
|
a597070
to
e92d933
Compare
/test connector=connectors/source-postgres
Build PassedTest summary info:
|
/test connector=connectors/source-cockroachdb
Build PassedTest summary info:
|
Update |
/test connector=connectors/source-postgres
Build PassedTest summary info:
|
/test connector=connectors/source-cockroachdb
Build PassedTest summary info:
|
@@ -95,6 +95,7 @@ Your database user should now be ready for use with Airbyte. | |||
|
|||
| Version | Date | Pull Request | Subject | | |||
|:--------|:-----------| :--- | :--- | | |||
| 0.1.21 | 2022-03-14 | [24000](https://github.com/airbytehq/airbyte/pull/24000) | Removed check method call on read. | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fix year in date :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh good call! On it now.
@@ -396,6 +396,7 @@ The root causes is that the WALs needed for the incremental sync has been remove | |||
|
|||
| Version | Date | Pull Request | Subject | | |||
|:--------|:-----------|:---------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | |||
| 2.0.3 | 2022-03-14 | [24000](https://github.com/airbytehq/airbyte/pull/24000) | Removed check method call on read. | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fix year, i suggest correcting all the ones showing up as 2022 since the start of this year.
connectors.md
Outdated
@@ -78,6 +78,7 @@ | |||
| **Freshdesk** | <img alt="Freshdesk icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config/init/src/main/resources/icons/freshdesk.svg" height="30" height="30"/> | Source | airbyte/source-freshdesk:3.0.2 | generally_available | [link](https://docs.airbyte.com/integrations/sources/freshdesk) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-freshdesk) | <small>`ec4b9503-13cb-48ab-a4ab-6ade4be46567`</small> | | |||
| **Freshsales** | <img alt="Freshsales icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config/init/src/main/resources/icons/freshsales.svg" height="30" height="30"/> | Source | airbyte/source-freshsales:0.1.2 | alpha | [link](https://docs.airbyte.com/integrations/sources/freshsales) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-freshsales) | <small>`eca08d79-7b92-4065-b7f3-79c14836ebe7`</small> | | |||
| **Freshservice** | <img alt="Freshservice icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config/init/src/main/resources/icons/freshservice.svg" height="30" height="30"/> | Source | airbyte/source-freshservice:0.1.1 | alpha | [link](https://docs.airbyte.com/integrations/sources/freshservice) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-freshservice) | <small>`9bb85338-ea95-4c93-b267-6be89125b267`</small> | | |||
| **GCS** | <img alt="GCS icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config/init/src/main/resources/icons/gcs.svg" height="30" height="30"/> | Source | airbyte/source-gcs:0.1.0 | unknown | [link](https://docs.airbyte.com/integrations/sources/gcs) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-gcs) | <small>`2a8c41ae-8c23-4be0-a73f-2ab10ca1a820`</small> | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
GCS was unaffected by this PR, yet it shows up here. I believe this happens when the feature branch is out of date of master. Once you merge this section should be removed. I'd double check before merging the PR in.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep! Merging in master fixed it.
@@ -99,11 +99,6 @@ public AutoCloseableIterator<AirbyteMessage> read(final JsonNode config, | |||
final ConfiguredAirbyteCatalog catalog, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can just rid of the read()
method override here and below in general, since all it's doing is delegating to the parent class
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, just a quick comment!
/test connector=connectors/source-cockroachdb
Build PassedTest summary info:
|
/test connector=connectors/source-postgres
Build PassedTest summary info:
|
/publish connector=connectors/source-postgres run-tests=false
if you have connectors that successfully published but failed definition generation, follow step 4 here |
/publish connector=connectors/source-postgres-strict-encrypt run-tests=false
if you have connectors that successfully published but failed definition generation, follow step 4 here |
/publish connector=connectors/source-cockroachdb run-tests=false if you have connectors that successfully published but failed definition generation, follow step 4 here if you have connectors that successfully published but failed definition generation, follow step 4 here |
/publish connector=connectors/source-cockroachdb-strict-encrypt run-tests=false
if you have connectors that successfully published but failed definition generation, follow step 4 here |
/publish connector=connectors/source-cockroachdb run-tests=false
if you have connectors that successfully published but failed definition generation, follow step 4 here |
…ch DB and Postgres (airbytehq#24000) * Removed extraneous read on the check() method for CockroachDB and Postgres * Bumped dockerfile version for relevant conenctors * Bumped versions for strick-encrypt tests * Removed withoutPublication and withoutReplicationSlot tests for postgres cdc * Automated Change * Updated sources docs * Update incorrect year in change logs doc for Cockroach and Postgres * Removed read override method from cockroachDB * Removed read method override for source-postgres * auto-bump connector version --------- Co-authored-by: nguyenaiden <nguyenaiden@users.noreply.github.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
… Postgres (#24000) * Removed extraneous read on the check() method for CockroachDB and Postgres * Bumped dockerfile version for relevant conenctors * Bumped versions for strick-encrypt tests * Removed withoutPublication and withoutReplicationSlot tests for postgres cdc * Automated Change * Updated sources docs * Update incorrect year in change logs doc for Cockroach and Postgres * Removed read override method from cockroachDB * Removed read method override for source-postgres * auto-bump connector version --------- Co-authored-by: nguyenaiden <nguyenaiden@users.noreply.github.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
… Postgres (#24000) * Removed extraneous read on the check() method for CockroachDB and Postgres * Bumped dockerfile version for relevant conenctors * Bumped versions for strick-encrypt tests * Removed withoutPublication and withoutReplicationSlot tests for postgres cdc * Automated Change * Updated sources docs * Update incorrect year in change logs doc for Cockroach and Postgres * Removed read override method from cockroachDB * Removed read method override for source-postgres * auto-bump connector version --------- Co-authored-by: nguyenaiden <nguyenaiden@users.noreply.github.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
What
Currently, we call
check()
during theread()
method for sources CockroachDB and Postgres. This forces a interdependency between protocol methods.Issue: https://app.zenhub.com/workspaces/db--dw-source-connectors-6333360e0a41155061efbcbd/issues/gh/airbytehq/airbyte/23949
Solution:
testReadWithoutPublication
andtestReadWithoutReplicationSlot
inCdcPostgresSourceTest
since they were related to the check() call. These are now redundant since we already havetestCheckWithoutReplicationSlot
andtestCheckWithoutPublication