Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Source postgres: Add SSL certificates to source postgres #13840

Merged
merged 42 commits into from
Jul 18, 2022
Merged

Source postgres: Add SSL certificates to source postgres #13840

merged 42 commits into from
Jul 18, 2022

Conversation

andriikorotkov
Copy link
Contributor

What

Add SSL certificates to source and destination postgres

How

2022-06-16_12-51

Recommended reading order

  1. PostgresSource.java
  2. PostgresDestination.java

🚨 User Impact 🚨

No impact

Pre-merge Checklist

Expand the relevant checklist and delete the others.

New Connector

Community member or Airbyter

  • Community member? Grant edit access to maintainers (instructions)
  • Secrets in the connector's spec are annotated with airbyte_secret
  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • Code reviews completed
  • Documentation updated
    • Connector's README.md
    • Connector's bootstrap.md. See description and examples
    • docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
    • docs/integrations/README.md
    • airbyte-integrations/builds.md
  • PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • If new credentials are required for use in CI, add them to GSM. Instructions.
  • /test connector=connectors/<name> command is passing
  • New Connector version released on Dockerhub by running the /publish command described here
  • After the connector is published, connector added to connector index as described here
  • Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here
Updating a connector

Community member or Airbyter

  • Grant edit access to maintainers (instructions)
  • Secrets in the connector's spec are annotated with airbyte_secret
  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • Code reviews completed
  • Documentation updated
    • Connector's README.md
    • Connector's bootstrap.md. See description and examples
    • Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
  • PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • If new credentials are required for use in CI, add them to GSM. Instructions.
  • /test connector=connectors/<name> command is passing
  • New Connector version released on Dockerhub and connector version bumped by running the /publish command described here
Connector Generator
  • Issue acceptance criteria met
  • PR name follows PR naming conventions
  • If adding a new generator, add it to the list of scaffold modules being tested
  • The generator test modules (all connectors with -scaffold in their name) have been updated with the latest scaffold by running ./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates then checking in your changes
  • Documentation which references the generator is updated as needed

Tests

Unit

Put your unit tests output here.

Integration

Put your integration tests output here.

Acceptance

Put your acceptance tests output here.

@github-actions github-actions bot added the area/connectors Connector related issues label Jun 16, 2022
@alexandr-shegeda alexandr-shegeda linked an issue Jun 16, 2022 that may be closed by this pull request
Support Postgres SSL connection #4034
Closed
@alexandr-shegeda alexandr-shegeda marked this pull request as ready for review June 16, 2022 13:36
@andriikorotkov
Copy link
Contributor Author

andriikorotkov commented Jul 7, 2022 •
edited by github-actions bot
Loading

/test connector=connectors/source-postgres

🕑 connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/2630492445
✅ connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/2630492445
No Python unittests run

Build Passed

Test summary info:

All Passed

@andriikorotkov andriikorotkov temporarily deployed to more-secrets July 7, 2022 15:07 Inactive
@andriikorotkov andriikorotkov temporarily deployed to more-secrets July 8, 2022 07:24 Inactive
@andriikorotkov andriikorotkov temporarily deployed to more-secrets July 8, 2022 08:25 Inactive
@andriikorotkov
Copy link
Contributor Author

andriikorotkov commented Jul 8, 2022 •
edited by github-actions bot
Loading

/test connector=connectors/source-postgres

🕑 connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/2635035840
✅ connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/2635035840
No Python unittests run

Build Passed

Test summary info:

All Passed

@grishick
Copy link
Contributor

grishick commented Jul 8, 2022

Changes look good. However, to be safe, we should close this issue before publishing and merging this.

@grishick grishick mentioned this pull request Jul 8, 2022
Closed
4 tasks
@andriikorotkov andriikorotkov linked an issue Jul 14, 2022 that may be closed by this pull request
Update acceptance tests for Postgres source/destination with SSL certificates #12766
Closed
grishick
grishick approved these changes Jul 16, 2022
View reviewed changes
Copy link
Contributor

@grishick grishick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please rebase before publishing and merging

@andriikorotkov
Merge branch 'master' of github.com:airbytehq/airbyte into akorotkov/…
4c2225b 
…4034_ssl_certificate_postgres

� Conflicts:
�	airbyte-integrations/connectors/source-postgres-strict-encrypt/Dockerfile
�	airbyte-integrations/connectors/source-postgres/Dockerfile
�	airbyte-integrations/connectors/source-postgres/src/main/java/io/airbyte/integrations/source/postgres/PostgresSource.java
�	docs/integrations/sources/postgres.md
@andriikorotkov andriikorotkov requested a review from a team as a code owner July 17, 2022 19:33
@andriikorotkov
Copy link
Contributor Author

andriikorotkov commented Jul 17, 2022 •
edited by github-actions bot
Loading

/test connector=connectors/source-postgres

🕑 connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/2686654237
✅ connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/2686654237
No Python unittests run

Build Passed

Test summary info:

All Passed

@andriikorotkov andriikorotkov temporarily deployed to more-secrets July 17, 2022 19:36 Inactive
@andriikorotkov
Copy link
Contributor Author

andriikorotkov commented Jul 18, 2022 •
edited by github-actions bot
Loading

/publish connector=connectors/source-postgres

🕑 Publishing the following connectors:
connectors/source-postgres
https://github.com/airbytehq/airbyte/actions/runs/2688764392


Connector Did it publish? Were definitions generated?
connectors/source-postgres ✅ ✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@octavia-squidington-iii octavia-squidington-iii temporarily deployed to more-secrets July 18, 2022 08:39 Inactive
@andriikorotkov
Copy link
Contributor Author

andriikorotkov commented Jul 18, 2022 •
edited by github-actions bot
Loading

/publish connector=connectors/source-postgres-strict-encrypt auto-bump-version=false

🕑 Publishing the following connectors:
connectors/source-postgres-strict-encrypt
https://github.com/airbytehq/airbyte/actions/runs/2689220567


Connector Did it publish? Were definitions generated?
connectors/source-postgres-strict-encrypt ❌ ❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@andriikorotkov
Copy link
Contributor Author

andriikorotkov commented Jul 18, 2022 •
edited by github-actions bot
Loading

/publish connector=connectors/source-postgres-strict-encrypt

🕑 Publishing the following connectors:
connectors/source-postgres-strict-encrypt
https://github.com/airbytehq/airbyte/actions/runs/2689278331


Connector Did it publish? Were definitions generated?
connectors/source-postgres-strict-encrypt ❌ ❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@andriikorotkov
Copy link
Contributor Author

andriikorotkov commented Jul 18, 2022 •
edited by github-actions bot
Loading

/publish connector=connectors/source-postgres-strict-encrypt auto-bump-version=false

🕑 Publishing the following connectors:
connectors/source-postgres-strict-encrypt
https://github.com/airbytehq/airbyte/actions/runs/2689407971


Connector Did it publish? Were definitions generated?
connectors/source-postgres-strict-encrypt ❌ ❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@andriikorotkov
Copy link
Contributor Author

andriikorotkov commented Jul 18, 2022 •
edited by github-actions bot
Loading

/publish connector=connectors/source-postgres-strict-encrypt auto-bump-version=false

🕑 Publishing the following connectors:
connectors/source-postgres-strict-encrypt
https://github.com/airbytehq/airbyte/actions/runs/2689528824


Connector Did it publish? Were definitions generated?
connectors/source-postgres-strict-encrypt ✅ ❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@andriikorotkov andriikorotkov temporarily deployed to more-secrets July 18, 2022 09:35 Inactive
@andriikorotkov andriikorotkov merged commit 2dc4759 into master Jul 18, 2022
@andriikorotkov andriikorotkov deleted the akorotkov/4034_ssl_certificate_postgres branch July 18, 2022 09:58
@ryankfu ryankfu mentioned this pull request Jul 18, 2022
Closed
37 tasks
@VitaliiMaltsev
Copy link
Contributor

#14574 was merged. Please proceed

@octavia-squidington-iii octavia-squidington-iii mentioned this pull request Jul 23, 2022
Merged
@subodh1810 subodh1810 mentioned this pull request Aug 1, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mkhokh-33 mkhokh-33 mkhokh-33 approved these changes

@grishick grishick grishick approved these changes

@suhomud suhomud Awaiting requested review from suhomud

@alexandr-shegeda alexandr-shegeda Awaiting requested review from alexandr-shegeda

@edgao edgao Awaiting requested review from edgao

Assignees
No one assigned
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/destination/postgres connectors/destination/postgres-strict-encrypt connectors/source/postgres connectors/source/postgres-strict-encrypt
Projects
None yet
Milestone
No milestone
7 participants
@andriikorotkov @grishick @edgao @VitaliiMaltsev @alexandr-shegeda @mkhokh-33 @octavia-squidington-iii