Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[per-stream cdk] publish all GA API source connectors #13136

Closed
Tracked by #13134 ...
sherifnada opened this issue May 24, 2022 · 4 comments
Closed
Tracked by #13134 ...

[per-stream cdk] publish all GA API source connectors #13136

sherifnada opened this issue May 24, 2022 · 4 comments
Assignees

Comments

@sherifnada
Copy link
Contributor

sherifnada commented May 24, 2022

Once the CDK has been updated to support per-stream state, we need republish all GA API source connectors so that they use the latest version of the CDK.

For each connector:

  • Connectors can opt out of per-stream state by overriding per_stream_state_enabled() and setting it to False. By default, per-stream is enabled for integrations using the latest version of the CDK.
  • Optional (but recommended validation steps):
    • Generate expected_records.txt using the existing connector code
    • Add the file to /integration_tests
    • Update basic_read with expect_records in source-acceptance-tests.yaml
expect_records:
  path: "integration_tests/expected_records.txt"
  • Modify any integration_tests/input_state.json files to use the new STREAM state format
  • Validate Source Acceptance Tests from the PR
  • Publish a new version of the connector
  • Release connector to Airbyte Cloud using the normal process
@brianjlai brianjlai changed the title Python connectors: publish all GA / Beta connectors [per-stream cdk] publish all GA API source connectors Aug 28, 2022
@alafanechere
Copy link
Contributor

alafanechere commented Sep 29, 2022

PRs

#17326
#17304
#17120
#17110
#17410
#17412

Current migration status

All connector migrated in OSS will eventually be released to cloud unless their version is pinned via the masks.

OSS

Connector Release stage Migrated Version
source-amazon-ads generally_available 0.1.22
source-amplitude generally_available 0.1.14
source-bing-ads generally_available 0.1.14
source-chargebee generally_available 0.1.15
source-facebook-marketing generally_available 0.2.66
source-freshdesk generally_available 0.3.6
source-github generally_available 0.3.3
source-google-ads generally_available 0.2.1
source-google-analytics-v4 generally_available 0.1.26
source-google-search-console generally_available 0.1.16
source-google-sheets generally_available 0.2.19
source-greenhouse generally_available 0.2.11
source-harvest generally_available 0.1.11
source-hubspot generally_available 0.2.1
source-instagram generally_available 1.0.0
source-intercom generally_available 0.1.27
source-klaviyo generally_available 0.1.10
source-linkedin-ads generally_available 0.1.10
source-mailchimp generally_available 0.2.15
source-marketo generally_available 0.1.11
source-mixpanel generally_available 0.1.27
source-notion generally_available 0.1.10
source-pinterest generally_available 0.1.7
source-s3 generally_available 0.1.22
source-salesforce generally_available 1.0.20
source-slack generally_available 0.1.18
source-snapchat-marketing generally_available 0.1.8
source-stripe generally_available 0.1.39
source-surveymonkey generally_available 0.1.11
source-tiktok-marketing generally_available 0.1.16
source-twilio generally_available 0.1.11
source-zendesk-chat generally_available 0.1.10
source-zendesk-support generally_available 0.2.16
source-zendesk-talk generally_available 0.1.5
source-sentry generally_available 0.1.7

Cloud

Connector Release stage Migrated
source-amazon-ads generally_available
source-amplitude generally_available
source-bing-ads generally_available
source-chargebee generally_available
source-facebook-marketing generally_available
source-freshdesk generally_available
source-github generally_available
source-google-ads generally_available
source-google-analytics-v4 generally_available
source-google-search-console generally_available
source-google-sheets generally_available
source-greenhouse generally_available
source-harvest generally_available
source-hubspot generally_available
source-instagram generally_available
source-intercom generally_available
source-klaviyo generally_available
source-linkedin-ads generally_available
source-mailchimp generally_available
source-mixpanel generally_available
source-notion generally_available
source-pinterest generally_available
source-s3 generally_available
source-salesforce generally_available
source-sentry generally_available
source-snapchat-marketing generally_available
source-stripe generally_available
source-surveymonkey generally_available
source-tiktok-marketing generally_available
source-twilio generally_available
source-zendesk-chat generally_available
source-zendesk-support generally_available
source-zendesk-talk generally_available

@alafanechere
Copy link
Contributor

SAT is failing for source-snapchat-marketing, I preferred to not migrate it to per-stream once this issue is not fixed:
#17461

@alafanechere
Copy link
Contributor

All GA connectors except source-snapchat-marketing are migrated to per-stream state on OSS. I tried to update OSS reference on cloud but I'm facing an error while running the ./tools/bin/update_oss_reference.

@girarda girarda assigned brianjlai and unassigned alafanechere Oct 6, 2022
@brianjlai
Copy link
Contributor

GL made some recent changes to source-snapchat-marketing which fixed the tests and latest published version and on cloud should be using per-stream. Going to close out this ticket out as done since all the GA connectors should now be running per-stream.

One note is zendesk-support is pinned specifically to 0.2.14 for some reason, but i'll give it a look and if it looks okay i'll unpin it to use the latest in the cloud mask

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants