-
Notifications
You must be signed in to change notification settings - Fork 4.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[charts/airbyte-cron] Cleanup env vars #18787
[charts/airbyte-cron] Cleanup env vars #18787
Conversation
- name: REMOTE_CONNECTOR_CATALOG_URL | ||
valueFrom: | ||
configMapKeyRef: | ||
name: {{ .Release.Name }}-airbyte-env | ||
key: REMOTE_CONNECTOR_CATALOG_URL | ||
- name: TRACKING_STRATEGY | ||
valueFrom: | ||
configMapKeyRef: | ||
name: {{ .Release.Name }}-airbyte-env | ||
key: TRACKING_STRATEGY | ||
- name: UPDATE_DEFINITIONS_CRON_ENABLED | ||
valueFrom: | ||
configMapKeyRef: | ||
name: {{ .Release.Name }}-airbyte-env | ||
key: UPDATE_DEFINITIONS_CRON_ENABLED |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure about airbyte_role, but for these two (remote_connector_catalog_url and update_definitions_cron_enabled), while they could be set on OSS, they're mainly meant for cloud usage so it's probably ok to remove for now if it's causing an issue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@perangel is there a reason not to add them to the configmap?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see that AIRBYTE_ROLE
is declared in https://github.com/airbytehq/airbyte/blob/master/airbyte-cron/src/main/resources/application.yml#L69. Anything not declared by the ConfigMap can still be provided via env_vars
or global.env_vars
so there's still a means to pass them through. But since these keys are definitely not in the ConfigMap, it should be safe to remove them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@perangel is there a reason not to add them to the configmap?
None that I am aware of, but I would like to get a second opinion since I see cases where other config keys that are declared in microaut apps are also not present in the ConfigMap
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As for me, I've never seen those variables present in OSS deployments, only in cloud one, so if devs are ok with dropping those(meaning that it'll be safe), then we can drop them and test the deployment out.
For this you need to run the following:
mv charts/airbyte/Chart.yaml charts/airbyte/Chart.yaml.old
mv charts/airbyte/Chart.yaml.test charts/airbyte/Chart.yaml
cd charts/airbyte && helm dep update && cd -
helm upgrade --install --debug airbyte charts/airbyte
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since it's deploying and running(at least), LGTM
* master: (59 commits) 🪟🔧 Remove styled components (round 1) (#18766) Bump Airbyte version from 0.40.17 to 0.40.18 (#18827) ci: add job and run id to test reports (#18832) hide ConfigPersistence inside ConfigRepository to discourage use (#18803) Remove the bulk actions from ConfigPersistence (#18800) remove config persistence from seeding logic (#18749) Remove ConfigPersistence usage from SecretsMigrator (#18747) Add normalization changelog and bump normalization version in platform (#18813) 🐛Source Exchange Rates: Fix handling error during check connection (#18726) 🐛Destination Google Sheets: Fix empty headers list (#18729) Bump helm chart version reference to 0.40.40 (#18815) Use equalsIgnoreCase (#18810) [charts/airbyte-cron] Cleanup env vars (#18787) 🐛 Source Facebook Marketing: reduce request limit after specific error (#18734) Parameterize test_empty_streams and test_stream_with_1_airbyte_column by destination (#18197) Correct coinmarket spec (#18790) ci: use custom test-reporter action to upload job results (#18004) Fix unit tests in source relational db (#18789) query to include data plane attributes (#18531) Mark/update notification settings design (#18159) ...
* [charts/airbyte-cron] Cleanup env vars * Remove unused env var
* [charts/airbyte-cron] Cleanup env vars * Remove unused env var
* solve conflicts * solve conflict in json schema * bump to version 0.1.8 for the changes of this pr * change ad account id in the schemas * query to include data plane attributes (#18531) * query to include data plane attributes * rename functions * fix java build * more renaming fix * Fix unit tests in source relational db (#18789) * Fix unit tests * Add extra test case for record count > 1 * Store record count in variable * ci: use custom test-reporter action to upload job results (#18004) * ci: use custom action to upload job results * Correct coinmarket spec (#18790) * correct coinmarket spec * remove duplicate support normalization from source spec * rollback coinmarketcap version in source def seed * update connector version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Parameterize test_empty_streams and test_stream_with_1_airbyte_column by destination (#18197) * Remove lines that always add Postgres to list of destinations * Parameterize all tests in test_ephemeral by destination * 🐛 Source Facebook Marketing: reduce request limit after specific error (#18734) * 🐛 Source Facebook Marketing: reduce request limit after specific error * 🐛 Source Facebook Marketing: bump version; update docs * 🐛 Source Facebook Marketing: add test * 🐛 Source Facebook Marketing: increase timeout * [charts/airbyte-cron] Cleanup env vars (#18787) * [charts/airbyte-cron] Cleanup env vars * Remove unused env var * Use equalsIgnoreCase (#18810) * Bump helm chart version reference to 0.40.40 (#18815) Co-authored-by: perangel <perangel@users.noreply.github.com> Co-authored-by: Kyryl Skobylko <xpuska513@gmail.com> * 🐛Destination Google Sheets: Fix empty headers list (#18729) * Fix empty headers list * Updated PR number * Bumped version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🐛Source Exchange Rates: Fix handling error during check connection (#18726) * Fix handling error during check connection * Updated PR number * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Add normalization changelog and bump normalization version in platform (#18813) * Remove ConfigPersistence usage from SecretsMigrator (#18747) * remove config persistence from seeding logic (#18749) * Remove the bulk actions from ConfigPersistence (#18800) * hide ConfigPersistence inside ConfigRepository to discourage use (#18803) * ci: add job and run id to test reports (#18832) * Bump Airbyte version from 0.40.17 to 0.40.18 (#18827) Co-authored-by: grishick <grishick@users.noreply.github.com> * 🪟🔧 Remove styled components (round 1) (#18766) * refactor EditorHeader (untested) * refactor BaseClearView * delete unused Subtitle * refactor ConfirmationModal * refactor Arrow * refactor BulkHeader * refactor CatalogTreeSearch * refactor StreamFieldTable * refactor StreamHeader * refactor ConnectorIcon * refactor TreeRowWrapper * refactor DeleteBlock * refactor EmptyResourceBlock * revert unintended element change * fixed acceptance tests (#18699) * 🪟🔧 Reactor Breadcrumbs component to use anchors (#18764) * refactor breadcrumbs to use actual links * PR comments on styles * increase test timeout for some webapp tests to prevent flakes (#18807) * Remove "Filters and Segments" from Google Analytics v4 (#18508) Filters and Segments info was incorrectly added to the Google Analytics v4 connector instead of the Google Analytics (Universal Analytics) Connector. * Add notes about EU OAUth (#18835) EU OAuth is not fully tested so adding a note to account for that. * 🪟🐛 Fix: visual regression in ConnectorIcon (#18849) * fix visual regression * remove unused prop * Add links to demo page (#18828) * mysql-source:fix tinyint unsigned handling (#18619) * mysql-source:fix tinyint unsigned handling * update doc * format * upgrade version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🪟 🎉 Allow environment specific sections in docs (#18829) * Allow environment specific sections in docs * Change syntax to lower case * ci: replace GITHUB_OUTPUT with GITHUB_ENV on multiline variables (#18809) * ci: replace GITHUB_OUTPUT with GITHUB_ENV on multiline variables * ci: replace github set-ouput with new syntax in ./tools/bin/ * Add connection ID to span (#18852) * edited connector docs (#18855) * 🪟 🔧 Upgrade husky to 8.0.1 (#18719) * Upgrade Husky * Upgrade Husky * Upgrade Husky * Upgrade Husky * Upgrade Husky * ci: replace GITHUB_OUTPUT with GITHUB_ENV for multiline variables (#18857) * Avoid NPE when adding connection ID to trace (#18856) * Filter exit errors by operation name (#18850) * add label * auto-bump connector version Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Xiaohan Song <xiaohan@airbyte.io> Co-authored-by: Liren Tu <tuliren@gmail.com> Co-authored-by: Conor <cpdeethree@users.noreply.github.com> Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> Co-authored-by: Greg Solovyev <grishick@users.noreply.github.com> Co-authored-by: Artem Inzhyyants <36314070+artem1205@users.noreply.github.com> Co-authored-by: perangel <perangel@gmail.com> Co-authored-by: Jonathan Pearlin <jonathan@airbyte.io> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: perangel <perangel@users.noreply.github.com> Co-authored-by: Kyryl Skobylko <xpuska513@gmail.com> Co-authored-by: Serhii Lazebnyi <53845333+lazebnyi@users.noreply.github.com> Co-authored-by: Charles <charles@airbyte.io> Co-authored-by: Octavia Squidington III <90398440+octavia-squidington-iii@users.noreply.github.com> Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com> Co-authored-by: darynaishchenko <80129833+darynaishchenko@users.noreply.github.com> Co-authored-by: Michael Siega <109092231+mfsiega-airbyte@users.noreply.github.com> Co-authored-by: Tyler B <104733644+TBernstein4@users.noreply.github.com> Co-authored-by: Yowan Ramchoreeter <26179814+YowanR@users.noreply.github.com> Co-authored-by: Tim Roes <tim@airbyte.io> Co-authored-by: Subodh Kant Chaturvedi <subodh1810@gmail.com> Co-authored-by: Volodymyr Pochtar <v.pochtar@gmail.com> Co-authored-by: Amruta Ranade <11484018+Amruta-Ranade@users.noreply.github.com>
What
Removes environment variables in the airbyte-cron deployment.yaml (helm chart) that are no present in the ConfigMap
How
Describe the solution
Recommended reading order
x.java
y.python
🚨 User Impact 🚨
Are there any breaking changes? What is the end result perceived by the user? If yes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.
Pre-merge Checklist
Expand the relevant checklist and delete the others.
New Connector
Community member or Airbyter
airbyte_secret
./gradlew :airbyte-integrations:connectors:<name>:integrationTest
.README.md
bootstrap.md
. See description and examplesdocs/integrations/<source or destination>/<name>.md
including changelog. See changelog exampledocs/integrations/README.md
airbyte-integrations/builds.md
Airbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>
command is passing/publish
command described hereUpdating a connector
Community member or Airbyter
airbyte_secret
./gradlew :airbyte-integrations:connectors:<name>:integrationTest
.README.md
bootstrap.md
. See description and examplesdocs/integrations/<source or destination>/<name>.md
including changelog. See changelog exampleAirbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>
command is passing/publish
command described hereConnector Generator
-scaffold
in their name) have been updated with the latest scaffold by running./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates
then checking in your changesTests
Unit
Put your unit tests output here.
Integration
Put your integration tests output here.
Acceptance
Put your acceptance tests output here.