ci_credentials: fix overwriting 'data' before getting nextPageToken #24265
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
9 tasks
bnchrch
approved these changes
Mar 21, 2023
| @@ -44,12 +44,19 @@ The `VERSION=dev` will make it so it knows to use your local current working dir | |||
| ci_credentials --help | |||
| ``` | |||
|
|
|||
| ### Write to storage | |||
| ### Write credentials for a specific connector to local storage | |||
| To download GSM secrets to `airbyte-integrations/connectors/source-bings-ads/secrets`: | ||
| ```bash | ||
| ci_credentials source-bing-ads write-to-storage | ||
| ``` | ||
|
|
||
| ### Write credentials for all connectors to local storage |
| @@ -81,8 +81,8 @@ def __load_gsm_secrets(self) -> List[RemoteSecret]: | |||
| if next_token: | |||
| params["pageToken"] = next_token | |||
|
|
|||
| data = self.api.get(url, params=params) | |||
| for secret_info in data.get("secrets") or []: | |||
| all_secrets_data = self.api.get(url, params=params) | |||
| @@ -103,14 +103,14 @@ def __load_gsm_secrets(self) -> List[RemoteSecret]: | |||
| self.logger.info(f"found GSM secret: {log_name} = > {filename}") | |||
|
|
|||
| versions_url = f"https://secretmanager.googleapis.com/v1/{secret_name}/versions" | |||
| data = self.api.get(versions_url) | |||
| enabled_versions = [version["name"] for version in data["versions"] if version["state"] == "ENABLED"] | |||
| versions_data = self.api.get(versions_url) | |||
There was a problem hiding this comment.
(Dreaming) If only python had an immutable variable feature
There was a problem hiding this comment.
Lol, too spoiled by java's pickiness!
erohmensing
added a commit
that referenced
this pull request
Mar 22, 2023
…24265) * ci_credentials: fix overwriting 'data' before getting nextPageToken * Updates to readme
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
VERSION=dev ci_credentials all write-to-storagewould only write 100 secrets instead of the full 408How
Use diffferent variable names when getting
datafrom various endpoints. Previously we were overwriting thedatavariable with different contents - therefore thenextPageTokenwas either not there or invalid by the time we tried to use it, so we never got the next page.Cherrypicked from #24254 as it was quasi-necessary to run all CATs locally (you can run ci_credentials per connector, but that's slow), and I figured other people might be able to use this now.