Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix column selection handling for syncs without cursors #21647

Merged
merged 3 commits into from
Jan 20, 2023
Merged

fix column selection handling for syncs without cursors #21647

merged 3 commits into from
Jan 20, 2023

Conversation

mfsiega-airbyte
Copy link
Contributor

@mfsiega-airbyte mfsiega-airbyte commented Jan 20, 2023 •
edited
Loading

What

Fix the column selection handling for syncs without cursors.

The way we handle syncs when column selection is enabled assumes that there will be a cursor field present, and all of our test cases supply it.

How

Correctly stop relying on a cursor being configured.

Testing

Added unit test coverage, also deployed to dev-3 and verified that it worked there.

@octavia-squidington-iv octavia-squidington-iv added area/platform issues related to the platform area/server labels Jan 20, 2023
@mfsiega-airbyte mfsiega-airbyte temporarily deployed to more-secrets January 20, 2023 12:38 — with GitHub Actions Inactive
@mfsiega-airbyte mfsiega-airbyte temporarily deployed to more-secrets January 20, 2023 12:38 — with GitHub Actions Inactive
@github-actions
Copy link
Contributor

github-actions bot commented Jan 20, 2023 •
edited
Loading

Airbyte Code Coverage

File Coverage [93.93%] 🍏
CatalogConverter.java 93.93% 🍏
Total Project Coverage 26.75% 🍏

josephkmh
josephkmh reviewed Jan 20, 2023
View reviewed changes
Copy link
Contributor

@josephkmh josephkmh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll leave the approval to @pmossman, but the logic in CatalogConverter.java looks good, now I see where the bug was!

@mfsiega-airbyte mfsiega-airbyte temporarily deployed to more-secrets January 20, 2023 13:27 — with GitHub Actions Inactive
@mfsiega-airbyte mfsiega-airbyte temporarily deployed to more-secrets January 20, 2023 13:27 — with GitHub Actions Inactive
@mfsiega-airbyte mfsiega-airbyte temporarily deployed to more-secrets January 20, 2023 14:47 — with GitHub Actions Inactive
@mfsiega-airbyte mfsiega-airbyte temporarily deployed to more-secrets January 20, 2023 14:47 — with GitHub Actions Inactive
@mfsiega-airbyte mfsiega-airbyte temporarily deployed to more-secrets January 20, 2023 15:12 — with GitHub Actions Inactive
@mfsiega-airbyte mfsiega-airbyte temporarily deployed to more-secrets January 20, 2023 15:12 — with GitHub Actions Inactive
pmossman
pmossman approved these changes Jan 20, 2023
View reviewed changes
Copy link
Contributor

@pmossman pmossman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logic checks out to me, and appreciate the test refactoring!

@mfsiega-airbyte mfsiega-airbyte enabled auto-merge (squash) January 20, 2023 17:42
@mfsiega-airbyte mfsiega-airbyte temporarily deployed to more-secrets January 20, 2023 17:48 — with GitHub Actions Inactive
@mfsiega-airbyte mfsiega-airbyte temporarily deployed to more-secrets January 20, 2023 17:48 — with GitHub Actions Inactive
@pmossman pmossman temporarily deployed to more-secrets January 20, 2023 21:51 — with GitHub Actions Inactive
@pmossman pmossman temporarily deployed to more-secrets January 20, 2023 21:51 — with GitHub Actions Inactive
@mfsiega-airbyte mfsiega-airbyte merged commit 5257f69 into master Jan 20, 2023
@mfsiega-airbyte mfsiega-airbyte deleted the msiega/column-selection-fix branch January 20, 2023 22:26
@josephkmh josephkmh mentioned this pull request Jan 21, 2023
Merged
@octavia-squidington-iii octavia-squidington-iii mentioned this pull request Jan 24, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@josephkmh josephkmh josephkmh left review comments

@pmossman pmossman pmossman approved these changes

Assignees
No one assigned
Labels
area/platform issues related to the platform area/server
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@mfsiega-airbyte @josephkmh @pmossman @octavia-squidington-iv