Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

connector acceptance tests: bug fixes #44529

Merged
merged 2 commits into from
Aug 21, 2024
Merged

Conversation

postamar
Copy link
Contributor

@postamar postamar commented Aug 21, 2024

What

Bug fixes for dagger execution caching and for failure trace message test case in the connector acceptance test suite.

User Impact

Decreased chance of false positive CAT test runs.

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

@postamar postamar requested a review from a team as a code owner August 21, 2024 19:23
Copy link

vercel bot commented Aug 21, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview Aug 21, 2024 9:07pm

cursor_field=[],
minimum_generation_id=1,
generation_id=1,
sync_id=1,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without this fix, source connectors built with the bulk CDK will fail unexpectedly. The Bulk CDK performs schema validation on the connector CLI inputs, so if these fields are missing, its execution fails very early.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind adding a comment with a link to the bulk CDK validation?

@@ -54,7 +55,7 @@ async def _run_with_config(container: dagger.Container, command: List[str], conf


async def _run(container: dagger.Container, command: List[str]) -> dagger.Container:
return await container.with_exec(command, skip_entrypoint=True)
return await (container.with_env_variable("CACHEBUSTER", str(uuid.uuid4())).with_exec(command, skip_entrypoint=True))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we don't add this cache-busting layer, the with_exec layer itself can be cached. This became apparent when the container setup auto-fixture didn't actually run like I thought it would (because the run was identical to another, earlier test case)

Copy link
Contributor

@clnoll clnoll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM @postamar, just one nonblocking request.

cursor_field=[],
minimum_generation_id=1,
generation_id=1,
sync_id=1,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind adding a comment with a link to the bulk CDK validation?

@postamar postamar enabled auto-merge (squash) August 21, 2024 21:07
@postamar postamar merged commit 0f3b136 into master Aug 21, 2024
30 checks passed
@postamar postamar deleted the postamar/acceptance-test-fixes branch August 21, 2024 21:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants