Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

connectors-ci: setup nightly builds #24335

Conversation

alafanechere
Copy link
Contributor

@alafanechere alafanechere commented Mar 21, 2023

What

Closes #23970

Fixes

  • Only send github commit status when the CI context is pull request. We'll face errors during nightly builds otherwise.

Improvements:

  • Use anyio.Semaphore to control concurrency: we only allow 5 connectors to be tested in parallel.

Testing and problems:

  • The pipeline are stuck on the large-runner...
  • I disabled acceptance test to check if this situation is related to this heavy tests or not: the pipelines remain stuck on different steps...

Example of stuck pipelines:

https://github.com/airbytehq/airbyte/actions/runs/4488313878

https://github.com/airbytehq/airbyte/actions/runs/4488086795
https://github.com/airbytehq/airbyte/actions/runs/4484530674
https://github.com/airbytehq/airbyte/actions/runs/4485087205
https://github.com/airbytehq/airbyte/actions/runs/4488035164

@alafanechere alafanechere changed the title connectors-ci: only send commit status on PR connectors-ci: setup nightly builds Mar 22, 2023
@alafanechere alafanechere marked this pull request as draft March 22, 2023 09:31
@alafanechere alafanechere marked this pull request as ready for review March 23, 2023 17:25
@alafanechere
Copy link
Contributor Author

@alafanechere
Copy link
Contributor Author

/approve-and-merge reason="POC nightly builds dagger + GHA"

@octavia-approvington
Copy link
Contributor

After a careful ML study,
we think this looks okay.
imagine code being okay

@octavia-approvington octavia-approvington merged commit 810f9e9 into master Mar 23, 2023
@octavia-approvington octavia-approvington deleted the augustin/connectors-ci/do-not-update-commit-status-if-not-on-pr branch March 23, 2023 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

connectors-ci: use the new connectors-ci command to run connector nightly builds
2 participants