airbyte-ci: compute GHA runs-on from --ci-requirements

[alafanechere]

What

Closes #34076

We want to dynamically determine the self hosted CI runner to use according to the output of --ci-requirements

How

• Introduce a new airbyte-ci-requirements action: it will run --ci-requirements on inputted commands, on a github hosted ubuntu runner.
• Use this action in an upstream job in all our existing GHA workflow using airbyte-ci. We run this preliminary job to compute the name of the self-hosted runner the main job will use (runs-on field).

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	⬜️ Ignored (Inspect)	Visit Preview		Jan 16, 2024 2:09pm

[alafanechere]

In this stack: 🔧 Add CI requirements to Airbyte CLI(?)

• master
  • airbyte-ci: CLI exposes CI requirements #34218
    • airbyte-ci: compute GHA runs-on from --ci-requirements #34220 👈

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @alafanechere and the rest of your teammates on Graphite

[evantahler]

Nice work!
Testing GHA is always tricky, but this looks right to me.

I think maybe a note should be added in the docs that all N of the actions that run stuff with airbyte-ci will dynamically choose which runner to work on based on this (link to python code)

[alafanechere]

Nice work! Testing GHA is always tricky, but this looks right to me.

I think maybe a note should be added in the docs that all N of the actions that run stuff with airbyte-ci will dynamically choose which runner to work on based on this (link to python code)

I updated the command documentation in #34218