ci_connector_ops: POC rewriting airbyte-python.gradle and airbyte-connector-acceptance.gradle in Python

[alafanechere]

Tell us about the problem you're trying to solve

I believe we could take back the ownership of the connector test pipeline by rewriting them with Python code.
I'd like to tackle this by rewriting airbyte-python.gradle and airbyte-connector-acceptance-test.gradle in Python.

Describe the solution you’d like

Declaring test pipelines in Python has the following advantages:

It’s easily readable and understandable by the Airbyte engineering team.
Enhances the code discoverability and debugging experience: we’ll no longer rely on a combination of shell scripts, GitHub workflows, Gradle/Python code to get the full picture of our test pipeline. We’ll have a centralized Python project to look at.
The workflow can be unit tested.
We can benefit from abstractions to declare reusable pipeline components and grow them with scale and flexibility.
The test pipelines can be executed locally in the same way they’ll be run in the CI.
A significant portion of the existing test tooling is already written in Python: ci-credentials to manage connector test secrets, ci-connector-ops which bundle multiple checks (QA, allowed host, test strictness level etc.), connector acceptance tests. It means we could directly load and reuse these packages.
We can use Dagger’s SDK to benefit from free incremental builds and isolation by design and continue to consider containers Airbyte’s first class citizens.

We’ll need to pay extra care at building reusable components (a.k.a actions) to populate the workflow. This is the key to have a lean test pipeline and make sure this project can grow with new use cases in the future.