connectors-ci: maximize concurrency and document high level DAG

[alafanechere]

Tell us about the problem you're trying to solve

While implementing the POC we left aside a couple of optimizations related to testing execution concurrency.
We should ensure all the steps not depending on each other can run in parallel and are not blocking each other.

Describe the solution you’d like

Basically follow Dagger's team suggestion:
Discard calls made to asyncio and rely only on anyio / asyncer to benefit from the structured concurrency paradigm which makes understanding concurrent code easier.

We should target the following DAG for each connector pipeline on running connectors-ci test-connectors:

flowchart TB
    conn([Connector pipeline])==>qa & fmt & tu
    src[[Mount connector source code to container]]-->fmt & pkg & ta
    fmt & tu & ti & ta & qa -.-> r([Build test report])--if in CI-->s3[[Upload to S3]]
    subgraph QA Checks
        repo[[Mount Airbyte repo to container]]-->qa[Run QA checks]
    end
    subgraph Tests
        tu[Run unit tests]==>ti[Run integration tests] & ta[Run acceptance tests]
        pkg[[Install connector package in container]]-->tu & ti
        dsec[[Download secrets from GSM]] --> ti & ta
        subgraph Acceptance
            bld[[Build connector dev image]]-->ta-->upsec[[Upload updated secrets to GSM]]
        end
    end
    subgraph Code format
        fmt[Run code format checks]
    end

Loading