Remove release stage references in favor of support level or ab_internal #29232

bnchrch · 2023-08-08T21:35:26Z

Overview

Updates the following to use the new support levels and ab_internal fields

CI check for high strictness
Registry Entry Generation
Registry Reports
QA Engine
Pipeline release stage selection
Nightly/Weekly Connector reports

…ert/update-ci-check

bnchrch · 2023-08-08T23:57:11Z

airbyte-ci/connectors/connector_ops/connector_ops/acceptance_test_config_checks.py

 BACKWARD_COMPATIBILITY_REVIEWERS = {"connector-operations", "connector-extensibility"}
 TEST_STRICTNESS_LEVEL_REVIEWERS = {"connector-operations"}
 GA_BYPASS_REASON_REVIEWERS = {"connector-operations"}
 GA_CONNECTOR_REVIEWERS = {"gl-python"}
 REVIEW_REQUIREMENTS_FILE_PATH = ".github/connector_org_review_requirements.yaml"

+def is_important_connector(connector: utils.Connector) -> bool:


Opted to use the idea of important connector which is either a high sl and/or high ql.

This is because we may have a connector that is deemed qualitatively important (SL) and we should enable high strictness in advance of a high QL.

defn: https://docs.google.com/document/d/161FF__wOjwS6j-1fVyHexDD1cJCQ5RJ2LMC8FI2N6yI/edit

The alternative is to use just QL.

bnchrch · 2023-08-08T23:57:36Z

airbyte-ci/connectors/connector_ops/connector_ops/allowed_hosts_checks.py

+    "ql": 300,
+}
+
+def _requires_allowed_hosts(connector: utils.Connector) -> bool:


Same idea as here https://github.com/airbytehq/airbyte/pull/29232/files#r1287793875

bnchrch · 2023-08-08T23:59:29Z

airbyte-ci/connectors/connector_ops/connector_ops/utils.py

+        return sl_value
+
+    @property
+    def ab_internal_ql(self) -> int:


These should already be defined but putting defaults here in case.

These should be converted to hard errors on the introduction of metadata v2 where ab_internal is required

octavia-squidington-iii · 2023-08-09T00:24:17Z

source-s3 test report (commit `4828b13dc2`) - ❌

⏲️ Total pipeline duration: 22mn45s

Step	Result
Validate airbyte-integrations/connectors/source-s3/metadata.yaml	✅
Connector version semver check	✅
QA checks	✅
Code format checks	✅
Connector package install	✅
Build source-s3 docker image for platform linux/x86_64	✅
Unit tests	✅
Integration tests	❌
Acceptance tests	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-s3 test

bnchrch · 2023-08-09T00:40:52Z

airbyte-ci/connectors/connector_ops/connector_ops/utils.py

+    def ab_internal_sl(self) -> int:
+        """Airbyte Internal Field.
+
+        More info can be found here: https://www.notion.so/Internal-Metadata-Fields-32b02037e7b244b7934214019d0b7cc9


Added a new page to notion since our wiki is now deprecated

This reverts commit 1861b95.

octavia-squidington-iii · 2023-08-09T01:02:31Z

source-s3 test report (commit `205fa6d24e`) - ✅

⏲️ Total pipeline duration: 20mn26s

Step	Result
Validate airbyte-integrations/connectors/source-s3/metadata.yaml	✅
Connector version semver check	✅
QA checks	✅
Code format checks	✅
Connector package install	✅
Build source-s3 docker image for platform linux/x86_64	✅
Unit tests	✅
Integration tests	✅
Acceptance tests	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-s3 test

alafanechere

Kudos for tackling this!
Nit comments. LGTM as long as all the tests are 🟢 .
I believe the next step is to remove the releaseStage from the metadata model + update all metadata.yaml files?

alafanechere · 2023-08-14T16:58:48Z

.github/workflows/connectors_nightly_build.yml

@@ -11,14 +11,14 @@ on:
        default: conn-nightly-xlarge-runner
        required: true
      test-connectors-options:
-        default: --concurrency=5 --release-stage=generally_available --release-stage=beta
+        default: --concurrency=5 --support-level=certified


All the metadata are up to date to match the same subset of connector nightly build originally targets?

They dont match the same subset unfortunately. There will likely be a bit of a mix up

alafanechere · 2023-08-14T17:00:35Z

airbyte-ci/connectors/connector_ops/connector_ops/acceptance_test_config_checks.py

 BACKWARD_COMPATIBILITY_REVIEWERS = {"connector-operations", "connector-extensibility"}
 TEST_STRICTNESS_LEVEL_REVIEWERS = {"connector-operations"}
 GA_BYPASS_REASON_REVIEWERS = {"connector-operations"}
 GA_CONNECTOR_REVIEWERS = {"gl-python"}
 REVIEW_REQUIREMENTS_FILE_PATH = ".github/connector_org_review_requirements.yaml"

+def is_important_connector(connector: utils.Connector) -> bool:


Would love to see this exposed as a Connector property as it might be useful in pipelines too.

alafanechere · 2023-08-14T17:02:32Z

airbyte-ci/connectors/connector_ops/connector_ops/acceptance_test_config_checks.py

+        requires_high_test_strictness_level = is_important_connector(connector)
+        check_for_high_strictness = connector.acceptance_test_config is not None and requires_high_test_strictness_level


Wondering if requires_high_test_strictness_level could be a Connector property too.

absolutely!

alafanechere · 2023-08-14T17:04:04Z

airbyte-ci/connectors/metadata_service/orchestrator/orchestrator/assets/metadata.py

@@ -100,79 +100,6 @@ def compute_registry_overrides(merged_df):
    return registries


-def merge_into_metadata_definitions(


Why is it going away?

This was an artifact from when we transformed the old registry into the metadata files at the very begining.

Because we no longer do that, this function is unused, it just happened to be missed by the grim reaper last time

alafanechere · 2023-08-14T20:08:09Z

airbyte-ci/connectors/pipelines/pipelines/commands/groups/connectors.py

-    help="Filter connectors to test by release stage.",
-    type=click.Choice(["alpha", "beta", "generally_available"]),
+    help="Filter connectors to test by support_level.",
+    type=click.Choice(["community", "certified"]),


nit: I did not made the extra mile to create an enum for releases stages, but I think it's worth it. (doing the same as ConnectorLanguage)

alafanechere · 2023-08-14T20:09:09Z

airbyte-ci/connectors/qa-engine/qa_engine/models.py

@@ -14,12 +14,10 @@ class ConnectorTypeEnum(str, Enum):
    source = "source"
    destination = "destination"

-
-class ReleaseStageEnum(str, Enum):
+class SupportLevelEnum(str, Enum):


Ahah the enum exists here :) Might be worth migrating it to connector_ops.utils...

alafanechere · 2023-08-14T20:11:26Z

airbyte-ci/connectors/pipelines/README.md

Bump the pipelines version in pyproject.toml and add a changelog entry please 😄

octavia-squidington-iii · 2023-08-15T18:35:01Z

source-amazon-ads test report (commit `dce9b8e39b`) - ❌

⏲️ Total pipeline duration: 02mn55s

Step	Result
Validate airbyte-integrations/connectors/source-amazon-ads/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	❌
Code format checks	✅
Connector package install	✅
Build source-amazon-ads docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-amazon-ads test

This reverts commit dce9b8e.

…nal (airbytehq#29232) * First pass on removing release stage references * Automated Commit - Format and Process Resources Changes * Fix metadata tests * Fix qa engine tests * Fix pipeline tests * Update registry report * Fix support-level arg * Add dummy change * Deal with --allow_community * Improve log message * Add docs * Revert "Add dummy change" This reverts commit 1861b95. * Code review comments * Run format * Install connector ops to qa-engine * Fix CI test * REVERT ME * Revert "REVERT ME" This reverts commit dce9b8e. --------- Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>

First pass on removing release stage references

320d982

octavia-squidington-iii added the area/documentation Improvements or additions to documentation label Aug 8, 2023

bnchrch and others added 9 commits August 8, 2023 21:57

Automated Commit - Format and Process Resources Changes

473d01e

Fix metadata tests

f714830

Fix qa engine tests

0e0c241

Fix pipeline tests

fd65b29

Merge remote-tracking branch 'origin/master' into bnchrch/connector-c…

54e79c5

…ert/update-ci-check

Update registry report

e7ddacc

Fix support-level arg

abb3046

Add dummy change

1861b95

Deal with --allow_community

4828b13

octavia-squidington-iii added area/connectors Connector related issues connectors/source/s3 labels Aug 8, 2023

bnchrch changed the title ~~WIP: Remove release stage references in favor of support level~~ Remove release stage references in favor of support level or ab_internal Aug 8, 2023

bnchrch commented Aug 8, 2023

View reviewed changes

Improve log message

692842d

bnchrch commented Aug 8, 2023

View reviewed changes

bnchrch marked this pull request as ready for review August 8, 2023 23:59

bnchrch requested a review from a team August 8, 2023 23:59

Add docs

205fa6d

bnchrch commented Aug 9, 2023

View reviewed changes

Revert "Add dummy change"

31aa865

This reverts commit 1861b95.

octavia-squidington-iii removed the area/connectors Connector related issues label Aug 9, 2023

alafanechere approved these changes Aug 14, 2023

View reviewed changes

bnchrch added 3 commits August 14, 2023 18:31

Merge branch 'master' into bnchrch/connector-cert/update-ci-check

0c7c6a6

Code review comments

ef3c687

Run format

00f0361

bnchrch added 3 commits August 14, 2023 19:02

Install connector ops to qa-engine

c571681

Fix CI test

6c0260c

REVERT ME

dce9b8e

octavia-squidington-iii added area/connectors Connector related issues connectors/source/amazon-ads labels Aug 15, 2023

Revert "REVERT ME"

9538097

This reverts commit dce9b8e.

octavia-squidington-iii removed the area/connectors Connector related issues label Aug 15, 2023

bnchrch enabled auto-merge (squash) August 15, 2023 19:03

bnchrch merged commit b666e2e into master Aug 15, 2023

bnchrch deleted the bnchrch/connector-cert/update-ci-check branch August 15, 2023 19:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove release stage references in favor of support level or ab_internal #29232

Remove release stage references in favor of support level or ab_internal #29232

bnchrch commented Aug 8, 2023

bnchrch Aug 8, 2023

bnchrch Aug 8, 2023

bnchrch Aug 8, 2023

octavia-squidington-iii commented Aug 9, 2023

bnchrch Aug 9, 2023

octavia-squidington-iii commented Aug 9, 2023

alafanechere left a comment

alafanechere Aug 14, 2023

bnchrch Aug 15, 2023

alafanechere Aug 14, 2023

alafanechere Aug 14, 2023

bnchrch Aug 15, 2023

alafanechere Aug 14, 2023

bnchrch Aug 15, 2023

alafanechere Aug 14, 2023 •

edited

Loading

alafanechere Aug 14, 2023

alafanechere Aug 14, 2023

octavia-squidington-iii commented Aug 15, 2023

		requires_high_test_strictness_level = is_important_connector(connector)
		check_for_high_strictness = connector.acceptance_test_config is not None and requires_high_test_strictness_level

		@@ -100,79 +100,6 @@ def compute_registry_overrides(merged_df):
		return registries


		def merge_into_metadata_definitions(

Remove release stage references in favor of support level or ab_internal #29232

Remove release stage references in favor of support level or ab_internal #29232

Conversation

bnchrch commented Aug 8, 2023

Overview

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

octavia-squidington-iii commented Aug 9, 2023

source-s3 test report (commit 4828b13dc2) - ❌

Choose a reason for hiding this comment

octavia-squidington-iii commented Aug 9, 2023

source-s3 test report (commit 205fa6d24e) - ✅

alafanechere left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alafanechere Aug 14, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

octavia-squidington-iii commented Aug 15, 2023

source-amazon-ads test report (commit dce9b8e39b) - ❌

source-s3 test report (commit `4828b13dc2`) - ❌

source-s3 test report (commit `205fa6d24e`) - ✅

alafanechere Aug 14, 2023 •

edited

Loading

source-amazon-ads test report (commit `dce9b8e39b`) - ❌