airbyte-ci: rework RC promotion #46675

alafanechere · 2024-10-09T15:15:19Z

What

We want to use -rc suffix on release candidate.
It means we have to rework our promotion flow to cut release a main version when the RC is deemed stable by the system.

Promoting a RC to a main version then just means:

Update its version to a non rc one
Set isReleaseCandidate to false

How

Rework the airbyte-ci connectors publish --promote-release-candidate command to:

Bump the connector version to a main one
Set isReleaseCandidate to false
Commit and push all this to a PR with the auto-merge label.
Update the changelog

Bonus: significant refacto on Step that modify files to have more shared logic and cleaner code.

Review guide

User Impact

Check that /bump-version still works following the refacto
Check that up-to-date still works following the refacto

vercel · 2024-10-09T15:15:21Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	⬜️ Ignored (Inspect)	Visit Preview		Oct 14, 2024 3:24pm

alafanechere · 2024-10-09T15:15:34Z

airbyte-ci: add rc bump type to bump version #46709
airbyte-ci: rework RC promotion #46675 👈
metadata-service[lib]: validate that release candidate have rc suffix in their version #46573 : 1 other dependent PR (#46678 )
master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @alafanechere and the rest of your teammates on Graphite

alafanechere · 2024-10-09T16:08:59Z

...s/pipelines/airbyte_ci/connectors/migrate_to_manifest_only/manifest_component_transformer.py

Unrelated change to make the linter happy

alafanechere · 2024-10-09T16:09:09Z

...tors/pipelines/pipelines/airbyte_ci/connectors/migrate_to_manifest_only/manifest_resolver.py

Unrelated change to make the linter happy

clnoll

Approving so I don't block merge, but there are a few places where I'm afraid we'll swallow any errors that might happen, which will make it difficult to debug. If you could add some logging and possibly error handling too that would address my concern.

clnoll · 2024-10-14T11:16:43Z

airbyte-ci/connectors/pipelines/pipelines/airbyte_ci/connectors/publish/pipeline.py

+            if reset_release_candidate_results.success:
+                all_modified_files.update(await reset_release_candidate.export_modified_files(context.connector.code_directory))


Same question as above.

clnoll · 2024-10-14T11:17:40Z

airbyte-ci/connectors/pipelines/pipelines/airbyte_ci/connectors/publish/pipeline.py

+            if set_promoted_version_results.success:
+                all_modified_files.update(await set_promoted_version.export_modified_files(context.connector.code_directory))


Should we be exiting early on error here? What's the justification for moving onto the next step if this one wasn't successful?

I think we at least want some logging.

I think we at least want some logging.
Errors will appear in the report as I added step results to the results list.
I choosed to not early exit because:

If the step result is skipped we want to continue to the next step (e.g. a partial bump worked previously)

If it's failing I think it's interesting to observe in the report if the next step is also failing or working.

Okay cool - maybe add a comment explaining why we're okay continuing on failure?

clnoll · 2024-10-14T11:24:24Z

airbyte-ci/connectors/pipelines/pipelines/airbyte_ci/connectors/publish/pipeline.py

+                initial_pr_creation_result = await initial_pr_creation.run(*pr_creation_args, **pr_creation_kwargs)
+                results.append(initial_pr_creation_result)
+                # Update changelog and update PR
+                if initial_pr_creation_result.success:


Mind adding some logging here if it wasn't successful?

clnoll · 2024-10-14T11:25:29Z

airbyte-ci/connectors/pipelines/pipelines/airbyte_ci/connectors/publish/pipeline.py

+                    )
+                    add_changelog_entry_result = await add_changelog_entry.run()
+                    results.append(add_changelog_entry_result)
+                    if add_changelog_entry_result.success:


Same comment as above.

sentry-io · 2024-10-17T20:11:33Z

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

‼️ TypeError: Method dagger.client.gen.Host.directory() parameter include='metadata.yaml' was expected to be of... pipelines.models.contexts.pipeline_context in g... View Issue

_{Did you find this useful? React with a 👍 or 👎}

alafanechere mentioned this pull request Oct 9, 2024

metadata-service[lib]: validate that release candidate have rc suffix in their version #46573

Merged

alafanechere force-pushed the augustin/10-08-metadata-service_lib_validate_that_release_candidate_have_rc_suffix_in_their_version branch from 0296422 to 3d5bcfa Compare October 9, 2024 16:00

alafanechere mentioned this pull request Oct 9, 2024

source-faker: set rc suffix #46678

Merged

alafanechere force-pushed the augustin/10-08-airbyte-ci_rework_RC_promotion branch from 94050e0 to ee619da Compare October 9, 2024 16:01

alafanechere commented Oct 9, 2024

View reviewed changes

alafanechere force-pushed the augustin/10-08-airbyte-ci_rework_RC_promotion branch from ee619da to 3eb9e57 Compare October 9, 2024 16:11

alafanechere marked this pull request as ready for review October 9, 2024 16:17

alafanechere requested a review from a team as a code owner October 9, 2024 16:17

alafanechere force-pushed the augustin/10-08-metadata-service_lib_validate_that_release_candidate_have_rc_suffix_in_their_version branch from 3d5bcfa to 1d7193a Compare October 10, 2024 09:23

alafanechere force-pushed the augustin/10-08-airbyte-ci_rework_RC_promotion branch from 3eb9e57 to 0a6c454 Compare October 10, 2024 09:24

alafanechere force-pushed the augustin/10-08-metadata-service_lib_validate_that_release_candidate_have_rc_suffix_in_their_version branch from 1d7193a to f0c71ef Compare October 10, 2024 09:37

alafanechere force-pushed the augustin/10-08-airbyte-ci_rework_RC_promotion branch from 0a6c454 to bd41b41 Compare October 10, 2024 09:37

alafanechere force-pushed the augustin/10-08-metadata-service_lib_validate_that_release_candidate_have_rc_suffix_in_their_version branch from f0c71ef to 1a998df Compare October 10, 2024 10:11

Base automatically changed from augustin/10-08-metadata-service_lib_validate_that_release_candidate_have_rc_suffix_in_their_version to master October 10, 2024 12:45

alafanechere force-pushed the augustin/10-08-airbyte-ci_rework_RC_promotion branch from bd41b41 to 5db8b0d Compare October 10, 2024 13:34

alafanechere mentioned this pull request Oct 10, 2024

airbyte-ci: add rc bump type to bump version #46709

Merged

alafanechere force-pushed the augustin/10-08-airbyte-ci_rework_RC_promotion branch from 5db8b0d to 81f1b5c Compare October 10, 2024 14:43

clnoll approved these changes Oct 14, 2024

View reviewed changes

alafanechere force-pushed the augustin/10-08-airbyte-ci_rework_RC_promotion branch from 0189a00 to 2ddecef Compare October 14, 2024 15:06

alafanechere added 3 commits October 14, 2024 17:10

airbyte-ci: rework RC promotion

c1bd085

fix types, lint and format

8148d01

fix connops tests

8e6a057

alafanechere force-pushed the augustin/10-08-airbyte-ci_rework_RC_promotion branch from 2ddecef to 8e6a057 Compare October 14, 2024 15:23

alafanechere enabled auto-merge (squash) October 14, 2024 15:24

alafanechere merged commit 19f556f into master Oct 14, 2024
34 checks passed

alafanechere deleted the augustin/10-08-airbyte-ci_rework_RC_promotion branch October 14, 2024 16:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

airbyte-ci: rework RC promotion #46675

airbyte-ci: rework RC promotion #46675

alafanechere commented Oct 9, 2024 •

edited

Loading

vercel bot commented Oct 9, 2024 •

edited

Loading

alafanechere commented Oct 9, 2024 •

edited

Loading

alafanechere Oct 9, 2024

alafanechere Oct 9, 2024

clnoll left a comment

clnoll Oct 14, 2024

clnoll Oct 14, 2024

alafanechere Oct 14, 2024

clnoll Oct 14, 2024

clnoll Oct 14, 2024

clnoll Oct 14, 2024

sentry-io bot commented Oct 17, 2024

		if reset_release_candidate_results.success:
		all_modified_files.update(await reset_release_candidate.export_modified_files(context.connector.code_directory))

		if set_promoted_version_results.success:
		all_modified_files.update(await set_promoted_version.export_modified_files(context.connector.code_directory))

airbyte-ci: rework RC promotion #46675

airbyte-ci: rework RC promotion #46675

Conversation

alafanechere commented Oct 9, 2024 • edited Loading

What

How

Review guide

User Impact

vercel bot commented Oct 9, 2024 • edited Loading

alafanechere commented Oct 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

clnoll left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sentry-io bot commented Oct 17, 2024

Suspect Issues

alafanechere commented Oct 9, 2024 •

edited

Loading

vercel bot commented Oct 9, 2024 •

edited

Loading

alafanechere commented Oct 9, 2024 •

edited

Loading