Community PR: Support CMEK or the GCS destination connector #21682

grishick · 2023-01-20T23:11:17Z

What

rebase and test community PR: #20351

github-actions · 2023-01-20T23:13:23Z

Affected Connector Report

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to do the following as needed:

Run integration tests
Bump connector or module version
Add changelog
Publish the new version

✅ Sources (0)

Connector	Version	Changelog	Publish

See "Actionable Items" below for how to resolve warnings and errors.

❌ Destinations (4)

Connector	Version	Changelog	Publish
`destination-bigquery`	`1.2.12`	✅	✅
`destination-bigquery-denormalized`	`1.2.12`	✅	❌ (diff seed version)
`destination-gcs`	`0.2.14`	✅	✅
`destination-snowflake`	`0.4.44`	✅	✅

See "Actionable Items" below for how to resolve warnings and errors.

✅ Other Modules (0)

Actionable Items

(click to expand)

Category	Status	Actionable Item
Version	❌ mismatch	The version of the connector is different from its normal variant. Please bump the version of the connector.
	⚠ doc not found	The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like `source-jdbc` is not published or documented). Please double-check to make sure that it is not a bug.
Changelog	⚠ doc not found	The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like `source-jdbc` is not published or documented). Please double-check to make sure that it is not a bug.
	❌ changelog missing	There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog.
Publish	⚠ not in seed	The connector is not in the seed file (e.g. `source_definitions.yaml`), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug.
	❌ diff seed version	The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the `/publish` command to publish the latest version.

grishick · 2023-01-20T23:14:20Z

/test connector=connectors/destination-gcs

🕑 connectors/destination-gcs https://github.com/airbytehq/airbyte/actions/runs/3971991589
✅ connectors/destination-gcs https://github.com/airbytehq/airbyte/actions/runs/3971991589
No Python unittests run

Build Passed

Test summary info:

All Passed

grishick · 2023-01-20T23:14:38Z

/test connector=connectors/destination-bigquery

🕑 connectors/destination-bigquery https://github.com/airbytehq/airbyte/actions/runs/3971993385
✅ connectors/destination-bigquery https://github.com/airbytehq/airbyte/actions/runs/3971993385
Python tests coverage:

Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
normalization/transform_config/__init__.py                            2      0   100%
normalization/transform_catalog/reserved_keywords.py                 14      0   100%
normalization/transform_catalog/__init__.py                           2      0   100%
normalization/destination_type.py                                    14      0   100%
normalization/__init__.py                                             4      0   100%
normalization/transform_catalog/destination_name_transformer.py     166      8    95%
normalization/transform_catalog/table_name_registry.py              174     34    80%
normalization/transform_config/transform.py                         189     48    75%
normalization/transform_catalog/utils.py                             51     14    73%
normalization/transform_catalog/dbt_macro.py                         22      7    68%
normalization/transform_catalog/catalog_processor.py                147     80    46%
normalization/transform_catalog/transform.py                         61     38    38%
normalization/transform_catalog/stream_processor.py                 595    400    33%
-------------------------------------------------------------------------------------
TOTAL                                                              1441    629    56%

Build Passed

Test summary info:

All Passed

grishick · 2023-01-20T23:14:54Z

/test connector=connectors/destination-snowflake

🕑 connectors/destination-snowflake https://github.com/airbytehq/airbyte/actions/runs/3971994758
✅ connectors/destination-snowflake https://github.com/airbytehq/airbyte/actions/runs/3971994758
Python tests coverage:

Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
normalization/transform_config/__init__.py                            2      0   100%
normalization/transform_catalog/reserved_keywords.py                 14      0   100%
normalization/transform_catalog/__init__.py                           2      0   100%
normalization/destination_type.py                                    14      0   100%
normalization/__init__.py                                             4      0   100%
normalization/transform_catalog/destination_name_transformer.py     166      8    95%
normalization/transform_catalog/table_name_registry.py              174     34    80%
normalization/transform_config/transform.py                         189     48    75%
normalization/transform_catalog/utils.py                             51     14    73%
normalization/transform_catalog/dbt_macro.py                         22      7    68%
normalization/transform_catalog/catalog_processor.py                147     80    46%
normalization/transform_catalog/transform.py                         61     38    38%
normalization/transform_catalog/stream_processor.py                 595    400    33%
-------------------------------------------------------------------------------------
TOTAL                                                              1441    629    56%

Build Passed

Test summary info:

All Passed

grishick · 2023-01-20T23:19:25Z

/test connector=connectors/destination-bigquery-denormalized

grishick · 2023-01-21T00:51:11Z

/test connector=connectors/destination-bigquery-denormalized

🕑 connectors/destination-bigquery-denormalized https://github.com/airbytehq/airbyte/actions/runs/3972436964
✅ connectors/destination-bigquery-denormalized https://github.com/airbytehq/airbyte/actions/runs/3972436964
Python tests coverage:

Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
normalization/transform_config/__init__.py                            2      0   100%
normalization/transform_catalog/reserved_keywords.py                 14      0   100%
normalization/transform_catalog/__init__.py                           2      0   100%
normalization/destination_type.py                                    14      0   100%
normalization/__init__.py                                             4      0   100%
normalization/transform_catalog/destination_name_transformer.py     166      8    95%
normalization/transform_catalog/table_name_registry.py              174     34    80%
normalization/transform_config/transform.py                         189     48    75%
normalization/transform_catalog/utils.py                             51     14    73%
normalization/transform_catalog/dbt_macro.py                         22      7    68%
normalization/transform_catalog/catalog_processor.py                147     80    46%
normalization/transform_catalog/transform.py                         61     38    38%
normalization/transform_catalog/stream_processor.py                 595    400    33%
-------------------------------------------------------------------------------------
TOTAL                                                              1441    629    56%

Build Passed

Test summary info:

All Passed

grishick · 2023-01-21T09:14:28Z

/publish connector=connectors/destination-gcs

🕑 Publishing the following connectors:
connectors/destination-gcs
https://github.com/airbytehq/airbyte/actions/runs/3974170462

Connector	Did it publish?	Were definitions generated?
connectors/destination-gcs	❌	❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

armsepehr · 2023-01-22T14:11:08Z

/publish connector=connectors/destination-gcs

🕑 Publishing the following connectors:connectors/destination-gcshttps://github.com/airbytehq/airbyte/actions/runs/3974170462

Connector Did it publish? Were definitions generated?
connectors/destination-gcs ❌ ❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@grishick
is there anything else i should do?

marcosmarxm · 2023-01-23T09:07:58Z

/publish connector=connectors/destination-gcs

🕑 Publishing the following connectors:connectors/destination-gcshttps://github.com/airbytehq/airbyte/actions/runs/3974170462

Connector Did it publish? Were definitions generated?
connectors/destination-gcs ❌ ❌
if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@grishick is there anything else i should do?

Only need to bump the connector version.

grishick · 2023-01-23T23:31:41Z

/publish connector=connectors/destination-gcs

🕑 Publishing the following connectors:
connectors/destination-gcs
https://github.com/airbytehq/airbyte/actions/runs/3991514025

Connector	Did it publish?	Were definitions generated?
connectors/destination-gcs	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

github-actions · 2023-01-24T00:07:48Z

Airbyte Code Coverage

There is no coverage information present for the Files changed

Total Project Coverage	26.7%	🍏

edgao

I missed this review request earlier today, but LGTM - thanks for publishing+merging!

grishick · 2023-01-24T01:58:49Z

auto-merge FTW

grishick requested a review from a team as a code owner January 20, 2023 23:11

octavia-squidington-iv added area/connectors Connector related issues connectors/destination/gcs labels Jan 20, 2023

grishick force-pushed the greg/destination-gcs-cmek branch from 4b97ce0 to 38e9252 Compare January 20, 2023 23:12

grishick changed the title ~~BUGS: CMEK is supported for the GCS destination connector~~ Community PR: Support CMEK or the GCS destination connector Jan 20, 2023

octavia-squidington-iv added the area/documentation Improvements or additions to documentation label Jan 20, 2023

grishick mentioned this pull request Jan 21, 2023

🐛 Destination GCS: CMEK is supported for the connector #20351

Closed

37 tasks

grishick requested review from marcosmarxm, edgao and etsybaev January 23, 2023 02:24

marcosmarxm approved these changes Jan 23, 2023

View reviewed changes

armsepehr and others added 2 commits January 23, 2023 15:29

BUGS: CMEK is supported for the GCS destination connector

9d745e3

Update destination-gcs version and changelog

ba9aa31

grishick force-pushed the greg/destination-gcs-cmek branch from 95a402f to ba9aa31 Compare January 23, 2023 23:31

auto-bump connector version

bf86adf

octavia-squidington-iii temporarily deployed to more-secrets January 23, 2023 23:58 — with GitHub Actions Inactive

grishick enabled auto-merge (squash) January 24, 2023 00:04

grishick merged commit 615bce0 into master Jan 24, 2023

grishick deleted the greg/destination-gcs-cmek branch January 24, 2023 00:31

edgao reviewed Jan 24, 2023

View reviewed changes

octavia-squidington-iii mentioned this pull request Jan 24, 2023

Bump Airbyte version from 0.40.28 to 0.40.29 #21767

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Community PR: Support CMEK or the GCS destination connector #21682

Community PR: Support CMEK or the GCS destination connector #21682

grishick commented Jan 20, 2023

github-actions bot commented Jan 20, 2023 •

edited

Loading

✅ Sources (0)

❌ Destinations (4)

✅ Other Modules (0)

Actionable Items

grishick commented Jan 20, 2023 •

edited by github-actions bot

Loading

grishick commented Jan 20, 2023 •

edited by github-actions bot

Loading

grishick commented Jan 20, 2023 •

edited by github-actions bot

Loading

grishick commented Jan 20, 2023

grishick commented Jan 21, 2023 •

edited by github-actions bot

Loading

grishick commented Jan 21, 2023 •

edited by github-actions bot

Loading

armsepehr commented Jan 22, 2023 •

edited

Loading

marcosmarxm commented Jan 23, 2023

grishick commented Jan 23, 2023 •

edited by github-actions bot

Loading

github-actions bot commented Jan 24, 2023

edgao left a comment

grishick commented Jan 24, 2023

Community PR: Support CMEK or the GCS destination connector #21682

Community PR: Support CMEK or the GCS destination connector #21682

Conversation

grishick commented Jan 20, 2023

What

github-actions bot commented Jan 20, 2023 • edited Loading

Affected Connector Report

✅ Sources (0)

❌ Destinations (4)

✅ Other Modules (0)

Actionable Items

grishick commented Jan 20, 2023 • edited by github-actions bot Loading

Build Passed

grishick commented Jan 20, 2023 • edited by github-actions bot Loading

Build Passed

grishick commented Jan 20, 2023 • edited by github-actions bot Loading

Build Passed

grishick commented Jan 20, 2023

grishick commented Jan 21, 2023 • edited by github-actions bot Loading

Build Passed

grishick commented Jan 21, 2023 • edited by github-actions bot Loading

armsepehr commented Jan 22, 2023 • edited Loading

marcosmarxm commented Jan 23, 2023

grishick commented Jan 23, 2023 • edited by github-actions bot Loading

github-actions bot commented Jan 24, 2023

Airbyte Code Coverage

edgao left a comment

Choose a reason for hiding this comment

grishick commented Jan 24, 2023

github-actions bot commented Jan 20, 2023 •

edited

Loading

grishick commented Jan 20, 2023 •

edited by github-actions bot

Loading

grishick commented Jan 20, 2023 •

edited by github-actions bot

Loading

grishick commented Jan 20, 2023 •

edited by github-actions bot

Loading

grishick commented Jan 21, 2023 •

edited by github-actions bot

Loading

grishick commented Jan 21, 2023 •

edited by github-actions bot

Loading

armsepehr commented Jan 22, 2023 •

edited

Loading

grishick commented Jan 23, 2023 •

edited by github-actions bot

Loading