Adding allowHosts to all non-GA source database connectors #23455

prateekmukhedkar · 2023-02-24T22:15:38Z

Adding allowedHosts property to all non-GA database source connectors.
These are the list of sources I have added allowedHosts to:

source-azure-table
source-clickhouse
source-cockroachdb
source-fauna
source-db2
source-oracle
source-redshift
source-snowflake
source-tidb
source-firebolt

What

As part of the network isolation epic, all connectors will need to provide which hosts the connector container will be granted access to.

This change is specific to DB Source Connectors, which are not GA. (separate PR will be sent out for GA source connectors)

Note that we are adding ${tunnel_method.tunnel_host} for only those connectors that we support connection via SSH.

How

For databases, this information is dynamic and read from the connector's config. It probably looks a lot like:

airbyte/airbyte-config/init/src/main/resources/seed/source_definitions.yaml

Lines 1370 to 1373 in e6441bf

    
           allowedHosts: 
        
             hosts: 
        
               - "${host}" 
        
               - "${tunnel_method.tunnel_host}"

Verification

I tried with an MSSQL source that needs to be connected via an SSH tunnel and saw these in logs.

2023-02-24 19:30:30 �[32mINFO�[m i.a.w.p.DockerProcessFactory(create):125 - Creating docker container = source-mssql-read-78-0-yquyd with resources io.airbyte.config.ResourceRequirements@40cf4b83[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] and allowedHosts io.airbyte.config.AllowedHosts@32230bc8[hosts=[mssql-ssh-gl.cevykyaz98rn.us-east-2.rds.amazonaws.com, 3.18.93.32, *.datadoghq.com, *.datadoghq.eu, *.sentry.io],additionalProperties={}]

Recommended reading order

source_definitions.yaml

CLAassistant · 2023-02-24T22:15:42Z

All committers have signed the CLA.

evantahler

I didn't double check all of the connectors, but this looks good to me!

evantahler · 2023-02-24T23:17:59Z

airbyte-config/init/src/main/resources/seed/source_definitions.yaml

@@ -553,7 +563,10 @@
  icon: fauna.svg
  sourceType: database
  releaseStage: alpha
- name: File (CSV, JSON, Excel, Feather, Parquet)


Why the name change?

evantahler · 2023-02-24T23:21:24Z

airbyte-config/init/src/main/resources/seed/source_definitions.yaml

+  allowedHosts:
+    hosts:
+      - "${host}"


I think it might be called "account_name" for snowflake:

Nope! I was wrong - it's still "host":

airbyte/airbyte-config/init/src/main/resources/seed/source_specs.yaml

Lines 14068 to 14075 in 7da6a3b

host:

description: "The host domain of the snowflake instance (must include the\

\ account, region, cloud environment, and end with snowflakecomputing.com)."

examples:

- "accountname.us-east-2.aws.snowflakecomputing.com"

type: "string"

title: "Account Name"

order: 1

Thanks for pointing me to source_specs.yaml. Turns out it is not host name in all places.

…s-non-ga

2. Removing host names under allowed hosts for BigQuery, Azure Table Storage and MongoDB sources until we figure out what to configure

prateekmukhedkar · 2023-02-28T19:41:30Z

/test connector=connectors/source-mssql

🕑 connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/4296766791
✅ connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/4296766791
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
================= 19 passed, 6 skipped, 27 warnings in 18.92s ==================

prateekmukhedkar · 2023-02-28T19:42:15Z

/test connector=connectors/source-snowflake

🕑 connectors/source-snowflake https://github.com/airbytehq/airbyte/actions/runs/4296771969
✅ connectors/source-snowflake https://github.com/airbytehq/airbyte/actions/runs/4296771969
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:509: The previous and actual discovered catalogs are identical.
======================== 31 passed, 3 skipped in 56.10s ========================

…s-non-ga

prateekmukhedkar · 2023-03-01T00:34:56Z

/test connector=source-cockroachdb

🕑 source-cockroachdb https://github.com/airbytehq/airbyte/actions/runs/4298752876
✅ source-cockroachdb https://github.com/airbytehq/airbyte/actions/runs/4298752876
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
================= 19 passed, 6 skipped, 27 warnings in 15.69s ==================

prateekmukhedkar · 2023-03-01T00:35:02Z

/test connector=source-fauna

🕑 source-fauna https://github.com/airbytehq/airbyte/actions/runs/4298753113
❌ source-fauna https://github.com/airbytehq/airbyte/actions/runs/4298753113
🐛 https://gradle.com/s/u5axt56onp4sm

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs1] - AssertionError: Stre...
FAILED test_incremental.py::TestIncremental::test_two_sequential_reads[inputs0]
FAILED test_incremental.py::TestIncremental::test_two_sequential_reads[inputs1]
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [2] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:509: The previous and actual discovered catalogs are identical.
============ 3 failed, 46 passed, 3 skipped, 37 warnings in 43.92s =============

prateekmukhedkar · 2023-03-01T00:35:09Z

/test connector=source-db2

🕑 source-db2 https://github.com/airbytehq/airbyte/actions/runs/4298753696
✅ source-db2 https://github.com/airbytehq/airbyte/actions/runs/4298753696
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
================= 19 passed, 6 skipped, 27 warnings in 15.74s ==================

prateekmukhedkar · 2023-03-01T00:35:14Z

/test connector=source-kafka

🕑 source-kafka https://github.com/airbytehq/airbyte/actions/runs/4298753985
✅ source-kafka https://github.com/airbytehq/airbyte/actions/runs/4298753985
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
================= 19 passed, 6 skipped, 27 warnings in 14.80s ==================

prateekmukhedkar · 2023-03-01T00:35:31Z

/test connector=source-mongodb-v2

🕑 source-mongodb-v2 https://github.com/airbytehq/airbyte/actions/runs/4298755961
❌ source-mongodb-v2 https://github.com/airbytehq/airbyte/actions/runs/4298755961
🐛 https://gradle.com/s/s7zcav2onwceu

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestConnection::test_check[inputs0] - AssertionError: as...
FAILED test_core.py::TestDiscovery::test_discover[inputs0] - docker.errors.Co...
ERROR test_core.py::TestDiscovery::test_defined_cursors_exist_in_schema[inputs0]
ERROR test_core.py::TestDiscovery::test_defined_refs_exist_in_schema[inputs0]
ERROR test_core.py::TestDiscovery::test_defined_keyword_exist_in_schema[inputs0-allOf]
ERROR test_core.py::TestDiscovery::test_defined_keyword_exist_in_schema[inputs0-not]
ERROR test_core.py::TestDiscovery::test_primary_keys_exist_in_schema[inputs0]
ERROR test_core.py::TestDiscovery::test_streams_has_sync_modes[inputs0] - doc...
ERROR test_core.py::TestDiscovery::test_additional_properties_is_true[inputs0]
ERROR test_core.py::TestDiscovery::test_backward_compatibility[inputs0] - doc...
ERROR test_core.py::TestBasicRead::test_read[inputs0] - docker.errors.Contain...
ERROR test_full_refresh.py::TestFullRefresh::test_sequential_reads[inputs0]
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
============= 2 failed, 21 passed, 1 skipped, 10 errors in 33.74s ==============

prateekmukhedkar · 2023-03-01T00:35:37Z

/test connector=source-mysql

🕑 source-mysql https://github.com/airbytehq/airbyte/actions/runs/4298756378
✅ source-mysql https://github.com/airbytehq/airbyte/actions/runs/4298756378
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
================= 19 passed, 6 skipped, 27 warnings in 16.35s ==================

prateekmukhedkar · 2023-03-01T00:35:42Z

/test connector=source-oracle

🕑 source-oracle https://github.com/airbytehq/airbyte/actions/runs/4298756833
✅ source-oracle https://github.com/airbytehq/airbyte/actions/runs/4298756833
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
================= 19 passed, 6 skipped, 27 warnings in 17.79s ==================

prateekmukhedkar · 2023-03-01T00:35:49Z

/test connector=source-redshift

🕑 source-redshift https://github.com/airbytehq/airbyte/actions/runs/4298757548
❌ source-redshift https://github.com/airbytehq/airbyte/actions/runs/4298757548
🐛 https://gradle.com/s/tkh3oefvchjhq

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_incremental.py::TestIncremental::test_two_sequential_reads[inputs0]
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:509: The previous and actual discovered catalogs are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_incremental.py:30: `future_state` not specified, skipping.
============= 1 failed, 32 passed, 3 skipped in 121.97s (0:02:01) ==============

prateekmukhedkar · 2023-03-01T00:35:54Z

/test connector=source-tidb

🕑 source-tidb https://github.com/airbytehq/airbyte/actions/runs/4298757878
✅ source-tidb https://github.com/airbytehq/airbyte/actions/runs/4298757878
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
================= 19 passed, 6 skipped, 27 warnings in 15.66s ==================

prateekmukhedkar · 2023-03-01T00:36:00Z

/test connector=source-firebolt

prateekmukhedkar · 2023-03-01T00:40:02Z

/test connector=source-azure-table

🕑 source-azure-table https://github.com/airbytehq/airbyte/actions/runs/4298782313
❌ source-azure-table https://github.com/airbytehq/airbyte/actions/runs/4298782313
🐛 https://gradle.com/s/olsyewjjka4w4

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs0] - docker.errors.Contai...
FAILED test_incremental.py::TestIncremental::test_two_sequential_reads[inputs0]
FAILED test_incremental.py::TestIncremental::test_read_sequential_slices[inputs0]
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:509: The previous and actual discovered catalogs are identical.
============ 3 failed, 32 passed, 2 skipped, 37 warnings in 37.24s =============

prateekmukhedkar · 2023-03-01T00:40:08Z

/test connector=source-clickhouse

🕑 source-clickhouse https://github.com/airbytehq/airbyte/actions/runs/4298783046
✅ source-clickhouse https://github.com/airbytehq/airbyte/actions/runs/4298783046
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
================= 19 passed, 6 skipped, 27 warnings in 16.36s ==================

…s-non-ga

…nges go through.

…github.com:airbytehq/airbyte into prateekmukhedkar/add-allowedhosts-dbsources-non-ga

prateekmukhedkar · 2023-03-08T00:08:58Z

/publish connector=connectors/source-snowflake,connectors/source-cockroachdb,source-db2,source-kafka,source-oracle,source-tidb,source-clickhouse

🕑 Publishing the following connectors:
connectors/source-snowflake,connectors/source-cockroachdb,source-db2,source-kafka,source-oracle,source-tidb,source-clickhouse
https://github.com/airbytehq/airbyte/actions/runs/4359754603

Connector	Did it publish?	Were definitions generated?
connectors/source-snowflake	✅	✅
connectors/source-cockroachdb	✅	✅
source-db2	❌	❌
source-kafka	❌	❌
source-oracle	❌	❌
source-tidb	❌	❌
source-clickhouse	❌	❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

prateekmukhedkar · 2023-03-08T00:34:44Z

/publish connector=connectors/source-db2,connectors/source-kafka,connectors/source-oracle,connectors/source-tidb,connectors/source-clickhouse

🕑 Publishing the following connectors:
connectors/source-db2,connectors/source-kafka,connectors/source-oracle,connectors/source-tidb,connectors/source-clickhouse
https://github.com/airbytehq/airbyte/actions/runs/4359913676

Connector	Did it publish?	Were definitions generated?
connectors/source-db2	✅	✅
connectors/source-clickhouse	✅	✅
connectors/source-kafka	❌	❌
connectors/source-oracle	✅	✅
connectors/source-tidb	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

…github.com:airbytehq/airbyte into prateekmukhedkar/add-allowedhosts-dbsources-non-ga

…s-non-ga

…github.com:airbytehq/airbyte into prateekmukhedkar/add-allowedhosts-dbsources-non-ga

prateekmukhedkar · 2023-03-08T02:23:58Z

/publish connector=connectors/source-oracle-strict-encrypt auto-bump-version=false

🕑 Publishing the following connectors:
connectors/source-oracle-strict-encrypt
https://github.com/airbytehq/airbyte/actions/runs/4360504435

Connector	Did it publish?	Were definitions generated?
connectors/source-oracle-strict-encrypt	✅	❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

prateekmukhedkar · 2023-03-08T02:24:07Z

/publish connector=connectors/source-db2-strict-encrypt auto-bump-version=false

🕑 Publishing the following connectors:
connectors/source-db2-strict-encrypt
https://github.com/airbytehq/airbyte/actions/runs/4360505487

Connector	Did it publish?	Were definitions generated?
connectors/source-db2-strict-encrypt	✅	❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

prateekmukhedkar · 2023-03-08T02:24:17Z

/publish connector=connectors/source-clickhouse-strict-encrypt auto-bump-version=false

🕑 Publishing the following connectors:
connectors/source-clickhouse-strict-encrypt
https://github.com/airbytehq/airbyte/actions/runs/4360506044

Connector	Did it publish?	Were definitions generated?
connectors/source-clickhouse-strict-encrypt	✅	❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

prateekmukhedkar · 2023-03-08T02:35:58Z

/publish connector=connectors/source-fauna,connectors/source-redshift,connectors/source-azure-table

🕑 Publishing the following connectors:
connectors/source-fauna,connectors/source-redshift,connectors/source-azure-table
https://github.com/airbytehq/airbyte/actions/runs/4360572274

Connector	Did it publish?	Were definitions generated?
connectors/source-azure-table	❌	❌
connectors/source-redshift	❌	❌
connectors/source-fauna	❌	❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

…e tests in master.

…ce tests on master.

prateekmukhedkar · 2023-03-08T03:38:21Z

The following DB source connectors were published:

source-clickhouse
source-clickhouse-strict-encrypt
source-cockroachdb
source-db2
source-db2-strict-encrypt
source-oracle
source-oracle-strict-encrypt
source-snowflake
source-tidb

…#23455) Adding allowedHosts to clickhouse, cockroachdb, db2, oracle, snowflake, tidb source Connectors. --------- Co-authored-by: prateekmukhedkar <prateekmukhedkar@users.noreply.github.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

#22372 Adding allowedHosts to all non GA source connectors

27d5229

prateekmukhedkar requested review from a team and evantahler February 24, 2023 22:19

Automated Change

91c1deb

evantahler approved these changes Feb 24, 2023

View reviewed changes

prateekmukhedkar and others added 5 commits February 24, 2023 15:54

Merge branch 'master' into prateekmukhedkar/add-allowedhosts-dbsource…

b7e09a3

…s-non-ga

Updated name of "File" source

0a7de2f

Automated Change

49a8a82

Merge branch 'master' into prateekmukhedkar/add-allowedhosts-dbsource…

0af62b4

…s-non-ga

1. Validting with source_specs.yaml on how host name is configured.

7ff4afa

2. Removing host names under allowed hosts for BigQuery, Azure Table Storage and MongoDB sources until we figure out what to configure

Merge branch 'master' into prateekmukhedkar/add-allowedhosts-dbsource…

2b80402

…s-non-ga

subodh1810 changed the title ~~https://github.com/airbytehq/airbyte/issues/22372 Adding allowHosts to all non-GA source database connectors~~ Adding allowHosts to all non-GA source database connectors Mar 1, 2023

subodh1810 approved these changes Mar 1, 2023

View reviewed changes

prateekmukhedkar and others added 2 commits March 6, 2023 15:59

Merge branch 'master' into prateekmukhedkar/add-allowedhosts-dbsource…

e03db0f

…s-non-ga

Bumped up version numbers and release notes.

422a502

prateekmukhedkar and others added 4 commits March 7, 2023 22:53

Automated Change

da2d157

Not releasing MySQL and MSSQL allowed host changes until Debezium cha…

b8ba970

…nges go through.

Merge branch 'prateekmukhedkar/add-allowedhosts-dbsources-non-ga' of …

6287322

…github.com:airbytehq/airbyte into prateekmukhedkar/add-allowedhosts-dbsources-non-ga

Fixed formatting, year (2023)

830e844

octavia-squidington-iii added 2 commits March 8, 2023 00:23

auto-bump connector version

5c2fef0

auto-bump connector version

2e27d8e

prateekmukhedkar mentioned this pull request Mar 8, 2023

Redshift DB Source : CAT failing #23854

Closed

octavia-squidington-iii and others added 9 commits March 8, 2023 01:05

auto-bump connector version

a247f8c

auto-bump connector version

9a05296

auto-bump connector version

55ebece

auto-bump connector version

e67118e

Not releasing allowedHost change for MySQL and MSSQL

f692e42

Merge branch 'prateekmukhedkar/add-allowedhosts-dbsources-non-ga' of …

d726a96

…github.com:airbytehq/airbyte into prateekmukhedkar/add-allowedhosts-dbsources-non-ga

Merge branch 'master' into prateekmukhedkar/add-allowedhosts-dbsource…

bb6345f

…s-non-ga

Bumped up version number for DB2 and Clickhouse strict-encrypt versions.

3d213a3

Merge branch 'prateekmukhedkar/add-allowedhosts-dbsources-non-ga' of …

33533ec

…github.com:airbytehq/airbyte into prateekmukhedkar/add-allowedhosts-dbsources-non-ga

octavia-squidington-iii added connectors/source/db2-strict-encrypt connectors/source/clickhouse-strict-encrypt labels Mar 8, 2023

prateekmukhedkar added 2 commits March 7, 2023 19:18

Removing Azure-table, Fauna and Redshift because of failing acceptanc…

1926279

…e tests in master.

Removing allowedHosts for Source Firebolt because of failing acceptan…

46092af

…ce tests on master.

prateekmukhedkar merged commit 33ca7f3 into master Mar 8, 2023

prateekmukhedkar deleted the prateekmukhedkar/add-allowedhosts-dbsources-non-ga branch March 8, 2023 03:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding allowHosts to all non-GA source database connectors #23455

Adding allowHosts to all non-GA source database connectors #23455

prateekmukhedkar commented Feb 24, 2023 •

edited

Loading

CLAassistant commented Feb 24, 2023 •

edited

Loading

evantahler left a comment •

edited

Loading

evantahler Feb 24, 2023

prateekmukhedkar Feb 25, 2023

evantahler Feb 24, 2023

evantahler Feb 24, 2023

prateekmukhedkar Feb 27, 2023

prateekmukhedkar commented Feb 28, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Feb 28, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 8, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 8, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 8, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 8, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 8, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 8, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 8, 2023

	allowedHosts:
	hosts:
	- "${host}"
	- "${tunnel_method.tunnel_host}"

	host:
	description: "The host domain of the snowflake instance (must include the\
	\ account, region, cloud environment, and end with snowflakecomputing.com)."
	examples:
	- "accountname.us-east-2.aws.snowflakecomputing.com"
	type: "string"
	title: "Account Name"
	order: 1

Adding allowHosts to all non-GA source database connectors #23455

Adding allowHosts to all non-GA source database connectors #23455

Conversation

prateekmukhedkar commented Feb 24, 2023 • edited Loading

What

How

Verification

Recommended reading order

CLAassistant commented Feb 24, 2023 • edited Loading

evantahler left a comment • edited Loading

Choose a reason for hiding this comment

evantahler Feb 24, 2023

Choose a reason for hiding this comment

prateekmukhedkar Feb 25, 2023

Choose a reason for hiding this comment

evantahler Feb 24, 2023

Choose a reason for hiding this comment

evantahler Feb 24, 2023

Choose a reason for hiding this comment

prateekmukhedkar Feb 27, 2023

Choose a reason for hiding this comment

prateekmukhedkar commented Feb 28, 2023 • edited by github-actions bot Loading

Build Passed

prateekmukhedkar commented Feb 28, 2023 • edited by github-actions bot Loading

Build Passed

prateekmukhedkar commented Mar 1, 2023 • edited by github-actions bot Loading

Build Passed

prateekmukhedkar commented Mar 1, 2023 • edited by github-actions bot Loading

Build Failed

prateekmukhedkar commented Mar 1, 2023 • edited by github-actions bot Loading

Build Passed

prateekmukhedkar commented Mar 1, 2023 • edited by github-actions bot Loading

Build Passed

prateekmukhedkar commented Mar 1, 2023 • edited by github-actions bot Loading

Build Failed

prateekmukhedkar commented Mar 1, 2023 • edited by github-actions bot Loading

Build Passed

prateekmukhedkar commented Mar 1, 2023 • edited by github-actions bot Loading

Build Passed

prateekmukhedkar commented Mar 1, 2023 • edited by github-actions bot Loading

Build Failed

prateekmukhedkar commented Mar 1, 2023 • edited by github-actions bot Loading

Build Passed

prateekmukhedkar commented Mar 1, 2023

prateekmukhedkar commented Mar 1, 2023 • edited by github-actions bot Loading

Build Failed

prateekmukhedkar commented Mar 1, 2023 • edited by github-actions bot Loading

Build Passed

prateekmukhedkar commented Mar 8, 2023 • edited by github-actions bot Loading

prateekmukhedkar commented Mar 8, 2023 • edited by github-actions bot Loading

prateekmukhedkar commented Mar 8, 2023 • edited by github-actions bot Loading

prateekmukhedkar commented Mar 8, 2023 • edited by github-actions bot Loading

prateekmukhedkar commented Mar 8, 2023 • edited by github-actions bot Loading

prateekmukhedkar commented Mar 8, 2023 • edited by github-actions bot Loading

prateekmukhedkar commented Mar 8, 2023

prateekmukhedkar commented Feb 24, 2023 •

edited

Loading

CLAassistant commented Feb 24, 2023 •

edited

Loading

evantahler left a comment •

edited

Loading

prateekmukhedkar commented Feb 28, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Feb 28, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 1, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 8, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 8, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 8, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 8, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 8, 2023 •

edited by github-actions bot

Loading

prateekmukhedkar commented Mar 8, 2023 •

edited by github-actions bot

Loading