Create a DbSourceDiscoverUtil for common utilities #23929

akashkulk · 2023-03-10T04:59:18Z

Creating a Util class for common discover utilities
Part of : #23808

akashkulk · 2023-03-10T04:59:47Z

/test connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/4381477592
✅ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/4381477592
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
================= 19 passed, 6 skipped, 27 warnings in 17.23s ==================

akashkulk · 2023-03-10T04:59:55Z

/test connector=connectors/source-postgres

🕑 connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/4381478077
✅ connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/4381478077
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [2] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:109: Backward compatibility tests are disabled for version 1.0.52.
SKIPPED [2] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:509: The previous and actual discovered catalogs are identical.
=================== 62 passed, 5 skipped in 85.54s (0:01:25) ===================

akashkulk · 2023-03-10T05:00:02Z

/test connector=connectors/source-bigquery

🕑 connectors/source-bigquery https://github.com/airbytehq/airbyte/actions/runs/4381479112
❌ connectors/source-bigquery https://github.com/airbytehq/airbyte/actions/runs/4381479112
🐛 https://gradle.com/s/jgg2tc4yzyckc

Build Failed

Test summary info:

Could not find result summary

akashkulk · 2023-03-10T05:00:24Z

/test connector=connectors/source-mongodb-v2

🕑 connectors/source-mongodb-v2 https://github.com/airbytehq/airbyte/actions/runs/4381481490
❌ connectors/source-mongodb-v2 https://github.com/airbytehq/airbyte/actions/runs/4381481490
🐛 https://gradle.com/s/m5gz3l5ut6bge

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestConnection::test_check[inputs0] - AssertionError: as...
FAILED test_core.py::TestDiscovery::test_discover[inputs0] - docker.errors.Co...
ERROR test_core.py::TestDiscovery::test_defined_cursors_exist_in_schema[inputs0]
ERROR test_core.py::TestDiscovery::test_defined_refs_exist_in_schema[inputs0]
ERROR test_core.py::TestDiscovery::test_defined_keyword_exist_in_schema[inputs0-allOf]
ERROR test_core.py::TestDiscovery::test_defined_keyword_exist_in_schema[inputs0-not]
ERROR test_core.py::TestDiscovery::test_primary_keys_exist_in_schema[inputs0]
ERROR test_core.py::TestDiscovery::test_streams_has_sync_modes[inputs0] - doc...
ERROR test_core.py::TestDiscovery::test_additional_properties_is_true[inputs0]
ERROR test_core.py::TestDiscovery::test_backward_compatibility[inputs0] - doc...
ERROR test_core.py::TestBasicRead::test_read[inputs0] - docker.errors.Contain...
ERROR test_full_refresh.py::TestFullRefresh::test_sequential_reads[inputs0]
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
============= 2 failed, 21 passed, 1 skipped, 10 errors in 29.94s ==============

github-actions · 2023-03-10T05:02:38Z

Affected Connector Report

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to do the following as needed:

Run integration tests
Bump connector or module version
Add changelog
Publish the new version

✅ Sources (25)

Connector	Version	Changelog	Publish
`source-alloydb`	`2.0.2`	✅	✅
`source-alloydb-strict-encrypt`	`2.0.2`	🔵 (ignored)	🔵 (ignored)
`source-bigquery`	`0.2.3`	✅	✅
`source-clickhouse`	`0.1.16`	✅	✅
`source-clickhouse-strict-encrypt`	`0.1.16`	🔵 (ignored)	🔵 (ignored)
`source-cockroachdb`	`0.1.20`	✅	✅
`source-cockroachdb-strict-encrypt`	`0.1.20`	🔵 (ignored)	🔵 (ignored)
`source-db2`	`0.1.18`	✅	✅
`source-db2-strict-encrypt`	`0.1.18`	🔵 (ignored)	🔵 (ignored)
`source-dynamodb`	`0.1.2`	✅	✅
`source-jdbc`	`0.3.5`	🔵 (ignored)	🔵 (ignored)
`source-mongodb-strict-encrypt`	`0.1.19`	🔵 (ignored)	🔵 (ignored)
`source-mongodb-v2`	`0.1.19`	✅	✅
`source-mssql`	`1.0.0`	✅	✅
`source-mssql-strict-encrypt`	`1.0.0`	🔵 (ignored)	🔵 (ignored)
`source-mysql`	`2.0.0`	✅	✅
`source-mysql-strict-encrypt`	`2.0.0`	🔵 (ignored)	🔵 (ignored)
`source-oracle`	`0.3.23`	✅	✅
`source-oracle-strict-encrypt`	`0.3.23`	🔵 (ignored)	🔵 (ignored)
`source-postgres`	`2.0.2`	✅	✅
`source-postgres-strict-encrypt`	`2.0.2`	🔵 (ignored)	🔵 (ignored)
`source-redshift`	`0.3.16`	✅	✅
`source-scaffold-java-jdbc`	`0.1.0`	🔵 (ignored)	🔵 (ignored)
`source-snowflake`	`0.1.31`	✅	✅
`source-tidb`	`0.2.3`	✅	✅

See "Actionable Items" below for how to resolve warnings and errors.

✅ Destinations (0)

Connector	Version	Changelog	Publish

See "Actionable Items" below for how to resolve warnings and errors.

✅ Other Modules (0)

Actionable Items

(click to expand)

Category	Status	Actionable Item
Version	❌ mismatch	The version of the connector is different from its normal variant. Please bump the version of the connector.
	⚠ doc not found	The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like `source-jdbc` is not published or documented). Please double-check to make sure that it is not a bug.
Changelog	⚠ doc not found	The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like `source-jdbc` is not published or documented). Please double-check to make sure that it is not a bug.
	❌ changelog missing	There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog.
Publish	⚠ not in seed	The connector is not in the seed file (e.g. `source_definitions.yaml`), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug.
	❌ diff seed version	The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the `/publish` command to publish the latest version.

akashkulk · 2023-03-14T17:20:42Z

/test connector=connectors/source-postgres

🕑 connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/4418458146
✅ connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/4418458146
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [2] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [2] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:509: The previous and actual discovered catalogs are identical.
=================== 62 passed, 5 skipped in 83.81s (0:01:23) ===================

akashkulk · 2023-03-14T21:54:01Z

/test connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/4420533591
✅ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/4420533591
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
================= 19 passed, 6 skipped, 27 warnings in 17.48s ==================

Create a DbSourceDiscoverUtil for common utilities

c4abb3d

octavia-squidington-iii added area/connectors Connector related issues connectors/source/relational-db labels Mar 10, 2023

akashkulk added 2 commits March 10, 2023 16:32

Merge branch 'master' into refac_util

59c1ba2

Merge branch 'master' into refac_util

01a15e9

akashkulk marked this pull request as ready for review March 14, 2023 15:12

akashkulk requested a review from rodireich March 14, 2023 15:12

Merge branch 'master' into refac_util

7871462

rodireich approved these changes Mar 14, 2023

View reviewed changes

akashkulk merged commit 16e81b7 into master Mar 14, 2023

akashkulk deleted the refac_util branch March 14, 2023 22:49

adriennevermorel pushed a commit to adriennevermorel/airbyte that referenced this pull request Mar 17, 2023

Create a DbSourceDiscoverUtil for common utilities (airbytehq#23929)

c8948d4

erohmensing pushed a commit that referenced this pull request Mar 22, 2023

Create a DbSourceDiscoverUtil for common utilities (#23929)

acd0c50

erohmensing pushed a commit that referenced this pull request Mar 22, 2023

Create a DbSourceDiscoverUtil for common utilities (#23929)

e22e44a

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create a DbSourceDiscoverUtil for common utilities #23929

Create a DbSourceDiscoverUtil for common utilities #23929

akashkulk commented Mar 10, 2023 •

edited

Loading

akashkulk commented Mar 10, 2023 •

edited by github-actions bot

Loading

akashkulk commented Mar 10, 2023 •

edited by github-actions bot

Loading

akashkulk commented Mar 10, 2023 •

edited by github-actions bot

Loading

akashkulk commented Mar 10, 2023 •

edited by github-actions bot

Loading

github-actions bot commented Mar 10, 2023 •

edited

Loading

✅ Sources (25)

✅ Destinations (0)

✅ Other Modules (0)

Actionable Items

akashkulk commented Mar 14, 2023 •

edited by github-actions bot

Loading

akashkulk commented Mar 14, 2023 •

edited by github-actions bot

Loading

Create a DbSourceDiscoverUtil for common utilities #23929

Create a DbSourceDiscoverUtil for common utilities #23929

Conversation

akashkulk commented Mar 10, 2023 • edited Loading

akashkulk commented Mar 10, 2023 • edited by github-actions bot Loading

Build Passed

akashkulk commented Mar 10, 2023 • edited by github-actions bot Loading

Build Passed

akashkulk commented Mar 10, 2023 • edited by github-actions bot Loading

Build Failed

akashkulk commented Mar 10, 2023 • edited by github-actions bot Loading

Build Failed

github-actions bot commented Mar 10, 2023 • edited Loading

Affected Connector Report

✅ Sources (25)

✅ Destinations (0)

✅ Other Modules (0)

Actionable Items

akashkulk commented Mar 14, 2023 • edited by github-actions bot Loading

Build Passed

akashkulk commented Mar 14, 2023 • edited by github-actions bot Loading

Build Passed

akashkulk commented Mar 10, 2023 •

edited

Loading

akashkulk commented Mar 10, 2023 •

edited by github-actions bot

Loading

akashkulk commented Mar 10, 2023 •

edited by github-actions bot

Loading

akashkulk commented Mar 10, 2023 •

edited by github-actions bot

Loading

akashkulk commented Mar 10, 2023 •

edited by github-actions bot

Loading

github-actions bot commented Mar 10, 2023 •

edited

Loading

akashkulk commented Mar 14, 2023 •

edited by github-actions bot

Loading

akashkulk commented Mar 14, 2023 •

edited by github-actions bot

Loading