feat: Filter out system views out of system namespaces #22221

sergio-ropero · 2023-02-01T17:45:34Z

This changes allows to filter out system views created out of system namespaces.
Issue: #1323

As the only impacted DB is Postgres, we will bump only that image.

TEST

Current version (1.0.42)

PR version (dev)

…namespaces

github-actions · 2023-02-01T17:48:01Z

Affected Connector Report

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to do the following as needed:

Run integration tests
Bump connector or module version
Add changelog
Publish the new version

✅ Sources (25)

Connector	Version	Changelog	Publish
`source-alloydb`	`1.0.36`	✅	✅
`source-alloydb-strict-encrypt`	`1.0.36`	🔵 (ignored)	🔵 (ignored)
`source-bigquery`	`0.2.3`	✅	✅
`source-clickhouse`	`0.1.15`	✅	✅
`source-clickhouse-strict-encrypt`	`0.1.15`	🔵 (ignored)	🔵 (ignored)
`source-cockroachdb`	`0.1.19`	✅	✅
`source-cockroachdb-strict-encrypt`	`0.1.19`	🔵 (ignored)	🔵 (ignored)
`source-db2`	`0.1.17`	✅	✅
`source-db2-strict-encrypt`	`0.1.17`	🔵 (ignored)	🔵 (ignored)
`source-dynamodb`	`0.1.0`	✅	✅
`source-jdbc`	`0.3.5`	🔵 (ignored)	🔵 (ignored)
`source-mongodb-strict-encrypt`	`0.1.19`	🔵 (ignored)	🔵 (ignored)
`source-mongodb-v2`	`0.1.19`	✅	✅
`source-mssql`	`0.4.28`	✅	✅
`source-mssql-strict-encrypt`	`0.4.28`	🔵 (ignored)	🔵 (ignored)
`source-mysql`	`1.0.21`	✅	✅
`source-mysql-strict-encrypt`	`1.0.21`	🔵 (ignored)	🔵 (ignored)
`source-oracle`	`0.3.22`	✅	✅
`source-oracle-strict-encrypt`	`0.3.22`	🔵 (ignored)	🔵 (ignored)
`source-postgres`	`1.0.43`	✅	✅
`source-postgres-strict-encrypt`	`1.0.43`	🔵 (ignored)	🔵 (ignored)
`source-redshift`	`0.3.16`	✅	✅
`source-scaffold-java-jdbc`	`0.1.0`	🔵 (ignored)	🔵 (ignored)
`source-snowflake`	`0.1.29`	✅	✅
`source-tidb`	`0.2.2`	✅	✅

See "Actionable Items" below for how to resolve warnings and errors.

✅ Destinations (0)

Connector	Version	Changelog	Publish

See "Actionable Items" below for how to resolve warnings and errors.

✅ Other Modules (0)

Actionable Items

(click to expand)

Category	Status	Actionable Item
Version	❌ mismatch	The version of the connector is different from its normal variant. Please bump the version of the connector.
	⚠ doc not found	The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like `source-jdbc` is not published or documented). Please double-check to make sure that it is not a bug.
Changelog	⚠ doc not found	The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like `source-jdbc` is not published or documented). Please double-check to make sure that it is not a bug.
	❌ changelog missing	There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog.
Publish	⚠ not in seed	The connector is not in the seed file (e.g. `source_definitions.yaml`), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug.
	❌ diff seed version	The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the `/publish` command to publish the latest version.

github-actions · 2023-02-01T17:56:40Z

Airbyte Code Coverage

There is no coverage information present for the Files changed

Total Project Coverage	24.52%	❌

github-actions · 2023-02-01T19:20:19Z

Platform Test Results

  235 files -   8   235 suites - 8 5m 22s ⏱️ - 19m 4s
1 619 tests - 44 1 614 ✔️ - 38 5 💤 - 6 0 ❌ ±0
1 634 runs - 48 1 629 ✔️ - 42 5 💤 - 6 0 ❌ ±0

Results for commit e25df94. ± Comparison against base commit 6a10ae3.

♻️ This comment has been updated with latest results.

…es' into feat/exclude-system-views-postgres

sergio-ropero · 2023-02-02T14:21:36Z

/test connector=connectors/source-postgres

🕑 connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/4075414056
✅ connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/4075414056
Python tests coverage:

	 Name                                                    Stmts   Miss  Cover   Missing
	 -------------------------------------------------------------------------------------
	 connector_acceptance_test/base.py                          12      4    67%   16-19
	 connector_acceptance_test/config.py                       141      5    96%   87, 93, 239, 243-244
	 connector_acceptance_test/conftest.py                     211     95    55%   36, 42-44, 49, 54, 77, 83, 89-91, 110, 115-117, 123-125, 131-132, 137-138, 143, 149, 158-167, 173-178, 193, 217, 248, 254, 262-267, 275-285, 293-306, 311-317, 324-335, 342-358
	 connector_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 connector_acceptance_test/tests/test_core.py              476    117    75%   53, 58, 97-108, 113-120, 124-125, 129-130, 380, 400, 438, 476-493, 506-517, 521-526, 532, 565-570, 608-615, 658-660, 663, 728-736, 748-751, 756, 812-813, 819, 822, 858-868, 881-906
	 connector_acceptance_test/tests/test_incremental.py       160     14    91%   58-65, 70-83, 246
	 connector_acceptance_test/utils/asserts.py                 39      2    95%   62-63
	 connector_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 connector_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 connector_acceptance_test/utils/connector_runner.py       133     33    75%   24-27, 46-47, 50-54, 57-58, 73-75, 78-80, 83-85, 88-90, 93-95, 124-125, 159-161, 208
	 connector_acceptance_test/utils/json_schema_helper.py     114     13    89%   31-32, 39, 42, 66-69, 97, 121, 203-205
	 -------------------------------------------------------------------------------------
	 TOTAL                                                    1690    341    80%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [2] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:98: The previous and actual specifications are identical.
SKIPPED [2] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:507: The previous and actual discovered catalogs are identical.
SKIPPED [2] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_incremental.py:22: `future_state` has a bypass reason, skipping.
================== 66 passed, 6 skipped in 717.84s (0:11:57) ===================

sergio-ropero · 2023-02-02T14:22:06Z

/test connector=connectors/source-postgres-strict-encrypt

🕑 connectors/source-postgres-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/4075418950
✅ connectors/source-postgres-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/4075418950
No Python unittests run

Build Passed

Test summary info:

All Passed

This reverts commit 3bf87a9.

…" (#22425) This reverts commit 3bf87a9.

This changes allows to filter out system views created out of system …

b31b7b8

…namespaces

octavia-squidington-iv added area/connectors Connector related issues connectors/source/bigquery connectors/source/postgres connectors/source/mongodb-v2 connectors/source/jdbc connectors/source/relational-db labels Feb 1, 2023

sergio-ropero temporarily deployed to more-secrets February 1, 2023 17:47 — with GitHub Actions Inactive

Add extra view

6a9460b

sergio-ropero temporarily deployed to more-secrets February 1, 2023 17:49 — with GitHub Actions Inactive

sergio-ropero marked this pull request as ready for review February 2, 2023 14:04

sergio-ropero requested a review from a team as a code owner February 2, 2023 14:04

Merge branch 'master' into feat/exclude-system-views-postgres

e25df94

sergio-ropero temporarily deployed to more-secrets February 2, 2023 14:06 — with GitHub Actions Inactive

Sergio Ropero added 2 commits February 2, 2023 15:20

Fix issue

c617d47

Merge remote-tracking branch 'origin/feat/exclude-system-views-postgr…

f3140e6

…es' into feat/exclude-system-views-postgres

sergio-ropero temporarily deployed to more-secrets February 2, 2023 14:22 — with GitHub Actions Inactive

sergio-ropero temporarily deployed to more-secrets February 2, 2023 14:23 — with GitHub Actions Inactive

subodh1810 approved these changes Feb 6, 2023

View reviewed changes

sergio-ropero temporarily deployed to more-secrets February 6, 2023 17:17 — with GitHub Actions Inactive

Merge branch 'master' into feat/exclude-system-views-postgres

666fa2a

Bump Postgres source version

7394465

octavia-squidington-iii added connectors/source/postgres-strict-encrypt area/documentation Improvements or additions to documentation labels Feb 6, 2023

sergio-ropero temporarily deployed to more-secrets February 6, 2023 17:31 — with GitHub Actions Inactive

sergio-ropero merged commit 3bf87a9 into master Feb 6, 2023

sergio-ropero deleted the feat/exclude-system-views-postgres branch February 6, 2023 18:10

sergio-ropero restored the feat/exclude-system-views-postgres branch February 6, 2023 18:22

sergio-ropero added a commit that referenced this pull request Feb 6, 2023

Revert "feat: Filter out system views out of system namespaces (#22221)"

68b04d5

This reverts commit 3bf87a9.

sergio-ropero mentioned this pull request Feb 6, 2023

Revert "feat: Filter out system views out of system namespaces" #22425

Merged

subodh1810 pushed a commit that referenced this pull request Feb 6, 2023

Revert "feat: Filter out system views out of system namespaces (#22221)…

b87647d

…" (#22425) This reverts commit 3bf87a9.

sergio-ropero mentioned this pull request Feb 6, 2023

feat: Filter out system views out of system namespaces #22427

Merged

github-actions bot mentioned this pull request Feb 7, 2023

Bump helm chart version reference to 0.43.28 #22489

Closed

This was referenced Feb 8, 2023

Bump Airbyte version from 0.40.32 to 0.40.33 #22578

Closed

Bump Airbyte version from 0.40.32 to 0.40.33 #22689

Closed

Bump Airbyte version from 0.40.32 to 0.40.33 #22839

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Filter out system views out of system namespaces #22221

feat: Filter out system views out of system namespaces #22221

sergio-ropero commented Feb 1, 2023 •

edited

Loading

github-actions bot commented Feb 1, 2023 •

edited

Loading

✅ Sources (25)

✅ Destinations (0)

✅ Other Modules (0)

Actionable Items

github-actions bot commented Feb 1, 2023 •

edited

Loading

github-actions bot commented Feb 1, 2023 •

edited

Loading

sergio-ropero commented Feb 2, 2023 •

edited by github-actions bot

Loading

sergio-ropero commented Feb 2, 2023 •

edited by github-actions bot

Loading

feat: Filter out system views out of system namespaces #22221

feat: Filter out system views out of system namespaces #22221

Conversation

sergio-ropero commented Feb 1, 2023 • edited Loading

TEST

Current version (1.0.42)

PR version (dev)

github-actions bot commented Feb 1, 2023 • edited Loading

Affected Connector Report

✅ Sources (25)

✅ Destinations (0)

✅ Other Modules (0)

Actionable Items

github-actions bot commented Feb 1, 2023 • edited Loading

Airbyte Code Coverage

github-actions bot commented Feb 1, 2023 • edited Loading

Platform Test Results

sergio-ropero commented Feb 2, 2023 • edited by github-actions bot Loading

Build Passed

sergio-ropero commented Feb 2, 2023 • edited by github-actions bot Loading

Build Passed

sergio-ropero commented Feb 1, 2023 •

edited

Loading

github-actions bot commented Feb 1, 2023 •

edited

Loading

github-actions bot commented Feb 1, 2023 •

edited

Loading

github-actions bot commented Feb 1, 2023 •

edited

Loading

sergio-ropero commented Feb 2, 2023 •

edited by github-actions bot

Loading

sergio-ropero commented Feb 2, 2023 •

edited by github-actions bot

Loading