Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Destination TiDB: install normalization #27025

Merged
merged 4 commits into from
Jun 6, 2023
Merged

Conversation

edgao
Copy link
Contributor

@edgao edgao commented Jun 5, 2023

closes #25501

Install normalization into tidb. Verified that spec correctly uses run_with_normalization.sh

image

Didn't run a local sync though. I don't think that's necessary, given that this will be the 4th connector with this exact change.

... I also just realized that tidb isn't even on cloud, so arguably this has no value. Either way, it was a fast pr and I think we can merge it aggressively.


also add us as codeowners on this connector for consistency with the rest of the db destinations.

@edgao

This comment was marked as outdated.

@github-actions
Copy link
Contributor

github-actions bot commented Jun 5, 2023

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

  • PR name follows PR naming conventions
  • Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan and you've followed all steps in the Breaking Changes Checklist
  • Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
  • Secrets in the connector's spec are annotated with airbyte_secret
  • All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
  • Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
  • You, or an Airbyter, have run /test successfully on this PR - or on a non-forked branch
  • You've updated the connector's metadata.yaml file (new!)

If the checklist is complete, but the CI check is failing,

  1. Check for hidden checklists in your PR description

  2. Toggle the github label checklist-action-run on/off to re-run the checklist CI.

@github-actions
Copy link
Contributor

github-actions bot commented Jun 5, 2023

Affected Connector Report

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to do the following as needed:

  • Run integration tests
  • Bump connector or module version
  • Add changelog
  • Publish the new version

✅ Sources (0)

Connector Version Changelog Publish
  • See "Actionable Items" below for how to resolve warnings and errors.

❌ Destinations (1)

Connector Version Changelog Publish
destination-tidb 0.1.3
(diff seed version)
  • See "Actionable Items" below for how to resolve warnings and errors.

👀 Other Modules (1)

  • base-normalization

Actionable Items

(click to expand)

Category Status Actionable Item
Version
mismatch
The version of the connector is different from its normal variant. Please bump the version of the connector.

doc not found
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.
Changelog
doc not found
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.

changelog missing
There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog.
Publish
not in seed
The connector is not in the cloud or oss registry, so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that you have added a metadata.yaml file and the expected registries are enabled.

@octavia-squidington-iii octavia-squidington-iii added the area/documentation Improvements or additions to documentation label Jun 5, 2023
@edgao
Copy link
Contributor Author

edgao commented Jun 5, 2023

/test connector=connectors/destination-tidb

🕑 connectors/destination-tidb https://github.com/airbytehq/airbyte/actions/runs/5180225701
✅ connectors/destination-tidb https://github.com/airbytehq/airbyte/actions/runs/5180225701
Python tests coverage:

Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
normalization/transform_config/__init__.py                            2      0   100%
normalization/transform_catalog/reserved_keywords.py                 15      0   100%
normalization/transform_catalog/__init__.py                           2      0   100%
normalization/destination_type.py                                    18      0   100%
normalization/__init__.py                                             4      0   100%
normalization/transform_catalog/destination_name_transformer.py     171     10    94%
normalization/transform_catalog/table_name_registry.py              174     34    80%
normalization/transform_config/transform.py                         195     48    75%
normalization/transform_catalog/utils.py                             51     14    73%
normalization/transform_catalog/dbt_macro.py                         22      7    68%
normalization/transform_catalog/catalog_processor.py                147     80    46%
normalization/transform_catalog/transform.py                         65     39    40%
normalization/transform_catalog/stream_processor.py                 603    407    33%
-------------------------------------------------------------------------------------
TOTAL                                                              1469    639    57%

Build Passed

Test summary info:

All Passed

@edgao edgao marked this pull request as ready for review June 5, 2023 18:37
@edgao edgao requested a review from a team June 5, 2023 23:37
@edgao edgao enabled auto-merge (squash) June 5, 2023 23:54
@edgao edgao merged commit ae43522 into master Jun 6, 2023
@edgao edgao deleted the edgao/normalization/tidb branch June 6, 2023 00:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation checklist-action-run connectors/destination/tidb
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement in-destination normalization for tidb
3 participants