Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 Destination Clickhouse : Normalization doesn't work in the acceptance tests. #12582

Closed
DoNotPanicUA opened this issue May 4, 2022 · 1 comment · Fixed by #14783
Closed
Assignees
Labels
type/bug Something isn't working

Comments

@DoNotPanicUA
Copy link
Contributor

Current Behavior

The acceptance tests don't check any cases with normalization due to some problems with port sharing inside the container.
All tests with normalization are disabled. Also, we run DAT tests without normalization due to the false value in the supportsNormalization method.

Screenshot from 2022-05-04 22-26-36

Expected Behavior

Run all test cases using normalization.

@etsybaev
Copy link
Contributor

etsybaev commented Jun 17, 2022

There are 2 ports to connect to clickhouse:
8123 - HTTP, which is used by JDBC driver, that’s used in clickhouse destination
9000 - TCP, which is used by DBT, that’s used in clickhouse normalization
The difficulty is that you can only specify one port in the config.

So...in destination-clickhouse spec.json, we could have both port fields:
HTTP port, default 8123 for the clickhouse destination connector
TCP port, default 9000 for clickhouse normalization.

Once this is done, also need to update the java test and python's normalization config for click house airbyte-integrations/bases/base-normalization/normalization/transform_config/transform.py -> transform_clickhouse method.
Also need to bump Normalization's version in NormalizationRunnerFactory -> NORMALIZATION_VERSION

  • NormalizationRunnerFactory

(https://github.com/airbytehq/airbyte/pull/7620/files#r779428143)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Something isn't working
Projects
No open projects
Status: Ready for implementation (prioritized)
Development

Successfully merging a pull request may close this issue.

4 participants