Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(clickhouse-driver): Switch from apla-clickhouse to @clickhouse/client #8928

Merged
merged 2 commits into from
Nov 19, 2024

Conversation

mcheshkov
Copy link
Member

@mcheshkov mcheshkov commented Nov 7, 2024

Check List

  • Tests has been run in packages where changes made if available
  • Linter has been run for changed code
  • Tests for the changes have been added if not covered yet
  • Docs have been added / updated if required

Description of Changes Made (if issue reference is not provided)

Switch client library used to talk to ClickHouse to upstream one. Lots of related changes:

  • Streaming now does not use streaming JSON parser, so we can't rely on meta field in JSON format. Instread it relies on JSONCompactEachRowWithNamesAndTypes: first two rows returned should contain names and types. https://clickhouse.com/docs/en/sql-reference/formats#jsoncompacteachrowwithnamesandtypes
  • Streaming now use async iterators instread of Node.js streams internally. External API returns stream, as before
  • Pooling moved completely to client library. generic-pool is not used at all, dbMaxPoolSize is passed to client library to limit open sockets. New client maintains http.Agent internally, and have it's own idle timers, looks fine for us.
  • Queries now does not send session_id, as we anyway expect queries to be independent, and don't use session-bound stuff, like temporary tables. Previous behaviour was kind of weird: session ids were attached to client in pool, but for every query it would pull acquire new client from pool, so nothing could actually utilize same session.
  • KILL QUERY on cancellation now uses separate client instance, to avoid getting stuck on busy pool
  • query method supports only SELECT queries, or other queries that return result sets. For DDL queries on this client library one have to use other methods. Because of that more overrides were necessary, like dropTable, createSchemaIfNotExists or createTable.
  • Driver now respects per-datasource dbQueryTimeout config

Copy link

vercel bot commented Nov 7, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

8 Skipped Deployments
Name Status Preview Comments Updated (UTC)
examples-angular-dashboard ⬜️ Ignored (Inspect) Visit Preview Nov 19, 2024 5:16pm
examples-react-d3 ⬜️ Ignored (Inspect) Visit Preview Nov 19, 2024 5:16pm
examples-react-dashboard ⬜️ Ignored (Inspect) Visit Preview Nov 19, 2024 5:16pm
examples-react-data-table ⬜️ Ignored (Inspect) Visit Preview Nov 19, 2024 5:16pm
examples-react-highcharts ⬜️ Ignored (Inspect) Visit Preview Nov 19, 2024 5:16pm
examples-react-material-ui ⬜️ Ignored (Inspect) Visit Preview Nov 19, 2024 5:16pm
examples-react-pivot-table ⬜️ Ignored (Inspect) Visit Preview Nov 19, 2024 5:16pm
examples-vue-query-builder ⬜️ Ignored (Inspect) Visit Preview Nov 19, 2024 5:16pm

@mcheshkov mcheshkov changed the title [WIP] feat(clickhouse-driver): Switch from apla-clickhouse to @clickhouse/client feat(clickhouse-driver): Switch from apla-clickhouse to @clickhouse/client Nov 19, 2024
@mcheshkov mcheshkov marked this pull request as ready for review November 19, 2024 17:57
@mcheshkov mcheshkov requested review from a team as code owners November 19, 2024 17:57
Copy link
Member

@ovr ovr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🔥

@mcheshkov mcheshkov merged commit e25e65f into master Nov 19, 2024
57 checks passed
@mcheshkov mcheshkov deleted the clickhouse-switch-driver branch November 19, 2024 21:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants