YQ-3493 support resource pool classifiers kqp_proxy cache #7688

GrigoriyPA · 2024-08-13T04:40:09Z

Changelog entry

Support resource pool classifiers cache in kqp_proxy

Changelog category

New feature

Additional information

github-actions · 2024-08-13T04:43:12Z

⚪ 2024-08-13 04:43:11 UTC Pre-commit check for 59be7f0 has started.
⚪ 2024-08-13 04:45:54 UTC Check linux-x86_64-release-clang14 is running...
🔴 2024-08-13 04:49:42 UTC Build failed. see the logs.

github-actions · 2024-08-13T04:43:38Z

⚪ 2024-08-13 04:43:37 UTC Pre-commit check for 59be7f0 has started.
⚪ 2024-08-13 04:46:31 UTC Check linux-x86_64-relwithdebinfo is running...
🔴 2024-08-13 04:54:55 UTC Build failed. see the logs.

github-actions · 2024-08-13T04:43:38Z

⚪ 2024-08-13 04:43:38 UTC Pre-commit check for 59be7f0 has started.
⚪ 2024-08-13 04:46:25 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-13 04:53:24 UTC Build failed. see the logs.

github-actions · 2024-08-13T06:36:34Z

⚪ 2024-08-13 06:36:34 UTC Pre-commit check for 9006b1c has started.
⚪ 2024-08-13 06:39:22 UTC Check linux-x86_64-release-asan is running...
⚫ 2024-08-13 07:02:04 UTC Check cancelled

github-actions · 2024-08-13T06:37:06Z

⚪ 2024-08-13 06:37:06 UTC Pre-commit check for 9006b1c has started.
⚪ 2024-08-13 06:39:59 UTC Check linux-x86_64-release-clang14 is running...
🔴 2024-08-13 06:44:10 UTC Build failed. see the logs.

github-actions · 2024-08-13T06:39:50Z

⚪ 2024-08-13 06:39:50 UTC Pre-commit check for 9006b1c has started.
⚪ 2024-08-13 06:42:44 UTC Check linux-x86_64-relwithdebinfo is running...
⚫ 2024-08-13 07:02:13 UTC Check cancelled

github-actions · 2024-08-13T07:04:25Z

⚪ 2024-08-13 07:04:25 UTC Pre-commit check for e14efb4 has started.
⚪ 2024-08-13 07:07:13 UTC Check linux-x86_64-relwithdebinfo is running...
⚫ 2024-08-13 08:17:41 UTC Check cancelled

github-actions · 2024-08-13T07:05:53Z

⚪ 2024-08-13 07:05:53 UTC Pre-commit check for e14efb4 has started.
⚪ 2024-08-13 07:08:43 UTC Check linux-x86_64-release-clang14 is running...
🔴 2024-08-13 07:27:58 UTC Build failed. see the logs.

github-actions · 2024-08-13T07:05:56Z

⚪ 2024-08-13 07:05:56 UTC Pre-commit check for e14efb4 has started.
⚪ 2024-08-13 07:08:39 UTC Check linux-x86_64-release-asan is running...
⚫ 2024-08-13 08:17:42 UTC Check cancelled

github-actions · 2024-08-13T08:21:00Z

⚪ 2024-08-13 08:21:00 UTC Pre-commit check for 1e08c25 has started.
⚪ 2024-08-13 08:23:44 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-13 09:53:09 UTC Some tests failed, follow the links below. Going to retry failed tests...

Details

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
39805	34435	0	49	5308	13

⚪ 2024-08-13 09:57:45 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-08-13 10:05:40 UTC Some tests failed, follow the links below. Going to retry failed tests...

Details

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
64 (only retried tests)	7	0	48	0	9

⚪ 2024-08-13 10:05:48 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🔴 2024-08-13 10:12:53 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
62 (only retried tests)	5	0	48	0	9

🟢 2024-08-13 10:13:00 UTC Build successful.
🔴 2024-08-13 10:13:35 UTC ydbd size 8.1 GiB changed* by +2.2 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash	main: `5ccb818`	merge: `1e08c25`	diff	diff %
ydbd size	8 700 411 808 Bytes	8 702 693 984 Bytes	+2.2 MiB	+0.026%
ydbd stripped size	473 084 488 Bytes	473 128 328 Bytes	+42.8 KiB	+0.009%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-08-13T08:21:22Z

⚪ 2024-08-13 08:21:22 UTC Pre-commit check for 1e08c25 has started.
⚪ 2024-08-13 08:24:15 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-13 11:00:05 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10989	10928	0	23	25	13

🟢 2024-08-13 11:01:22 UTC Build successful.
🟡 2024-08-13 11:01:51 UTC ydbd size 5.5 GiB changed* by +1.2 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `5ccb818`	merge: `1e08c25`	diff	diff %
ydbd size	5 851 252 280 Bytes	5 852 557 896 Bytes	+1.2 MiB	+0.022%
ydbd stripped size	1 469 332 688 Bytes	1 469 527 984 Bytes	+190.7 KiB	+0.013%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-08-13T08:21:27Z

⚪ 2024-08-13 08:21:27 UTC Pre-commit check for 1e08c25 has started.
⚪ 2024-08-13 08:24:15 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-13 08:31:22 UTC Build successful.

gridnevvvit · 2024-08-13T09:29:11Z

ydb/core/kqp/workload_service/ut/kqp_workload_service_ut.cpp


-        // Check pool creation after cleanup
-        TSampleQueries::TSelect42::CheckResult(ydb->ExecuteQuery(TSampleQueries::TSelect42::Query));
+        ydb->WaitPoolHandlersCount(0, std::nullopt, TDuration::Seconds(95));


а почему такой большой таймаут? это из кеширования? нельзя ли в тестах понизить?

Это из-за чеков на неактивность раз в минуту такой таймаут вышел:
https://github.com/ydb-platform/ydb/blob/main/ydb/core/kqp/workload_service/kqp_workload_service_impl.h#L13

В тестах его не выходит поменять

github-actions · 2024-08-13T12:50:40Z

⚪ 2024-08-13 12:50:39 UTC Pre-commit check for ee41fcb has started.
⚪ 2024-08-13 12:54:24 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-13 13:01:59 UTC Build successful.

github-actions · 2024-08-13T12:56:33Z

⚪ 2024-08-13 12:56:32 UTC Pre-commit check for ee41fcb has started.
⚪ 2024-08-13 13:00:28 UTC Check linux-x86_64-release-asan is running...
⚫ 2024-08-13 13:30:51 UTC Check cancelled

github-actions · 2024-08-13T13:03:36Z

⚪ 2024-08-13 13:03:31 UTC Pre-commit check for ee41fcb has started.

github-actions · 2024-08-13T13:32:18Z

⚪ 2024-08-13 13:32:18 UTC Pre-commit check for 148abae has started.
⚪ 2024-08-13 13:36:28 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-13 13:43:01 UTC Build successful.

github-actions · 2024-08-13T13:33:03Z

⚪ 2024-08-13 13:33:02 UTC Pre-commit check for 148abae has started.
⚪ 2024-08-13 13:38:51 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-13 15:40:40 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
11001	10959	0	6	22	14

🟢 2024-08-13 15:42:16 UTC Build successful.
🟡 2024-08-13 15:42:43 UTC ydbd size 5.5 GiB changed* by +1.2 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `e0e98b1`	merge: `148abae`	diff	diff %
ydbd size	5 851 479 824 Bytes	5 852 782 928 Bytes	+1.2 MiB	+0.022%
ydbd stripped size	1 469 374 288 Bytes	1 469 565 808 Bytes	+187.0 KiB	+0.013%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-08-13T13:34:38Z

⚪ 2024-08-13 13:34:37 UTC Pre-commit check for 148abae has started.
⚪ 2024-08-13 13:37:29 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-13 15:13:14 UTC Some tests failed, follow the links below. Going to retry failed tests...

Details

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
39810	34476	0	6	5315	13

⚪ 2024-08-13 15:20:01 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟢 2024-08-13 15:27:59 UTC Tests successful.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
26 (only retried tests)	18	0	0	0	8

🟢 2024-08-13 15:28:06 UTC Build successful.
🔴 2024-08-13 15:28:42 UTC ydbd size 8.1 GiB changed* by +2.2 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash	main: `e0e98b1`	merge: `148abae`	diff	diff %
ydbd size	8 700 604 288 Bytes	8 702 890 592 Bytes	+2.2 MiB	+0.026%
ydbd stripped size	473 093 896 Bytes	473 141 896 Bytes	+46.9 KiB	+0.010%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

…rm#7688)

GrigoriyPA requested a review from gridnevvvit August 13, 2024 04:40

github-actions bot added the new-feature label Aug 13, 2024

GrigoriyPA requested a review from dorooleg August 13, 2024 04:40

GrigoriyPA added 8 commits August 13, 2024 08:50

Added resource pool classifiers cache

85bf93a

Fixed resource pools subscriptions

9c9ef40

Added path canonization

a00e1fe

Added unit tests for pool subscriptions

73dc44d

Added classifer unit test for cpp sdk

e0b8e37

Fixed TResourcePoolsCache

fce155b

Fixed build errors

9ae7ad3

Added pool id checking

1f55844

gridnevvvit previously approved these changes Aug 13, 2024

View reviewed changes

GrigoriyPA dismissed gridnevvvit’s stale review via 1f55844 August 13, 2024 12:45

GrigoriyPA force-pushed the YQ-3493-support-resource-pool-classifiers-kqp-cache branch from 64dd5a5 to 1f55844 Compare August 13, 2024 12:45

dorooleg approved these changes Aug 13, 2024

View reviewed changes

GrigoriyPA added the rebase-and-check Rebase PR with the current base branch and check label Aug 13, 2024

github-actions bot removed the rebase-and-check Rebase PR with the current base branch and check label Aug 13, 2024

GrigoriyPA merged commit a00d286 into ydb-platform:main Aug 13, 2024

GrigoriyPA deleted the YQ-3493-support-resource-pool-classifiers-kqp-cache branch August 13, 2024 16:44

naspirato pushed a commit to naspirato/ydb that referenced this pull request Aug 14, 2024

YQ-3493 support resource pool classifiers kqp_proxy cache (ydb-platfo…

c81c7cd

…rm#7688)

GrigoriyPA mentioned this pull request Aug 14, 2024

YQ WM increase future wait timeout #7780

Merged

GrigoriyPA added a commit to GrigoriyPA/ydb that referenced this pull request Aug 15, 2024

YQ-3493 support resource pool classifiers kqp_proxy cache (ydb-platfo…

020ae0d

…rm#7688)

This was referenced Aug 15, 2024

YQ-3471 support resource pool classifiers #7812

Closed

YQ kqprun pass allow local files into runtime listing #7844

Merged

YQ change code owners for clickhouse/client #7898

Merged

GrigoriyPA added a commit to GrigoriyPA/ydb that referenced this pull request Aug 16, 2024

YQ-3493 support resource pool classifiers kqp_proxy cache (ydb-platfo…

3387e42

…rm#7688)

GrigoriyPA added a commit to GrigoriyPA/ydb that referenced this pull request Sep 4, 2024

YQ-3493 support resource pool classifiers kqp_proxy cache (ydb-platfo…

dce45d2

…rm#7688)

GrigoriyPA mentioned this pull request Sep 4, 2024

YQ-3471 support resource pool classifiers #8732

Merged

GrigoriyPA added a commit to GrigoriyPA/ydb that referenced this pull request Sep 9, 2024

YQ-3493 support resource pool classifiers kqp_proxy cache (ydb-platfo…

edf46c9

…rm#7688)

GrigoriyPA added a commit to GrigoriyPA/ydb that referenced this pull request Sep 11, 2024

YQ-3493 support resource pool classifiers kqp_proxy cache (ydb-platfo…

4b2b822

…rm#7688)

uzhastik pushed a commit to uzhastik/ydb that referenced this pull request Sep 17, 2024

YQ-3493 support resource pool classifiers kqp_proxy cache (ydb-platfo…

a262b53

…rm#7688)

StekPerepolnen mentioned this pull request Oct 29, 2024

viewer healthcheck pathmerge stable 24 #11044

Closed

GrigoriyPA mentioned this pull request Feb 21, 2025

merge to ydb stable YQ-4070 supported insert values for s3 #14914

Closed

This was referenced Jun 3, 2025

Enable kafka port for local ydb container #19218

Closed

Enable kafka port for local ydb container #19220

Closed

StekPerepolnen mentioned this pull request Jul 28, 2025

audit extra merge stream nb 24 4 #21776

Closed

YQ-3493 support resource pool classifiers kqp_proxy cache #7688

YQ-3493 support resource pool classifiers kqp_proxy cache #7688

Uh oh!

Conversation

GrigoriyPA commented Aug 13, 2024

Changelog entry

Changelog category

Additional information

Uh oh!

github-actions bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gridnevvvit Aug 13, 2024

Choose a reason for hiding this comment

Uh oh!

GrigoriyPA Aug 13, 2024

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 13, 2024

Uh oh!

github-actions bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions bot commented Aug 13, 2024 •

edited

Loading

github-actions bot commented Aug 13, 2024 •

edited

Loading

github-actions bot commented Aug 13, 2024 •

edited

Loading

github-actions bot commented Aug 13, 2024 •

edited

Loading

github-actions bot commented Aug 13, 2024 •

edited

Loading

github-actions bot commented Aug 13, 2024 •

edited

Loading

github-actions bot commented Aug 13, 2024 •

edited

Loading

github-actions bot commented Aug 13, 2024 •

edited

Loading

github-actions bot commented Aug 13, 2024 •

edited

Loading

github-actions bot commented Aug 13, 2024 •

edited

Loading

github-actions bot commented Aug 13, 2024 •

edited

Loading

github-actions bot commented Aug 13, 2024 •

edited

Loading

github-actions bot commented Aug 13, 2024 •

edited

Loading

github-actions bot commented Aug 13, 2024 •

edited

Loading

github-actions bot commented Aug 13, 2024 •

edited

Loading

github-actions bot commented Aug 13, 2024 •

edited

Loading

github-actions bot commented Aug 13, 2024 •

edited

Loading