Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support CollectDiagnostics in DATA_QUERY/GENERIC_QUERY #11371

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

shnikd
Copy link
Collaborator

@shnikd shnikd commented Nov 7, 2024

No description provided.

@shnikd shnikd linked an issue Nov 7, 2024 that may be closed by this pull request
Copy link

github-actions bot commented Nov 7, 2024

2024-11-07 12:12:04 UTC Pre-commit check linux-x86_64-release-asan for d28b458 has started.
2024-11-07 12:12:16 UTC Artifacts will be uploaded here
2024-11-07 12:15:57 UTC ya make is running...
🔴 2024-11-07 12:23:13 UTC Build failed, see the logs. Also see fail summary

Copy link

github-actions bot commented Nov 7, 2024

2024-11-07 12:12:27 UTC Pre-commit check linux-x86_64-relwithdebinfo for d28b458 has started.
2024-11-07 12:12:38 UTC Artifacts will be uploaded here
2024-11-07 12:16:23 UTC ya make is running...
🔴 2024-11-07 12:24:00 UTC Build failed, see the logs. Also see fail summary

Copy link

github-actions bot commented Nov 14, 2024

2024-11-14 00:31:17 UTC Pre-commit check linux-x86_64-release-asan for d798313 has started.
2024-11-14 00:31:28 UTC Artifacts will be uploaded here
2024-11-14 00:35:05 UTC ya make is running...
🔴 2024-11-14 00:43:18 UTC Build failed, see the logs. Also see fail summary

Copy link

github-actions bot commented Nov 14, 2024

2024-11-14 00:31:28 UTC Pre-commit check linux-x86_64-relwithdebinfo for d798313 has started.
2024-11-14 00:31:39 UTC Artifacts will be uploaded here
2024-11-14 00:35:02 UTC ya make is running...
🔴 2024-11-14 00:38:37 UTC Build failed, see the logs. Also see fail summary

Copy link

github-actions bot commented Nov 14, 2024

2024-11-14 04:57:08 UTC Pre-commit check linux-x86_64-relwithdebinfo for 086c351 has started.
2024-11-14 04:57:20 UTC Artifacts will be uploaded here
2024-11-14 05:00:50 UTC ya make is running...
2024-11-14 05:05:05 UTC Check cancelled

Copy link

github-actions bot commented Nov 14, 2024

2024-11-14 04:57:36 UTC Pre-commit check linux-x86_64-release-asan for 086c351 has started.
2024-11-14 04:57:47 UTC Artifacts will be uploaded here
2024-11-14 05:01:11 UTC ya make is running...
2024-11-14 05:05:02 UTC Check cancelled

Copy link

github-actions bot commented Nov 14, 2024

2024-11-14 05:09:08 UTC Pre-commit check linux-x86_64-relwithdebinfo for bf0bc5c has started.
2024-11-14 05:09:19 UTC Artifacts will be uploaded here
2024-11-14 05:12:42 UTC ya make is running...
🟡 2024-11-14 06:48:01 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
75585 61465 0 15 13990 115

2024-11-14 06:55:05 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-11-14 07:06:19 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
129 (only retried tests) 21 0 0 0 108

🟢 2024-11-14 07:06:26 UTC Build successful.
🟢 2024-11-14 07:06:47 UTC ydbd size 2.8 GiB changed* by +5.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: f961757 merge: bf0bc5c diff diff %
ydbd size 3 049 852 280 Bytes 3 049 857 656 Bytes +5.2 KiB +0.000%
ydbd stripped size 482 446 720 Bytes 482 441 632 Bytes -5.0 KiB -0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Nov 14, 2024

2024-11-14 05:09:13 UTC Pre-commit check linux-x86_64-release-asan for bf0bc5c has started.
2024-11-14 05:09:24 UTC Artifacts will be uploaded here
2024-11-14 05:12:48 UTC ya make is running...
🟡 2024-11-14 06:51:21 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
10911 10809 0 41 12 49

🟢 2024-11-14 06:52:19 UTC Build successful.
🟢 2024-11-14 06:52:53 UTC ydbd size 5.7 GiB changed* by -98.9 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash main: f961757 merge: bf0bc5c diff diff %
ydbd size 6 171 780 240 Bytes 6 171 678 968 Bytes -98.9 KiB -0.002%
ydbd stripped size 1 538 233 296 Bytes 1 538 191 248 Bytes -41.1 KiB -0.003%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@shnikd shnikd requested a review from Gazizonoki November 20, 2024 16:52
Copy link

github-actions bot commented Nov 22, 2024

2024-11-22 07:48:06 UTC Pre-commit check linux-x86_64-relwithdebinfo for f8fcdbb has started.
2024-11-22 07:48:18 UTC Artifacts will be uploaded here
2024-11-22 07:51:43 UTC ya make is running...
🟡 2024-11-22 09:20:06 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
42383 36708 0 2 5559 114

2024-11-22 09:23:42 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-11-22 09:35:26 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
116 (only retried tests) 9 0 0 0 107

🟢 2024-11-22 09:35:33 UTC Build successful.
🟢 2024-11-22 09:35:53 UTC ydbd size 2.5 GiB changed* by +62.9 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: b7cfb36 merge: f8fcdbb diff diff %
ydbd size 2 689 924 136 Bytes 2 689 988 584 Bytes +62.9 KiB +0.002%
ydbd stripped size 481 875 952 Bytes 481 883 280 Bytes +7.2 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Nov 22, 2024

2024-11-22 07:48:50 UTC Pre-commit check linux-x86_64-release-asan for f8fcdbb has started.
2024-11-22 07:49:01 UTC Artifacts will be uploaded here
2024-11-22 07:52:24 UTC ya make is running...
🟡 2024-11-22 09:30:22 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
10940 10870 0 12 10 48

🟢 2024-11-22 09:31:16 UTC Build successful.
🟡 2024-11-22 09:31:44 UTC ydbd size 4.9 GiB changed* by +171.5 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: b7cfb36 merge: f8fcdbb diff diff %
ydbd size 5 281 479 880 Bytes 5 281 655 544 Bytes +171.5 KiB +0.003%
ydbd stripped size 1 358 631 024 Bytes 1 358 649 776 Bytes +18.3 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@shnikd shnikd force-pushed the collect-full-diagnostics branch from 4f19586 to 2ce9291 Compare December 12, 2024 12:03
@shnikd shnikd requested review from a team as code owners December 12, 2024 12:03
Copy link

github-actions bot commented Dec 12, 2024

2024-12-12 12:07:35 UTC Pre-commit check linux-x86_64-release-asan for 1239efb has started.
2024-12-12 12:07:46 UTC Artifacts will be uploaded here
2024-12-12 12:11:14 UTC ya make is running...
🔴 2024-12-12 12:18:36 UTC Build failed, see the logs. Also see fail summary

Copy link

github-actions bot commented Dec 12, 2024

2024-12-12 12:07:54 UTC Pre-commit check linux-x86_64-relwithdebinfo for 1239efb has started.
2024-12-12 12:08:05 UTC Artifacts will be uploaded here
2024-12-12 12:11:25 UTC ya make is running...
🔴 2024-12-12 12:17:57 UTC Build failed, see the logs. Also see fail summary

Copy link

github-actions bot commented Dec 12, 2024

2024-12-12 12:56:46 UTC Pre-commit check linux-x86_64-relwithdebinfo for 1204bae has started.
2024-12-12 12:56:56 UTC Artifacts will be uploaded here
2024-12-12 13:00:17 UTC ya make is running...
🟡 2024-12-12 14:27:04 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
33355 30431 0 4 2810 110

2024-12-12 14:29:49 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-12-12 14:39:31 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
752 (only retried tests) 647 0 0 0 105

🟢 2024-12-12 14:39:41 UTC Build successful.
🟢 2024-12-12 14:40:06 UTC ydbd size 2.1 GiB changed* by +52.8 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: d78c86c merge: 1204bae diff diff %
ydbd size 2 256 263 536 Bytes 2 256 317 600 Bytes +52.8 KiB +0.002%
ydbd stripped size 483 943 952 Bytes 483 951 152 Bytes +7.0 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Dec 12, 2024

2024-12-12 12:58:59 UTC Pre-commit check linux-x86_64-release-asan for 1204bae has started.
2024-12-12 12:59:11 UTC Artifacts will be uploaded here
2024-12-12 13:02:35 UTC ya make is running...
🟡 2024-12-12 14:37:59 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15846 15773 0 19 8 46

🟢 2024-12-12 14:39:27 UTC Build successful.
🟡 2024-12-12 14:39:57 UTC ydbd size 3.6 GiB changed* by +101.6 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: d78c86c merge: 1204bae diff diff %
ydbd size 3 898 615 992 Bytes 3 898 720 000 Bytes +101.6 KiB +0.003%
ydbd stripped size 1 366 738 000 Bytes 1 366 756 816 Bytes +18.4 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Dec 16, 2024

2024-12-16 04:07:25 UTC Pre-commit check linux-x86_64-relwithdebinfo for f3ee3c3 has started.
2024-12-16 04:07:57 UTC Artifacts will be uploaded here
2024-12-16 04:11:35 UTC ya make is running...
🟡 2024-12-16 05:40:48 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
31448 28508 0 3 2824 113

2024-12-16 06:06:38 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-12-16 06:18:14 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
158 (only retried tests) 48 0 2 0 108

2024-12-16 06:18:22 UTC ya make is running... (failed tests rerun, try 3)
🔴 2024-12-16 06:29:49 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
155 (only retried tests) 46 0 2 0 107

🟢 2024-12-16 06:29:56 UTC Build successful.
🟢 2024-12-16 06:30:19 UTC ydbd size 2.1 GiB changed* by +46.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 0fddef1 merge: f3ee3c3 diff diff %
ydbd size 2 251 582 648 Bytes 2 251 630 464 Bytes +46.7 KiB +0.002%
ydbd stripped size 482 548 944 Bytes 482 555 696 Bytes +6.6 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Dec 16, 2024

2024-12-16 04:09:07 UTC Pre-commit check linux-x86_64-release-asan for f3ee3c3 has started.
2024-12-16 04:09:18 UTC Artifacts will be uploaded here
2024-12-16 04:12:38 UTC ya make is running...
🟡 2024-12-16 05:49:42 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14453 14378 0 19 11 45

🟢 2024-12-16 06:05:08 UTC Build successful.
🟢 2024-12-16 06:05:32 UTC ydbd size 3.6 GiB changed* by +87.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 0fddef1 merge: f3ee3c3 diff diff %
ydbd size 3 890 263 256 Bytes 3 890 352 728 Bytes +87.4 KiB +0.002%
ydbd stripped size 1 362 091 024 Bytes 1 362 104 656 Bytes +13.3 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

CyberROFL
CyberROFL previously approved these changes Dec 17, 2024
Copy link
Member

@CyberROFL CyberROFL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Изменения api — ок.

@@ -43,6 +44,8 @@ void TCommandSql::Config(TConfig& config) {
config.Opts->AddLongOption("syntax", "Query syntax [yql, pg]")
.RequiredArgument("[String]").DefaultValue("yql").StoreResult(&Syntax)
.Hidden();
config.Opts->AddLongOption("collect-diagnostics", "Collects diagnostics and saves it to file")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

А новая опция прошла саппорт?

@@ -235,6 +245,11 @@ int TCommandSql::PrintResponse(NQuery::TExecuteQueryIterator& result) {
queryPlanPrinter.Print(*plan);
}

if (CollectFullDiagnostics) {
TFileOutput file(TStringBuilder() << "diagnostics_" << TGUID::Create().AsGuidString() << ".txt");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

А почему диагностику грузим в рандомный файл? Не лучше дать пользователю дать возможность указать путь до файла?

Copy link

github-actions bot commented Dec 18, 2024

2024-12-18 04:59:20 UTC Pre-commit check linux-x86_64-release-asan for 12e26d3 has started.
2024-12-18 04:59:34 UTC Artifacts will be uploaded here
2024-12-18 05:02:57 UTC ya make is running...
🟡 2024-12-18 06:54:10 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14500 14376 0 62 14 48

🟢 2024-12-18 06:55:20 UTC Build successful.
🟢 2024-12-18 06:55:49 UTC ydbd size 3.6 GiB changed* by +89.6 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: dffc12a merge: 12e26d3 diff diff %
ydbd size 3 894 689 168 Bytes 3 894 780 968 Bytes +89.6 KiB +0.002%
ydbd stripped size 1 363 873 328 Bytes 1 363 887 536 Bytes +13.9 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Dec 18, 2024

2024-12-18 04:59:40 UTC Pre-commit check linux-x86_64-relwithdebinfo for 12e26d3 has started.
2024-12-18 04:59:51 UTC Artifacts will be uploaded here
2024-12-18 05:03:09 UTC ya make is running...
🟡 2024-12-18 06:37:13 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
31507 28552 0 7 2832 116

2024-12-18 06:39:54 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-12-18 06:51:58 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
172 (only retried tests) 61 0 0 1 110

🟢 2024-12-18 06:52:05 UTC Build successful.
🟢 2024-12-18 06:52:23 UTC ydbd size 2.1 GiB changed* by +47.9 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: dffc12a merge: 12e26d3 diff diff %
ydbd size 2 253 799 152 Bytes 2 253 848 152 Bytes +47.9 KiB +0.002%
ydbd stripped size 483 054 512 Bytes 483 061 328 Bytes +6.7 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Dec 23, 2024

2024-12-23 11:08:35 UTC Pre-commit check linux-x86_64-release-asan for e028994 has started.
2024-12-23 11:08:46 UTC Artifacts will be uploaded here
2024-12-23 11:12:05 UTC ya make is running...
🔴 2024-12-23 11:17:35 UTC Build failed, see the logs. Also see fail summary

Copy link

github-actions bot commented Dec 23, 2024

2024-12-23 11:08:37 UTC Pre-commit check linux-x86_64-relwithdebinfo for e028994 has started.
2024-12-23 11:08:49 UTC Artifacts will be uploaded here
2024-12-23 11:12:15 UTC ya make is running...
🔴 2024-12-23 11:17:15 UTC Build failed, see the logs. Also see fail summary

@@ -942,7 +942,6 @@ message ExecuteDataQueryRequest {
QueryCachePolicy query_cache_policy = 5;
Ydb.Operations.OperationParams operation_params = 6;
QueryStatsCollection.Mode collect_stats = 7;
bool collect_full_diagnostics = 8;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

А как тут обеспечится обратная совместимость для клиентов?

Copy link

github-actions bot commented Dec 27, 2024

2024-12-27 10:37:45 UTC Pre-commit check linux-x86_64-relwithdebinfo for c15077f has started.
2024-12-27 10:37:54 UTC Artifacts will be uploaded here
2024-12-27 10:41:11 UTC ya make is running...
🔴 2024-12-27 11:08:54 UTC Build failed, see the logs. Also see fail summary

Copy link

github-actions bot commented Dec 27, 2024

2024-12-27 10:39:18 UTC Pre-commit check linux-x86_64-release-asan for c15077f has started.
2024-12-27 10:39:29 UTC Artifacts will be uploaded here
2024-12-27 10:42:51 UTC ya make is running...
🔴 2024-12-27 11:12:04 UTC Build failed, see the logs. Also see fail summary

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support CollectDiagnostics in DATA_QUERY/GENERIC_QUERY
3 participants