Implemented batching in wilson uploader #1955

domwst · 2024-02-14T18:19:21Z

Implemented batching in wilson uploader

...

Changelog category

Not for changelog (changelog entry is not required)

Additional information

...

github-actions · 2024-02-14T18:22:32Z

⚪ 2024-02-14 18:22:31 UTC Pre-commit check for d66b888 has started.
⚪ 2024-02-14 18:22:34 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-14 18:24:02 UTC Build successful.
⚪ 2024-02-14 18:24:15 UTC Tests are running...
🔴 2024-02-14 20:00:50 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
67530	56448	0	17	10995	70

github-actions · 2024-02-14T18:22:56Z

⚪ 2024-02-14 18:22:56 UTC Pre-commit check for d66b888 has started.
⚪ 2024-02-14 18:22:58 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-14 18:24:20 UTC Build successful.
⚪ 2024-02-14 18:24:33 UTC Tests are running...
🔴 2024-02-14 20:04:10 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14780	14583	0	21	135	41

github-actions · 2024-02-15T13:51:31Z

⚪ 2024-02-15 13:51:30 UTC Pre-commit check for 978f9ba has started.
⚪ 2024-02-15 13:51:33 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-15 13:54:10 UTC Build successful.
⚪ 2024-02-15 13:54:22 UTC Tests are running...
🔴 2024-02-15 14:07:44 UTC Test run completed, no test results found for commit 0dd8f2c. Please check build logs.
⚫ 2024-02-15 14:07:48 UTC Check cancelled

github-actions · 2024-02-15T13:51:38Z

⚪ 2024-02-15 13:51:38 UTC Pre-commit check for 978f9ba has started.
⚪ 2024-02-15 13:51:40 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-15 13:54:50 UTC Build successful.
⚪ 2024-02-15 13:55:02 UTC Tests are running...
🔴 2024-02-15 14:07:27 UTC Test run completed, no test results found for commit 0dd8f2c. Please check build logs.
⚫ 2024-02-15 14:07:31 UTC Check cancelled

ydb/library/actors/wilson/wilson_uploader.cpp

github-actions · 2024-02-15T14:09:21Z

⚪ 2024-02-15 14:09:21 UTC Pre-commit check for 992d29c has started.
⚪ 2024-02-15 14:09:23 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-15 14:11:48 UTC Build successful.
⚪ 2024-02-15 14:12:01 UTC Tests are running...
🔴 2024-02-15 15:54:02 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14790	14521	0	29	175	65

github-actions · 2024-02-15T14:09:27Z

⚪ 2024-02-15 14:09:27 UTC Pre-commit check for 992d29c has started.
⚪ 2024-02-15 14:09:28 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-15 14:12:28 UTC Build successful.
⚪ 2024-02-15 14:12:37 UTC Tests are running...
🔴 2024-02-15 15:47:00 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
67533	56515	0	4	10972	42

ydb/library/actors/wilson/wilson_uploader.cpp

github-actions · 2024-02-16T10:56:50Z

⚪ 2024-02-16 10:56:50 UTC Pre-commit check for b839934 has started.
⚪ 2024-02-16 10:56:53 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-16 11:42:07 UTC Build successful.
⚪ 2024-02-16 11:42:20 UTC Tests are running...
🔴 2024-02-16 13:25:44 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14773	14576	0	24	133	40

github-actions · 2024-02-16T10:56:51Z

⚪ 2024-02-16 10:56:51 UTC Pre-commit check for b839934 has started.
⚪ 2024-02-16 10:56:53 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-16 11:41:13 UTC Build successful.
⚪ 2024-02-16 11:41:28 UTC Tests are running...
🔴 2024-02-16 13:29:30 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
67529	56512	0	12	10972	33

alexvru · 2024-02-16T13:39:05Z

ydb/library/actors/wilson/wilson_uploader.cpp

+                    Y_ABORT_UNLESS(WakeupScheduled);
+                    WakeupScheduled = false;
+                } else {
+                    Y_ABORT_UNLESS(std::exchange(BatchCompletionScheduled, false));


It's quite bad practice to evaluate something with side effects in assert.

github-actions · 2024-02-16T13:46:22Z

⚪ 2024-02-16 13:46:22 UTC Pre-commit check for 74206ee has started.
⚪ 2024-02-16 13:46:25 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-16 13:49:35 UTC Build successful.
⚪ 2024-02-16 13:49:52 UTC Tests are running...
🔴 2024-02-16 15:25:02 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
67535	56515	0	11	10972	37

github-actions · 2024-02-16T13:49:59Z

⚪ 2024-02-16 13:49:58 UTC Pre-commit check for 74206ee has started.
⚪ 2024-02-16 13:50:00 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-16 13:52:27 UTC Build successful.
⚪ 2024-02-16 13:52:37 UTC Tests are running...
🔴 2024-02-16 15:33:41 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14772	14580	0	25	132	35

github-actions bot added the not-for-changelog label Feb 14, 2024

domwst added 2 commits February 15, 2024 04:18

Implemented batching in wilson uploader

0015ff7

Fixed several issues

0dd8f2c

domwst force-pushed the wilson-batching branch from 94bc560 to 0dd8f2c Compare February 15, 2024 13:40

domwst commented Feb 15, 2024

View reviewed changes

ydb/library/actors/wilson/wilson_uploader.cpp Outdated Show resolved Hide resolved

Final touches

61b2184

alexvru requested changes Feb 16, 2024

View reviewed changes

Resolved comments, introduced node failure

b8b3971

Some progress

fed0407

alexvru requested changes Feb 16, 2024

View reviewed changes

alexvru approved these changes Feb 16, 2024

View reviewed changes

domwst merged commit 126ee0c into ydb-platform:main Feb 16, 2024
2 of 4 checks passed

This was referenced Feb 17, 2024

YQ 2786 unmute s3 test bindings #2042

Merged

YQ-2884 fixed fq cancel operation race #2055

Merged

domwst linked an issue Feb 20, 2024 that may be closed by this pull request

Change wilson actor logic to allow for batching parameters configuration #2003

Open

GrigoriyPA mentioned this pull request Feb 20, 2024

YQ-2883 fix retries for recover point COMPLETING #2107

Merged

domwst added a commit to domwst/ydb that referenced this pull request Feb 20, 2024

Implemented batching in wilson uploader (ydb-platform#1955)

6278987

This was referenced Feb 21, 2024

YQ-2873 fix hanging in kqp compute actor #2131

Merged

YQ-2786 partly unmute tests fq s3 #2143

Merged

YQ-2898 fix query failed with unknown uncommitted upload issue #2240

Merged

domwst added a commit to domwst/ydb that referenced this pull request Feb 28, 2024

Implemented batching in wilson uploader (ydb-platform#1955)

dca15b2

domwst added a commit to domwst/ydb that referenced this pull request Mar 11, 2024

Implemented batching in wilson uploader (ydb-platform#1955)

39ed7fd

domwst added a commit to domwst/ydb that referenced this pull request Mar 12, 2024

Implemented batching in wilson uploader (ydb-platform#1955)

d4551c0

domwst added a commit to domwst/ydb that referenced this pull request Mar 19, 2024

Implemented batching in wilson uploader (ydb-platform#1955)

7b5c62a

domwst added a commit to domwst/ydb that referenced this pull request Apr 3, 2024

Implemented batching in wilson uploader (ydb-platform#1955)

6c13347

SammyVimes pushed a commit that referenced this pull request Apr 4, 2024

Implemented batching in wilson uploader (#1955)

21bd7fe

serbel324 mentioned this pull request Jun 11, 2024

Merge tracing fixes to 24-1 #5458

Merged

serbel324 pushed a commit to serbel324/ydb that referenced this pull request Jun 11, 2024

Implemented batching in wilson uploader (ydb-platform#1955)

2922104

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implemented batching in wilson uploader #1955

Implemented batching in wilson uploader #1955

domwst commented Feb 14, 2024

github-actions bot commented Feb 14, 2024 •

edited

Loading

github-actions bot commented Feb 14, 2024 •

edited

Loading

github-actions bot commented Feb 15, 2024 •

edited

Loading

github-actions bot commented Feb 15, 2024 •

edited

Loading

github-actions bot commented Feb 15, 2024 •

edited

Loading

github-actions bot commented Feb 15, 2024 •

edited

Loading

github-actions bot commented Feb 16, 2024 •

edited

Loading

github-actions bot commented Feb 16, 2024 •

edited

Loading

alexvru Feb 16, 2024

github-actions bot commented Feb 16, 2024 •

edited

Loading

github-actions bot commented Feb 16, 2024 •

edited

Loading

Implemented batching in wilson uploader #1955

Implemented batching in wilson uploader #1955

Conversation

domwst commented Feb 14, 2024