Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feature(collator): queue diffs tail #480

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

feature(collator): queue diffs tail #480

wants to merge 8 commits into from

Conversation

drmick
Copy link
Member

@drmick drmick commented Nov 29, 2024

No description provided.

@drmick drmick requested a review from SmaGMan November 29, 2024 10:35
@drmick drmick force-pushed the feature/queue-diff-len-calc-ref branch 2 times, most recently from 7a3ec2e to c1b9a26 Compare November 29, 2024 10:41
@codecov Codecov
Copy link

codecov bot commented Nov 29, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 48.74275% with 265 lines in your changes missing coverage. Please review.

Project coverage is 43.80%. Comparing base (604d035) to head (15e2546).

Files with missing lines Patch % Lines
collator/src/collator/do_collate/finalize.rs 0.00% 52 Missing ⚠️
collator/src/collator/do_collate/mod.rs 57.28% 44 Missing ⚠️
core/src/block_strider/subscriber/mod.rs 0.00% 41 Missing ⚠️
storage/src/store/internal_queue/mod.rs 66.66% 30 Missing and 2 partials ⚠️
collator/src/manager/blocks_cache.rs 0.00% 21 Missing ⚠️
collator/src/manager/mod.rs 0.00% 16 Missing ⚠️
collator/src/internal_queue/queue.rs 85.55% 6 Missing and 7 partials ⚠️
collator/src/queue_adapter.rs 0.00% 12 Missing ⚠️
block-util/src/queue/queue_diff.rs 41.66% 7 Missing ⚠️
collator/src/types.rs 0.00% 5 Missing ⚠️
... and 11 more
Additional details and impacted files 
@@           Coverage Diff            @@
##           master     #480    +/-   ##
========================================
  Coverage   43.80%   43.80%            
========================================
  Files         269      269            
  Lines       47299    47531   +232     
  Branches    47299    47531   +232     
========================================
+ Hits        20720    20823   +103     
- Misses      25518    25645   +127     
- Partials     1061     1063     +2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@drmick drmick force-pushed the feature/queue-diff-len-calc-ref branch from c1b9a26 to f520b76 Compare November 29, 2024 10:55
Base automatically changed from feature/large-queue-in-master to master November 29, 2024 14:45
@drmick drmick force-pushed the feature/queue-diff-len-calc-ref branch 2 times, most recently from 627290e to a888612 Compare November 29, 2024 17:51
@drmick drmick requested a review from SmaGMan November 29, 2024 17:51
@drmick drmick changed the title Feature/queue diff len calc ref feature(collator): queue tail len Nov 29, 2024
@drmick drmick requested a review from Rexagon November 29, 2024 18:04
@drmick drmick force-pushed the feature/queue-diff-len-calc-ref branch from 6be90f0 to 61804d6 Compare November 30, 2024 21:20
@drmick drmick changed the title feature(collator): queue tail len feature(collator): queue diffs tail Nov 30, 2024
@drmick drmick force-pushed the feature/queue-diff-len-calc-ref branch from 5265656 to bde3adb Compare November 30, 2024 22:17
SmaGMan
SmaGMan reviewed Dec 2, 2024
View reviewed changes
collator/src/internal_queue/queue.rs
fn clear_session_state(&self) -> Result<()> {
self.diffs.clear();
self.session_state.truncate()
fn clear_uncommitted_state(&self) -> Result<()> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We call this method when block mismatched. Then we run sync to applied mc block and apply required queue diff for this mc block. Possibly we can get incorrect diff_tail_len because we keep committed_diffs unchanged. Should check this for all places were clear_uncommitted_state called.

@drmick drmick force-pushed the feature/queue-diff-len-calc-ref branch from caf2b07 to 1a57ed1 Compare December 5, 2024 09:22
@drmick drmick requested a review from SmaGMan December 5, 2024 14:07
@drmick drmick force-pushed the feature/queue-diff-len-calc-ref branch from 1a57ed1 to 738828e Compare December 5, 2024 19:12
@drmick drmick force-pushed the feature/queue-diff-len-calc-ref branch 2 times, most recently from 4a93855 to 40f9f43 Compare December 9, 2024 23:52
SmaGMan
SmaGMan reviewed Dec 10, 2024
View reviewed changes
core/src/block_strider/subscriber/mod.rs Outdated
.await?;
let tail_len = block.block().out_msg_queue_updates.tail_len as usize;

min_tail_len = Some(min_tail_len.map_or(tail_len, |min: usize| min.min(tail_len)));
Copy link
Contributor

@SmaGMan SmaGMan Dec 10, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we get the minimal tail length from shards? Should it be the longest tail instead?

@drmick drmick force-pushed the feature/queue-diff-len-calc-ref branch 2 times, most recently from 738828e to 40f9f43 Compare December 10, 2024 11:24
@drmick drmick force-pushed the feature/queue-diff-len-calc-ref branch from 40f9f43 to 15e2546 Compare December 10, 2024 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SmaGMan SmaGMan SmaGMan approved these changes

@Rexagon Rexagon Awaiting requested review from Rexagon

At least 2 approving reviews are required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@drmick @SmaGMan