Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: State dump creates dumps at the correct point #8794

Merged
merged 10 commits into from
Mar 27, 2023

Conversation

nikurt
Copy link
Contributor

@nikurt nikurt commented Mar 24, 2023

The dump needs to be of the state at the beginning of the last block of the previous epoch.

Add latency metrics to functions fn obtain_state_part() and fn apply_state_part().

Improves debugging to be more verbose about which state root is used for dumping and application.

Add a command neard view-state state-parts read-state-header to read a state header from the DB.

The dump needs to be of the state at the beginning of the last block of the previous epoch.
nikurt added a commit to nikurt/nearcore that referenced this pull request Mar 24, 2023
@akhi3030
Copy link
Collaborator

Please ping me when this PR is ready to be merged.

nearcore/src/metrics.rs Outdated Show resolved Hide resolved
nearcore/src/runtime/mod.rs Show resolved Hide resolved
let sync_prev_hash = sync_prev_header.prev_hash();
let prev_sync_block = chain.get_block(&sync_prev_hash)?;
if runtime.cares_about_shard(None, prev_sync_block.header().prev_hash(), shard_id, false) {
assert_eq!(num_shards, prev_sync_block.chunks().len() as u64);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for this whole thing, did you consider just using Chain::get_state_response_part()? The logic around finding the right state roots and such is already implemented there, so you could just call that function in the main loop of state_sync_dump() if InProgress just stored the shard ID, part_id and sync_hash. Unless maybe I'm missing something

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing suggestion. Done.

nikurt and others added 2 commits March 24, 2023 23:34
@nikurt
Copy link
Contributor Author

nikurt commented Mar 24, 2023

@akhi3030 Please take a look

@near-bulldozer near-bulldozer bot merged commit 88b1370 into master Mar 27, 2023
@near-bulldozer near-bulldozer bot deleted the nikurt-dump-off-by-one-block branch March 27, 2023 08:58
near-bulldozer bot pushed a commit that referenced this pull request Mar 30, 2023
…generated (#8798)

To help us understand why creation of certain parts is as slow as 10 second, look at the corresponding `StateRecord`s for the first and the last node of the state part.

Based on #8794
nikurt added a commit that referenced this pull request Mar 31, 2023
…generated (#8798)

To help us understand why creation of certain parts is as slow as 10 second, look at the corresponding `StateRecord`s for the first and the last node of the state part.

Based on #8794
nikurt added a commit to nikurt/nearcore that referenced this pull request Apr 5, 2023
The dump needs to be of the state at the beginning of the last block of the previous epoch.

Add latency metrics to functions `fn obtain_state_part()` and `fn apply_state_part()`.

Improves debugging to be more verbose about which state root is used for dumping and application.

Add a command `neard view-state state-parts read-state-header` to read a state header from the DB.
nikurt added a commit to nikurt/nearcore that referenced this pull request Apr 5, 2023
…generated (near#8798)

To help us understand why creation of certain parts is as slow as 10 second, look at the corresponding `StateRecord`s for the first and the last node of the state part.

Based on near#8794
nikurt added a commit to nikurt/nearcore that referenced this pull request Apr 6, 2023
nikurt added a commit to nikurt/nearcore that referenced this pull request Apr 13, 2023
nikurt added a commit that referenced this pull request Apr 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants