Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

blockstore: mark slot as dead on data shred merkle root conflict #3970

Merged
merged 1 commit into from
Dec 7, 2024
Merged

blockstore: mark slot as dead on data shred merkle root conflict #3970

merged 1 commit into from
Dec 7, 2024

Conversation

AshwinSekar
Copy link

@AshwinSekar AshwinSekar commented Dec 6, 2024 •
edited
Loading

We need to mark the slot as dead to ensure that the bank is completed

@AshwinSekar AshwinSekar changed the title blockstore: always insert conflicting shred after notifying duplicate blockstore: mark slot as dead on data shred merkle root conflict Dec 6, 2024
AshwinSekar
AshwinSekar commented Dec 6, 2024
View reviewed changes
ledger/src/blockstore.rs
0,
false,
0,
false, // merkle_variant
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this test was broken, legacy shreds are dropped on all clusters

@AshwinSekar AshwinSekar requested a review from carllin December 6, 2024 06:51
@AshwinSekar AshwinSekar merged commit 5564a94 into anza-xyz:master Dec 7, 2024
40 checks passed
@AshwinSekar AshwinSekar deleted the always-insert-shred branch December 11, 2024 16:34
@AshwinSekar AshwinSekar added v2.0 Backport to v2.0 branch v2.1 Backport to v2.1 branch labels Dec 12, 2024
@mergify Mergify
Copy link

mergify bot commented Dec 12, 2024

Backports to the stable branch are to be avoided unless absolutely necessary for fixing bugs, security issues, and perf regressions. Changes intended for backport should be structured such that a minimum effective diff can be committed separately from any refactoring, plumbing, cleanup, etc that are not strictly necessary to achieve the goal. Any of the latter should go only into master and ride the normal stabilization schedule.

@mergify Mergify
Copy link

mergify bot commented Dec 12, 2024

Backports to the beta branch are to be avoided unless absolutely necessary for fixing bugs, security issues, and perf regressions. Changes intended for backport should be structured such that a minimum effective diff can be committed separately from any refactoring, plumbing, cleanup, etc that are not strictly necessary to achieve the goal. Any of the latter should go only into master and ride the normal stabilization schedule. Exceptions include CI/metrics changes, CLI improvements and documentation updates on a case by case basis.

mergify bot pushed a commit that referenced this pull request Dec 12, 2024
@AshwinSekar @mergify
blockstore: mark slot as dead on data shred merkle root conflict (#3970)
543657a 
(cherry picked from commit 5564a94)

# Conflicts:
#	ledger/src/blockstore.rs
@mergify mergify bot mentioned this pull request Dec 12, 2024
Merged
mergify bot pushed a commit that referenced this pull request Dec 12, 2024
@AshwinSekar @mergify
blockstore: mark slot as dead on data shred merkle root conflict (#3970)
a9cb121 
(cherry picked from commit 5564a94)

# Conflicts:
#	ledger/src/blockstore.rs
@mergify mergify bot mentioned this pull request Dec 12, 2024
Merged
AshwinSekar added a commit that referenced this pull request Dec 20, 2024
@mergify @AshwinSekar
v2.0: blockstore: mark slot as dead on data shred merkle root conflic…
843c6a1 
…t (backport of #3970) (#4074)

* blockstore: mark slot as dead on data shred merkle root conflict (#3970)

(cherry picked from commit 5564a94)

# Conflicts:
#	ledger/src/blockstore.rs

* fix conflicts

---------

Co-authored-by: Ashwin Sekar <ashwin@anza.xyz>
Co-authored-by: Ashwin Sekar <ashwin@solana.com>
AshwinSekar added a commit that referenced this pull request Dec 20, 2024
@mergify @AshwinSekar
v2.1: blockstore: mark slot as dead on data shred merkle root conflic…
c79c3c9 
…t (backport of #3970) (#4075)

* blockstore: mark slot as dead on data shred merkle root conflict (#3970)

(cherry picked from commit 5564a94)

# Conflicts:
#	ledger/src/blockstore.rs

* fix conflicts

---------

Co-authored-by: Ashwin Sekar <ashwin@anza.xyz>
Co-authored-by: Ashwin Sekar <ashwin@solana.com>
nibty added a commit to x1-labs/tachyon that referenced this pull request Feb 5, 2025
@nibty @mergify
@brooksprumo @AshwinSekar @github-actions @yihau @behzadnouri @steveluscher @cavemanloverboy @apfitzge
Merge in upstream V2.0.24 (#12)
1834554 
* v2.0: Reclaims more old accounts in `clean` (backport of anza-xyz#4044) (anza-xyz#4089)

* Reclaims more old accounts in `clean` (anza-xyz#4044)

(cherry picked from commit 3d43824)

# Conflicts:
#	accounts-db/src/accounts_db.rs
#	accounts-db/src/accounts_db/tests.rs

* fix merge conflicts

---------

Co-authored-by: Brooks <brooks@anza.xyz>

* v2.0: Fixes clean_old_storages_with_reclaims tests (backport of anza-xyz#4147) (anza-xyz#4166)

* Fixes clean_old_storages_with_reclaims tests (anza-xyz#4147)

(cherry picked from commit 4eabeed)

# Conflicts:
#	accounts-db/src/accounts_db/tests.rs

* fix merge conflicts

---------

Co-authored-by: Brooks <brooks@anza.xyz>

* v2.0: blockstore: mark slot as dead on data shred merkle root conflict (backport of anza-xyz#3970) (anza-xyz#4074)

* blockstore: mark slot as dead on data shred merkle root conflict (anza-xyz#3970)

(cherry picked from commit 5564a94)

# Conflicts:
#	ledger/src/blockstore.rs

* fix conflicts

---------

Co-authored-by: Ashwin Sekar <ashwin@anza.xyz>
Co-authored-by: Ashwin Sekar <ashwin@solana.com>

* Bump version to v2.0.22 (anza-xyz#4200)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* v2.0: hardcode rust version for publish-crate (anza-xyz#4228)

* Bump version to v2.0.23 (anza-xyz#4419)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* v2.0: rolls out chained Merkle shreds to ~21% of mainnet slots (backport of anza-xyz#4431) (anza-xyz#4434)

rolls out chained Merkle shreds to ~21% of mainnet slots (anza-xyz#4431)

(cherry picked from commit 9d09787)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>

* v2.0: [rpc] Fatal `getSignaturesForAddress()` when Bigtable errors (backport of anza-xyz#3700) (anza-xyz#4442)

[rpc] Fatal `getSignaturesForAddress()` when Bigtable errors (anza-xyz#3700)

* Unindent code in `get_signatures_for_address`

* Add a custom JSON-RPC error to throw when long-term storage (ie. Bigtable) can't be reached

* When the `before`/`until` signatures can't be found, throw `SignatureNotFound` instead of `RowNotFound`

* Fatal `getSignaturesForAddress` calls when Bigtable must be queried but can't be reached

(cherry picked from commit 52f132c)

Co-authored-by: Steven Luscher <steveluscher@users.noreply.github.com>

* v2.0: ci: bump [upload|download]-artifact to v4 (anza-xyz#4501)

ci: bump [upload|download]-artifact to v4

* v2.0: ci: hardcode crate publishing version (anza-xyz#4515)

ci: hardcode rust version for publish-crate

* Bump version to v2.0.24 (anza-xyz#4528)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* v2.0: fix: reduce max packet receive time during leader window (backport of anza-xyz#2801) (anza-xyz#4544)

fix: reduce max packet receive time during leader window (anza-xyz#2801)

(cherry picked from commit 20e0df4)

Co-authored-by: cavemanloverboy <93507302+cavemanloverboy@users.noreply.github.com>

* v2.0: Scheduler Frequency Fixes (backport of anza-xyz#4545) (anza-xyz#4576)

* Change prio_graph_scheduler configurations for 1k maxs, 256 look ahead

* Break loop on scanned transaction count

* make Hold decision behave same as Consume during receive

* receive maximum of 5_000 packets - loose max

* receive_completed before process_transactions

---------

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Brooks <brooks@anza.xyz>
Co-authored-by: Ashwin Sekar <ashwin@anza.xyz>
Co-authored-by: Ashwin Sekar <ashwin@solana.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Yihau Chen <yihau.chen@icloud.com>
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
Co-authored-by: Steven Luscher <steveluscher@users.noreply.github.com>
Co-authored-by: cavemanloverboy <93507302+cavemanloverboy@users.noreply.github.com>
Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
nibty added a commit to x1-labs/tachyon that referenced this pull request Feb 6, 2025
@nibty @mergify
@brooksprumo @AshwinSekar @github-actions @yihau @behzadnouri @steveluscher @cavemanloverboy @apfitzge
Merge in upstream V2.0.24 (#12)
b8d109a 
* v2.0: Reclaims more old accounts in `clean` (backport of anza-xyz#4044) (anza-xyz#4089)

* Reclaims more old accounts in `clean` (anza-xyz#4044)

(cherry picked from commit 3d43824)

# Conflicts:
#	accounts-db/src/accounts_db.rs
#	accounts-db/src/accounts_db/tests.rs

* fix merge conflicts

---------

Co-authored-by: Brooks <brooks@anza.xyz>

* v2.0: Fixes clean_old_storages_with_reclaims tests (backport of anza-xyz#4147) (anza-xyz#4166)

* Fixes clean_old_storages_with_reclaims tests (anza-xyz#4147)

(cherry picked from commit 4eabeed)

# Conflicts:
#	accounts-db/src/accounts_db/tests.rs

* fix merge conflicts

---------

Co-authored-by: Brooks <brooks@anza.xyz>

* v2.0: blockstore: mark slot as dead on data shred merkle root conflict (backport of anza-xyz#3970) (anza-xyz#4074)

* blockstore: mark slot as dead on data shred merkle root conflict (anza-xyz#3970)

(cherry picked from commit 5564a94)

# Conflicts:
#	ledger/src/blockstore.rs

* fix conflicts

---------

Co-authored-by: Ashwin Sekar <ashwin@anza.xyz>
Co-authored-by: Ashwin Sekar <ashwin@solana.com>

* Bump version to v2.0.22 (anza-xyz#4200)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* v2.0: hardcode rust version for publish-crate (anza-xyz#4228)

* Bump version to v2.0.23 (anza-xyz#4419)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* v2.0: rolls out chained Merkle shreds to ~21% of mainnet slots (backport of anza-xyz#4431) (anza-xyz#4434)

rolls out chained Merkle shreds to ~21% of mainnet slots (anza-xyz#4431)

(cherry picked from commit 9d09787)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>

* v2.0: [rpc] Fatal `getSignaturesForAddress()` when Bigtable errors (backport of anza-xyz#3700) (anza-xyz#4442)

[rpc] Fatal `getSignaturesForAddress()` when Bigtable errors (anza-xyz#3700)

* Unindent code in `get_signatures_for_address`

* Add a custom JSON-RPC error to throw when long-term storage (ie. Bigtable) can't be reached

* When the `before`/`until` signatures can't be found, throw `SignatureNotFound` instead of `RowNotFound`

* Fatal `getSignaturesForAddress` calls when Bigtable must be queried but can't be reached

(cherry picked from commit 52f132c)

Co-authored-by: Steven Luscher <steveluscher@users.noreply.github.com>

* v2.0: ci: bump [upload|download]-artifact to v4 (anza-xyz#4501)

ci: bump [upload|download]-artifact to v4

* v2.0: ci: hardcode crate publishing version (anza-xyz#4515)

ci: hardcode rust version for publish-crate

* Bump version to v2.0.24 (anza-xyz#4528)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* v2.0: fix: reduce max packet receive time during leader window (backport of anza-xyz#2801) (anza-xyz#4544)

fix: reduce max packet receive time during leader window (anza-xyz#2801)

(cherry picked from commit 20e0df4)

Co-authored-by: cavemanloverboy <93507302+cavemanloverboy@users.noreply.github.com>

* v2.0: Scheduler Frequency Fixes (backport of anza-xyz#4545) (anza-xyz#4576)

* Change prio_graph_scheduler configurations for 1k maxs, 256 look ahead

* Break loop on scanned transaction count

* make Hold decision behave same as Consume during receive

* receive maximum of 5_000 packets - loose max

* receive_completed before process_transactions

---------

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Brooks <brooks@anza.xyz>
Co-authored-by: Ashwin Sekar <ashwin@anza.xyz>
Co-authored-by: Ashwin Sekar <ashwin@solana.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Yihau Chen <yihau.chen@icloud.com>
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
Co-authored-by: Steven Luscher <steveluscher@users.noreply.github.com>
Co-authored-by: cavemanloverboy <93507302+cavemanloverboy@users.noreply.github.com>
Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@carllin carllin carllin approved these changes

Assignees
No one assigned
Labels
v2.0 Backport to v2.0 branch v2.1 Backport to v2.1 branch
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@AshwinSekar @carllin