-
Notifications
You must be signed in to change notification settings - Fork 4.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
backport 19361 v16 #19379
Closed
Closed
backport 19361 v16 #19379
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…-labs#16724) (cherry picked from commit 4c94f89) Co-authored-by: carllin <carl@solana.com>
) (solana-labs#16770) first_coding_index is the same as the set_index and is so redundant: https://github.com/solana-labs/solana/blob/37b8587d4/ledger/src/blockstore_meta.rs#L49-L60 (cherry picked from commit 0319414) Co-authored-by: behzad nouri <behzadnouri@gmail.com>
…solana-labs#16771) Local timestamps are updated for records associated with a pubkey if the origin is still active: https://github.com/solana-labs/solana/blob/c8ed14c64/core/src/crds.rs#L301-L311 However this is done inconsistently on some gossip paths (pull requests and pull responses) but not all (e.g. push messages). Additionally update_record_timestamp is inefficient since there can be ~800 values associated with each pubkey. This commit updates records timestamps only on contact-infos; and, instead utilizes origin's timestamp when purging old values. (cherry picked from commit 2c82f21) Co-authored-by: behzad nouri <behzadnouri@gmail.com>
(cherry picked from commit c217ee3) Co-authored-by: strykerin <dacosta.pereirafabio@gmail.com>
…a-labs#16775) (cherry picked from commit e9a616c) Co-authored-by: strykerin <dacosta.pereirafabio@gmail.com>
(cherry picked from commit da58f20) Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
(cherry picked from commit 1500011)
(cherry picked from commit be29568) # Conflicts: # runtime/src/bank.rs Co-authored-by: Trent Nelson <trent@solana.com>
(cherry picked from commit c1829dd)
(cherry picked from commit 63436cc) Co-authored-by: Michael Vines <mvines@gmail.com>
…ase (solana-labs#16802) (solana-labs#16805) (cherry picked from commit ec37a84) Co-authored-by: Josh <josh.hundley@gmail.com>
…16633) (solana-labs#16828) (cherry picked from commit 9706512) Co-authored-by: behzad nouri <behzadnouri@gmail.com>
…olana-labs#16845) * Update bpf loader info on native-programs docs * Link to program deployment docs (cherry picked from commit 5eb5d9b) Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
(cherry picked from commit c2becbc)
(cherry picked from commit 3af8cb0) Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
…solana-labs#16858) (cherry picked from commit b66a689) Co-authored-by: Michael Vines <mvines@gmail.com>
…gle vote account (solana-labs#16859) (cherry picked from commit 59fc336) Co-authored-by: Michael Vines <mvines@gmail.com>
…16800) (solana-labs#16871) process_pull_responses should record hash of values purged by expired responses (as well as unexpired ones): https://github.com/solana-labs/solana/blob/c1829dd00/core/src/crds_gossip_pull.rs#L385-L387 otherwise, these values are not excluded from following pull-requests (from likely different nodes): https://github.com/solana-labs/solana/blob/c1829dd00/core/src/crds_gossip_pull.rs#L447-L452 and would waste bandwidth should they be included in subsequent pull-responses. (cherry picked from commit 3b8d6b5) Co-authored-by: behzad nouri <behzadnouri@gmail.com>
…olana-labs#16708) (solana-labs#16870) * limits to data_header.size when combining shreds' payloads (solana-labs#16708) Shredder::deshred is ignoring data_header.size when combining shreds' payloads: https://github.com/solana-labs/solana/blob/37b8587d4/ledger/src/shred.rs#L940-L961 Also adding more sanity checks on the alignment of data shreds indices. (cherry picked from commit 0f3ac51) # Conflicts: # ledger/src/shred.rs * removes backport merge conflicts Co-authored-by: behzad nouri <behzadnouri@gmail.com>
…olana-labs#16892) * Enable multiple payer keypairs * Suppress tx creation if batch size == 0 * Suppress logs when waiting to create txs * Double airdrop threshold to prevent stall when closing accounts (cherry picked from commit 283f587) Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
…olana-labs#16890) * Retain alloc'd and updated data in cpi (solana-labs#16850) (cherry picked from commit 9b3a59f) # Conflicts: # programs/bpf_loader/src/syscalls.rs # sdk/src/feature_set.rs * resolve conflicts Co-authored-by: Jack May <jack@solana.com>
…ana-labs#16894) * SDK: More conversions for `Pubkey` (cherry picked from commit 9b7120b) * SDK: More conversion for `DerivationPath` (cherry picked from commit 722de94) * remote-wallet: Add helpers for locating remote wallets (cherry picked from commit 64fcb79) * remote-wallet: Plumb `Locator` into `RemoteWalletInfo` (cherry picked from commit 3d12be2) * remote-wallet: `derivation-path` crate doesn't like empty trailing child indexes (cherry picked from commit 4ce4f04) * remote-wallet: Move `Locator` to its own module (cherry picked from commit cac666d) Co-authored-by: Trent Nelson <trent@solana.com>
…-labs#16808) (solana-labs#16896) local_message_pending_push_queue is recording timestamps at the time the value is created, and uses that when the pending values are flushed: https://github.com/solana-labs/solana/blob/ec37a843a/core/src/cluster_info.rs#L321 https://github.com/solana-labs/solana/blob/ec37a843a/core/src/crds_gossip.rs#L96-L102 which is then used as the insert_timestamp when inserting values in the crds table: https://github.com/solana-labs/solana/blob/ec37a843a/core/src/crds_gossip_push.rs#L183 The flushing may happen 100ms after the values are created (or even later if there is a lock contention). This will cause non-monotone insert_timestamps in the crds table (where time goes backward), hindering the usability of insert_timestamps for other computations. For example both ClusterInfo::get_votes and get_epoch_slots_since rely on monotone insert_timestamps when values are inserted into the table: https://github.com/solana-labs/solana/blob/ec37a843a/core/src/cluster_info.rs#L1197-L1215 https://github.com/solana-labs/solana/blob/ec37a843a/core/src/cluster_info.rs#L1274-L1298 This commit removes timestamps from local_message_pending_push_queue and uses current timestamp when flushing the queue. (cherry picked from commit b468ead) Co-authored-by: behzad nouri <behzadnouri@gmail.com>
(cherry picked from commit 36574c3) Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
…olana-labs#16898) (cherry picked from commit b17d5ee) Co-authored-by: behzad nouri <behzadnouri@gmail.com>
…#16806) (solana-labs#16905) It is crucial that VersionedCrdsValue::insert_timestamp does not go backward in time: https://github.com/solana-labs/solana/blob/ec37a843a/core/src/crds.rs#L67-L79 Otherwise methods such as get_votes and get_epoch_slots_since will break, which will break their downstream flow, including vote-listener and optimistic confirmation: https://github.com/solana-labs/solana/blob/ec37a843a/core/src/cluster_info.rs#L1197-L1215 https://github.com/solana-labs/solana/blob/ec37a843a/core/src/cluster_info.rs#L1274-L1298 For that, Crds::new_versioned is intended to be called "atomically" with Crds::insert_verioned (as the comment already says so): https://github.com/solana-labs/solana/blob/ec37a843a/core/src/crds.rs#L126-L129 However, currently this is violated in the code. For example, filter_pull_responses creates VersionedCrdsValues (with the current timestamp), then acquires an exclusive lock on gossip, then process_pull_responses writes those values to the crds table: https://github.com/solana-labs/solana/blob/ec37a843a/core/src/cluster_info.rs#L2375-L2392 Depending on the workload and lock contention, the insert_timestamps may well be in the past when these values finally are inserted into gossip. To avoid such scenarios, this commit: * removes Crds::new_versioned and Crd::insert_versioned. * makes VersionedCrdsValue constructor private, only invoked in Crds::insert, so that insert_timestamp is populated right before insert. This will improve insert_timestamp monotonicity as long as Crds::insert is not called with a stalled timestamp. Following commits may further improve this by calling timestamp() inside Crds::insert, and/or switching to std::time::Instant which guarantees monotonicity. (cherry picked from commit 1ac2a8c) Co-authored-by: behzad nouri <behzadnouri@gmail.com>
) (solana-labs#16907) ClusterInfo::new_pull_requests has to lookup contact-infos: https://github.com/solana-labs/solana/blob/a1ef2bd74/core/src/cluster_info.rs#L1663-L1673 when it was already available when making pull requests: https://github.com/solana-labs/solana/blob/a1ef2bd74/core/src/crds_gossip_pull.rs#L232 (cherry picked from commit 25054bf) Co-authored-by: behzad nouri <behzadnouri@gmail.com>
(cherry picked from commit 783bd79) Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
(cherry picked from commit 33718e5)
…#18665) * rpc: more params for `GetVoteAccountsConfig` (cherry picked from commit bf90ea2) # Conflicts: # docs/src/developing/clients/jsonrpc-api.md * cli: allow returning more `solana validators` (cherry picked from commit a4a24b6) # Conflicts: # Cargo.lock # cli/Cargo.toml # cli/src/cluster_query.rs Co-authored-by: Trent Nelson <trent@solana.com>
(cherry picked from commit 2ec81f6) Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
(cherry picked from commit e316586) # Conflicts: # core/src/broadcast_stage.rs # core/src/cluster_info.rs Co-authored-by: behzad nouri <behzadnouri@gmail.com>
* backporting bank_hash and program_costs column families from master to 1.6 for rocksdb backward compatibility * missed a line to allow dead code * include code for purge
…18780) (solana-labs#18804) * feature: add new token program feature * Fixup test * Cargo fmt * Add back whitespace for cargo fmt * Revert file totally
…ana-labs#18823) (solana-labs#18836) (cherry picked from commit d6f5945) Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
(cherry picked from commit 578f2aa) Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
…lana-labs#18969) (cherry picked from commit 611af87) Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
(cherry picked from commit 2ccbe47)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
solana-install init
can now select a pre-release from Github (solana-install init
can now select a pre-release from Github (bp #15934) #15936)tmp-
prefix so they'll automatically be cleaned up if interruptedNullSigner
s outside sign-only modeuse
(client/: Switch to a singleuse
(bp #16168) #16169)--memo
support to solana command-line #10831: added --with-memo option to all cli commands that submit (bp issue #10831: added --with-memo option to all cli commands that submit #16291) (issue #10831: added --with-memo option to all cli commands that submit (bp #16291) #16387)rent
panic (CLI: Fixrent
panic #16417) (CLI: Fixrent
panic (bp #16417) #16426)cargo-install-all.sh
resolve stable (CI: Letcargo-install-all.sh
resolve stable (bp #16428) #16430)wide_msg
everywhere for truncation on narrow terminals (dl-utils: usewide_msg
everywhere for truncation on narrow terminals (backport #16553) #16555)KeypairUrl
refactor (backport CLI BIP32 prep:KeypairUrl
refactor #16592) (CLI BIP32 prep:KeypairUrl
refactor (backport #16592) #16605)solana validators
now shows current epoch credits instead of lifetime credits (solana validators
now shows current epoch credits instead of lifetime credits (backport #16635) #16639)solana validators
: Restore the meaning of "credits" in the JSON output (solana validators
: Restore the meaning of "credits" in the JSON output (backport #16644) #16647)solana validators
(backport Add --sort argument tosolana validators
#16640) (Add --sort argument tosolana validators
(backport #16640) #16655)stake-history
output by default (CLI: Limitstake-history
output by default (backport #16670) #16673)epoch_credits
array (getVoteAccounts: Limit the length of theepoch_credits
array (backport #16662) #16692)pay
subcommand a proper alias oftransfer
(CLI: Makepay
subcommand a proper alias oftransfer
(backport #16714) #16721)getLeaderSchedule
now supports filtered results based on validator identity (backport #16723) #16731)getLeaderSchedule
options #16749) (Rpc: ReworkgetLeaderSchedule
options (backport #16749) #16752)solana validators
outputsolana leader-schedule -um
works againsolana validators
(cli: Add skip rate tosolana validators
(backport #16899) #16939)getRecentBlockhash
example (docs: fix copy-pasta breaking typo ingetRecentBlockhash
example (backport #16961) #16962)cli-output
crate (CLI: Print gossip nodes withcli-output
crate (backport #17071) #17072)solana-validator exit
now usesprocess::exit()
to ensure prompt termination (solana-validator exit
now usesprocess::exit()
to ensure prompt termination (backport #17086) #17107)prompt
, default to bip44 solana base key (SignerSource: rename input scheme toprompt
, default to bip44 solana base key #17154) (SignerSource: rename input scheme toprompt
, default to bip44 solana base key (backport #17154) #17159)Signer
types out of thesignature
module (backport MoveSigner
types out of thesignature
module #17099) (MoveSigner
types out of thesignature
module (backport #17099) #17177)solana-test-validator
(backport docs: Add docs forsolana-test-validator
#17199) (docs: Add docs forsolana-test-validator
(backport #17199) #17211)find_program_address
and example (docs: Addfind_program_address
and example #17515) (docs: Addfind_program_address
and example (backport #17515) #17517)getTransaction
RPC method (backport Plumb transaction-level rewards (aka "rent debits") into thegetTransaction
RPC method #17528) (Plumb transaction-level rewards (aka "rent debits") into thegetTransaction
RPC method (backport #17528) #17532)initialize
public (stake_program: Makestake_instruction::initialize
public #17605) (stake_program: Makestake_instruction::initialize
public (backport #17605) #17606)--slots-per-epoch
(test-validator: hold rent constant with--slots-per-epoch
(backport #18291) #18317)solana validators
output now includes average skip ratePubkey::create_with_seed
owners (SDK: Add test for illegalPubkey::create_with_seed
owners (backport #18398) #18401)--with-memo
to nonce and stake commands (backport Cli: expose--with-memo
to nonce and stake commands #18404) (Cli: expose--with-memo
to nonce and stake commands (backport #18404) #18409)is_interactive()
methodSigner
traitNonceError
withSystemError
archive_snapshot_package
IO error sources (Disambiguatearchive_snapshot_package
IO error sources (backport #18789) #18797)Problem
Summary of Changes
Fixes #