Big hashes in BlockHashes can fill process memory #251

obscuren · 2015-01-09T15:52:12Z

Block hashes have no size limit. Check hash size and kick of bad peers.

@zelig Do we currently have anything against random hashes? (e.g., malicious peer continuously sending unknown hashes)

fjl · 2015-01-09T15:58:13Z

You can use an array type for hashes. The rlp decoder will reject inputs that do not fit into the array and return an appropriate error message.

type message struct{
        ...
        Hash [32]byte
}

obscuren · 2015-01-09T15:59:05Z

Indeed. Second question still valid.

zelig · 2015-01-10T00:28:50Z

yes we do . ish . if a section of hashes

do not get all blocks from peers OR
do not get a continuation (further parent hashes toward the known blockchain)
then after a fix time the chain is killed.
I will make sure the peer supplying that section is expelled.

zelig · 2015-04-01T09:48:06Z

This is taken care of by conversion to common.Hash and by other measures

Make swarm-network-rewrite-syncer green - 2

Update library version

…hereum#250) (ethereum#251) * accounts,ethclient,internal,mobile: add eth api to get mru number * internal: use github.com/kevinburke/go-bindata

Squash the main verkle PR ahead of rebase don't call Bytes() in GetTreeKey (ethereum#137) trie: avoid endianness conversion in GetTreeKey (ethereum#140) * trie/utils: add concrete expected value in trie key generation test Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * mod: update to latest go-verkle Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * trie/utils: avoid endianness conversions Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * apply review changes & update to official go-verkle version Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> upgrade go-verkle to CoW version and get TestProcessVerkle to build (ethereum#138) updating ci to use self-hosted machine (ethereum#143) fix: storage offset in non-header group + reuse of value buffer (ethereum#145) dedup call to ChunkifyCode, same as replay branch (ethereum#156) * dedup call to ChunkifyCode, same as replay branch * fix some linter issues fix code offset in tree update (ethereum#157) fix REVERT in state processor test execution (ethereum#158) * fix code offset in tree update * fix REVERT in test execution save on key hashing: lump code size update with first code chunk group (ethereum#159) fix code chunk key calculation and storage key calculation (ethereum#161) * fix codeKey calculation * Remove * fix storageOffset * fix the fix to the fix to the offset fix * Remove copy/pasted, unused code in test * fix linter --------- Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com> fix: infinite loop when calling extcodecopy on empty code (ethereum#151) upgrade to latest go-verkle fix: only update code in the tree if it's dirty (ethereum#174) fix: read-touch the code size and Keccak of the origin (ethereum#175) List of changes for converting a sepolia database (ethereum#182) * naive conversion rebased on top of beverly hills * changes for the sepolia shadow fork conversion * fixes to please the linter * fixes to please the linter Unified point cache (ethereum#180) * Unified point cache * Use cache for Try*Account * alter Trie interface to use caching for slots (ethereum#181) * alter Trie interface to use caching for slots * fix: use a lock to protect the point cache (ethereum#185) * use fastest non-master go-verkle version & pull trie/Verkle.go changes to use new api (ethereum#184) * mod: update to fastest go-verkle version today Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * trie/verkle: use new batch serialization api Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> * fix: TryDelete signature in unit tests --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> trie/utils: fix potential overflow (ethereum#191) * trie/utils: fix potential overflow Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * trie/utils: receive storage key as a byte slice Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * revert formatter changes Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * trie/utils: fix mod 256 Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> trie/utils: fix incorrect bigint assignment (ethereum#193) Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> upgrade precomp link to fix CI fix: add missing code size&keccak leaves in empty accounts (ethereum#192) fixes to use the latest go-verkle@master (ethereum#197) * fixes to use the latest go-verkle@master * linter fixes * linter fixes for tests * fix: use jsign's go-verkle fix refactor: remove unused (*StateDB).GetXLittleEndian methods (ethereum#204) fix gas accounting issue in state_processor_test.go (ethereum#207) update go-verkle not to use StatelessNode anymore (ethereum#206) * update go-verkle not to use StatelessNode anymore * update go-verkle to latest refactor: move verkle gas accounting to its own block in TransitionDB (ethereum#208) fix a panic in deserializeVerkleProof if GetProofItems returns a nil ProofElements use the cachingDB instead of a custom VerkleDB (ethereum#209) * use the cachingDB instead of a custom VerkleDB * fix stack trace in LES remove holiman from CODEOWNERS as he gets too many emails read from tree in state object if the snapshot is nil (ethereum#205) add missing error checks for the root node type (ethereum#214) implement OpenStorageTrie for verkle trees (ethereum#210) * implement OpenStorageTrie for verkle trees * add a few comments for future maintenance * fix linter issue fix: copy balance leaf to new buffer in TryGetAccount (ethereum#217) implement some heretofore unimplemented iterator methods (ethereum#219) params: move verkle params to their own file (ethereum#228) fix: proper number of chunk evals (ethereum#215) overlay transition (ethereum#244) * overlay transition Fix some bugs identified in the code review Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> Include base -> overlay key-values migration logic (ethereum#199) * mod: add go-verkle version with key-value migration new apis Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * core/stateprocessor: use constant for max number of migrated key-values Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * core: add base->overlay key-values migration logic Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * core: fix some compiler errors Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * trie: consider removing transition trie api in the future Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * mod: use latest go-verkle Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> fix some unit tests errors get convresion block from file fix compilation issues fix initialization issue in migrator fix: changes needed to run the first 28 blocks important sutff: fix the banner fix: use nonce instead of balance in nonce leaf (ethereum#202) fixes for performing the overlay transition (ethereum#203) * fixes for performing the overlay transition * fixes for the full replay * fix: deletion-and-recreation of EoA * fixes to replay 2M+ blocks * upgrade to go-verkle@master * fix: proper number of chunk evals * rewrite conversion loop to fix known issues changes to make replay work with the overlay method (ethereum#216) * fixes for performing the overlay transition fixes for the full replay fix: deletion-and-recreation of EoA fixes to replay 2M+ blocks upgrade to go-verkle@master fix: proper number of chunk evals rewrite conversion loop to fix known issues changes to make replay work with the overlay method fixes to replay 2M+ blocks update to latest go-verkle@master * use a PBSS-like scheme for internal nodes (ethereum#221) * use a PBSS-like scheme for internal nodes * a couple of fixes coming from debugging replay * fix: use an error to notify the transition tree that a deleted account was found in the overlay tree (ethereum#222) * fixes for pbss replay (ethereum#227) * fixes for pbss replay * trie/verkle: use capped batch size (ethereum#229) * trie/verkle: use capped batch size Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * trie/verkle: avoid path variable allocation per db.Put Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * don't keep more than 32 state root conversions in RAM (ethereum#230) --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com> * cleanup some code * mod: update go-verkle Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * re-enable snapshot (ethereum#231) * re-enable cancun block / snapshot (ethereum#226) * clear storage conversion key upon translating account (ethereum#234) * clear storage conversion key upon translating account * mod: use latest go-verkle Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> * fix: self-deadlock with translated root map mutex (ethereum#236) * return compressed commitment as root commitment (ethereum#237) --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> fix first panic in *TransitionTrie.Copy() upgrade go-verkle to latest master mod: update go-verkle (ethereum#239) Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> core: print state root every 100 blocks (ethereum#240) Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> fix: only Commit the account trie (ethereum#242) fixes to get TestProcessVerkle to work with the overlay branch (ethereum#238) * fixes to get TestProcessVerkle to work with the overlay branch * fix all panics in verkle state processor test * fix proof verification move transition management to cachingDB * fix: mark the verkle transition as started if it's ended without being started * fix the verkle state processing test * fix linter errors * Add a function to clear verkle params for replay * fix: handle TransitionTrie in OpenStorageTrie * fix linter issue * fix the deleted account error (ethereum#247) * code cleanup (ethereum#248) * fix: don't error on a missing conversion.txt (ethereum#249) * Overlay Tree preimages exporting and usage (ethereum#246) * export overlay preimages tool Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * use preimages flat file in overlay tree migration logic Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * cmd/geth: add --roothash to overlay tree preimage exporting command Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * cleanup Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * review feedback Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * fix: reduce the PR footprint (ethereum#250) * fix: don't fail when preimages.bin is missing (ethereum#251) * fix: don't fail when preimages.bin is missing * fix: don't open the preimages file when outside of transition --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> review changes remove replay-specific code

* add l1 config in genesis config (ethereum#249) * add l1 config in genesis config * fix lint * Update params/config.go Co-authored-by: Péter Garamvölgyi <th307q@gmail.com> --------- Co-authored-by: Péter Garamvölgyi <th307q@gmail.com> * extend node configuration (ethereum#251) * extend node configuration * use block number instead of hash * accept safe, finalized and numbers for L1Confirmations * fix typos --------- Co-authored-by: Péter Garamvölgyi <th307q@gmail.com> Co-authored-by: Péter Garamvölgyi <peter@scroll.io> * Fix/improve node config parsing (ethereum#260) * raise error on failed parsing * default value * add l1-message-type, transaction methods (ethereum#252) * add l1-message-type, transaction methods * goimports * Update core/types/transaction.go Co-authored-by: Péter Garamvölgyi <th307q@gmail.com> * txpool l1 check, pointer change, marhsal test * draft: start implementing l1message gas behavior * draft: start implementing l1message gas behavior * change to gas usage * error comment typo Co-authored-by: Haichen Shen <shenhaichen@gmail.com> * goimports * update nonce, add hash test (fails), marshal test * goimports * target addr cant be nil * change call msg * comment out test * lint --------- Co-authored-by: Péter Garamvölgyi <th307q@gmail.com> Co-authored-by: Haichen Shen <shenhaichen@gmail.com> * Add L1 message database (ethereum#255) * add l1-message-type, transaction methods * goimports * Update core/types/transaction.go Co-authored-by: Péter Garamvölgyi <th307q@gmail.com> * add L1 message store to rawdb * remove comments * rename to l1_message * rename variables and add comments * write l1 msgs in a batch * add more comments * update tests * allow batched and non-batched writes * rename to accessors_l1_message * handle error * add range check * fix tests * update comments * nit * support blocks with 0 l1 messages --------- Co-authored-by: Max Wolff <maxcwolff@gmail.com> Co-authored-by: Max Wolff <max@scroll.io> * Fix L1Message Deep Copy, Complete Bridge Tx Hash test (ethereum#269) * deep copy value field, add tx hash test comment * typo * Rename nonce to queueindex, increment sender nonce on L1 message execution (ethereum#271) * change nonce to queueindex, increment nonce on L1 message * fix db acccessors * Update core/types/transaction_marshalling.go Co-authored-by: Péter Garamvölgyi <th307q@gmail.com> --------- Co-authored-by: Péter Garamvölgyi <th307q@gmail.com> * Fix db inspect command (ethereum#276) fix db inspect command * Add l1 sync service (ethereum#256) * extend node configuration * add l1-message-type, transaction methods * goimports * Update core/types/transaction.go Co-authored-by: Péter Garamvölgyi <th307q@gmail.com> * use block number instead of hash * accept safe, finalized and numbers for L1Confirmations * add L1 message store to rawdb * remove comments * fix typos * add L1 message sync service * use l1 contract address and chain ID * use L1DeploymentBlock * add confirmation config * move bridge client to separate file * use uint64 block number * fix bigint comparison * rename constants * add more logs * rename to l1_message * rename variables and add comments * write l1 msgs in a batch * add more comments * update tests * allow batched and non-batched writes * rename to accessors_l1_message * handle error * check if config is provided * improve sync service DB batched writes * add range check * fix tests * update comments * nit * fix flush range and improve comments * solve circular dependency * update stress tests * initialize l1 client for geth * start sync service * add more comments * check nil correctly * address comments * fix merge * fix genesis l1config deserialization * add sync progress logs * initial sync * handle leveldb not found error * use errors.Is * address comments * update DefaultPollInterval --------- Co-authored-by: Nazarii Denha <dengaaa2002@gmail.com> Co-authored-by: Max Wolff <maxcwolff@gmail.com> Co-authored-by: Max Wolff <max@scroll.io> * Add L1 message validation (ethereum#272) * add L1 message validation * add comments and better error handling * handle leveldb not found error * update incorrect condition for genesis block * typo * change inclusion index logic * disable L1 message check for legacy tests * set NumL1MessagesPerBlock to 0 in tests * update default genesis config * Add L1 msg validation tests (ethereum#303) add L1 msg validation tests * Update miner include l1 messages (ethereum#265) * add l1-message-type, transaction methods * goimports * Update core/types/transaction.go Co-authored-by: Péter Garamvölgyi <th307q@gmail.com> * add L1 message store to rawdb * add L1 message sync service * remove comments * use l1 contract address and chain ID * extend node configuration * use block number instead of hash * accept safe, finalized and numbers for L1Confirmations * fix typos * use L1DeploymentBlock * add confirmation config * move bridge client to separate file * use uint64 block number * fix bigint comparison * rename constants * add more logs * Fix/improve node config parsing (ethereum#260) * raise error on failed parsing * default value * rename to l1_message * rename variables and add comments * write l1 msgs in a batch * add more comments * update tests * allow batched and non-batched writes * rename to accessors_l1_message * handle error * check if config is provided * improve sync service DB batched writes * include l1 messages in blocks: part 1 * add l1-message-type, transaction methods (ethereum#252) * add l1-message-type, transaction methods * goimports * Update core/types/transaction.go Co-authored-by: Péter Garamvölgyi <th307q@gmail.com> * txpool l1 check, pointer change, marhsal test * draft: start implementing l1message gas behavior * draft: start implementing l1message gas behavior * change to gas usage * error comment typo Co-authored-by: Haichen Shen <shenhaichen@gmail.com> * goimports * update nonce, add hash test (fails), marshal test * goimports * target addr cant be nil * change call msg * comment out test * lint --------- Co-authored-by: Péter Garamvölgyi <th307q@gmail.com> Co-authored-by: Haichen Shen <shenhaichen@gmail.com> * Add L1 message database (ethereum#255) * add l1-message-type, transaction methods * goimports * Update core/types/transaction.go Co-authored-by: Péter Garamvölgyi <th307q@gmail.com> * add L1 message store to rawdb * remove comments * rename to l1_message * rename variables and add comments * write l1 msgs in a batch * add more comments * update tests * allow batched and non-batched writes * rename to accessors_l1_message * handle error * add range check * fix tests * update comments * nit * support blocks with 0 l1 messages --------- Co-authored-by: Max Wolff <maxcwolff@gmail.com> Co-authored-by: Max Wolff <max@scroll.io> * build(docker): auto docker push when pushing git tags (ethereum#258) * build(docker): update docker trigger tag prefix (ethereum#259) * Fix L1Message Deep Copy, Complete Bridge Tx Hash test (ethereum#269) * deep copy value field, add tx hash test comment * typo * commitl1messages * lint * Revert "add L1 message sync service" This reverts commit 5305e8a. * Revert "move bridge client to separate file" This reverts commit 0b220be. * update branch * use commitMessages for l1Txs * little fix * fix config * fix test * comment fixes * fix * fix config check --------- Co-authored-by: Max Wolff <maxcwolff@gmail.com> Co-authored-by: Max Wolff <max@scroll.io> Co-authored-by: Péter Garamvölgyi <th307q@gmail.com> Co-authored-by: Péter Garamvölgyi <peter@scroll.io> Co-authored-by: Haichen Shen <shenhaichen@gmail.com> Co-authored-by: HAOYUatHZ <37070449+HAOYUatHZ@users.noreply.github.com> * Add ErrUnknownAncestor tests (ethereum#305) add ErrUnknownAncestor tests * worker test include l1 msgs (ethereum#306) * worker test include l1 msgs * move L1 message index update next to block insertion --------- Co-authored-by: Péter Garamvölgyi <peter@scroll.io> * exclude l1 messages from transaction count limit in block (ethereum#307) * exclude l1 messages from transaction count limit in block * fix comments * trigger ci * nit --------- Co-authored-by: Péter Garamvölgyi <th307q@gmail.com> Co-authored-by: Péter Garamvölgyi <peter@scroll.io> * Expose queueIndex on Transaction (ethereum#316) expose queueIndex on Transaction * test that l1msg doesn't count in maxTxPerBlock limit (ethereum#312) * test that l1msg doesn't count in maxTxPerBlock limit * fix, comment * retrigger ci * change order inside test --------- Co-authored-by: HAOYUatHZ <37070449+HAOYUatHZ@users.noreply.github.com> Co-authored-by: Péter Garamvölgyi <th307q@gmail.com> * reuse trace nonce field for queueIndex * expose scroll APIs on the geth console * add L1 message query APIs * Trigger new block on new l1 messages (ethereum#343) * trigger new block on new l1 messages * typo * initialize l1MsgCh * fix worker l1msg tests (ethereum#345) --------- Co-authored-by: Nazarii Denha <dengaaa2002@gmail.com> * test(worker): ensure that l1 messages are included in the correct order (ethereum#346) test that l1msgs added in correct order * rename enqueueIndex --> queueIndex * move QueueIndex into transaction * improve l1 db interface * formatting * bump version * print l1config * add API to query latest included message queue index * clean up tx limit logic * add clarifying comments and todos to ValidateL1Messages * improve db comments and logs * clean up L1MessageTx type handling * format * format * improve L1 message block check * fix missing L1 event handling * fix TestL1MessageValidationFailure * simplify sync height resume logic * make l1Config.l1MessageQueueAddress non-pointer * improve command line flags * remove todo * use abigen tools for log filtering * cache block L1 message count * nit: fix variable name case * improve logs * flush pending writes to DB before shutdown --------- Co-authored-by: Nazarii Denha <dengaaa2002@gmail.com> Co-authored-by: Max Wolff <max@scroll.io> Co-authored-by: Haichen Shen <shenhaichen@gmail.com> Co-authored-by: Max Wolff <maxcwolff@gmail.com> Co-authored-by: HAOYUatHZ <37070449+HAOYUatHZ@users.noreply.github.com> Co-authored-by: HAOYUatHZ <haoyu@protonmail.com>

…rn-error Immediately return an error from updateTrie if DeleteStorage fails

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

obscuren added the vuln label Jan 9, 2015

obscuren added this to the PoC8 milestone Jan 9, 2015

obscuren assigned zelig Jan 9, 2015

obscuren modified the milestones: PoC8, Frontier Mar 9, 2015

zelig closed this as completed Apr 1, 2015

acud pushed a commit to acud/go-ethereum that referenced this issue Mar 1, 2018

Merge pull request ethereum#251 from ethersphere/make-snrs-green-2

071a07d

Make swarm-network-rewrite-syncer green - 2

ngtuna added a commit to ngtuna/tomochain that referenced this issue Nov 10, 2018

Merge pull request ethereum#251 from nguyenbatam/update_library_version

05d7698

Update library version

sduchesneau pushed a commit to streamingfast/go-ethereum that referenced this issue Jan 4, 2024

Merge pull request ethereum#251 from OffchainLabs/delete-storage-retu…

11e7b08

…rn-error Immediately return an error from updateTrie if DeleteStorage fails

luanxu-mxc pushed a commit to MXCzkEVM/mxc-geth that referenced this issue Sep 2, 2024

chore(taiko): release 1.0.0 (ethereum#251)

f2d2957

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Big hashes in BlockHashes can fill process memory #251

Big hashes in BlockHashes can fill process memory #251

obscuren commented Jan 9, 2015

fjl commented Jan 9, 2015

obscuren commented Jan 9, 2015

zelig commented Jan 10, 2015

zelig commented Apr 1, 2015

Big hashes in BlockHashes can fill process memory #251

Big hashes in BlockHashes can fill process memory #251

Comments

obscuren commented Jan 9, 2015

fjl commented Jan 9, 2015

obscuren commented Jan 9, 2015

zelig commented Jan 10, 2015

zelig commented Apr 1, 2015