Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upstream beta-1 #596

Merged
merged 64 commits into from
Jan 23, 2025
Merged

Upstream beta-1 #596

merged 64 commits into from
Jan 23, 2025

Conversation

blxdyx
Copy link
Collaborator

@blxdyx blxdyx commented Jan 22, 2025

No description provided.

michelemodolo and others added 30 commits January 14, 2025 09:28
This activates the erigonQA dashboard backup.

Co-authored-by: Michele Modolo <michele@nospam.nospam>
…3425)

The erigonQA dashboard got changed so the new version should overtake
the currently stored one, which should in turn get archived in a
compressed form
…rigontech#13424)

The erigon_custom_metrics dashboard got changed so the new version
should overtake the currently stored one, which should in turn get
archived in a compressed form
The updated EIP-7685 says requests with empty `request_data` should be
dropped from `executionRequests` field in the API and ignored for hash
calculation.
See ethereum/EIPs#8989,
ethereum/execution-apis#599

Issue board: erigontech#12401
…ts (erigontech#13436)

`erigon init genesis.json` was causing a genesis block to be created
with the wrong hash due to using inccorrect RequestsHash for blocks
before Prague

this caused the Hive tests to fail with:
```
parent's total difficulty not found with hash 6549118a1de2c28b204c671688e0aa4f6b522e72bf3a4a3f76cc8def0a7ecc8f and height 0: <nil>
```

when trying to insert blocks, causing a considerable amount of tests in
rpc-compat to fail, hence making the CI red
…rigontech#13435)

The erigon_custom_metrics dashboard got changed so the new version
should overtake the currently stored one, which should in turn get
archived in a compressed form

Co-authored-by: Michele Modolo <nospam@nospam.nospam>
fixes node falling out of sync sometimes
- trying to solve this -
erigontech#13432
- doesn't solve it completely...next step is to AccountsHistoryIdx,
StorageHistoryIdx etc. linked to AccountsDomain, StorageDomain etc. this
will make IndexRange one-liner
This PR changes the default flags for erigon to run astrid sync as the
default for pos networks

---------

Co-authored-by: taratorio <94537774+taratorio@users.noreply.github.com>
…#13452)

QA test failed after erigontech#13376
with
```
!!->[EROR] [01-15|15:37:50.790] catch panic                              err="interface conversion: chain.BorConfig is nil, not *borcfg.BorConfig" stack="[main.go:46 panic.go:770 iface.go:262 iface.go:272 service.go:52 backend.go:983 node.go:143 main.go:98 make_app.go:71 command.go:276 app.go:333 app.go:307 main.go:51 proc.go:271 asm_amd64.s:1695]"
*** - 2025-01-15 15:38:48 - Check failed: panic
```


https://github.com/erigontech/erigon/actions/runs/12791773578/job/35660649947
- updated link for blog
- removed duplicate links for blog and Twitter in the "Getting in touch
section"
…tech#13446)

NewPayload function when processing a new block spends time on two main
sub-functions - `InsertBlockAndWait` and `ValidateChain`.

I want to see how the time we are spending there is distributed among
them and see how specific optimisation affects them.

Co-authored-by: shota.silagadze <shota.silagadze@taal.com>
JkLondon and others added 27 commits January 16, 2025 16:17
closes erigontech#13220

---------

Co-authored-by: JkLondon <ilya@mikheev.fun>
This PR extracts `HashAndNibblize` as a utility function, not dependent
on `HexPatriciaTrieHashed` .

Doing some benchmark comparisons this change doesn't impact performance
(in fact it looks to make an improvement).

These are the comparisons:

- For 25 million keys:
  - (before) `hph.HashAndNibblizeKey`: 136.664s
  - (after) pure `HashAndNibblizeKey` : 108.215s

- For 35 million keys:
  - (before) `hph.HashAndNibblizeKey`: 351.598s
  - (after) pure `HashAndNibblizeKey` :   326.218s

- For 50 million keys:
  - (before) `hph.HashAndNibblizeKey`: 1055.217s
  - (after) pure `HashAndNibblizeKey` :   568.075s

---------

Co-authored-by: antonis19 <antonis19@users.noreply.github.com>
Co-authored-by: awskii <artem.tsskiy@gmail.com>
closes erigontech#13233

Co-authored-by: JkLondon <ilya@mikheev.fun>
```
==================
WARNING: DATA RACE
Write at 0x00c029988f40 by goroutine 322:
  github.com/erigontech/erigon/txnprovider/txpool.(*bestSlice).UnsafeAdd()
      /home/ubuntu/erigon/txnprovider/txpool/queues.go:50 +0x444
  github.com/erigontech/erigon/txnprovider/txpool.(*PendingPool).Add()
      /home/ubuntu/erigon/txnprovider/txpool/pending_pool.go:97 +0x325
  github.com/erigontech/erigon/txnprovider/txpool.(*TxPool).promote()
      /home/ubuntu/erigon/txnprovider/txpool/pool.go:1819 +0x194d
  github.com/erigontech/erigon/txnprovider/txpool.(*TxPool).addTxns()
      /home/ubuntu/erigon/txnprovider/txpool/pool.go:1327 +0xd84
  github.com/erigontech/erigon/txnprovider/txpool.(*TxPool).processRemoteTxns()
      /home/ubuntu/erigon/txnprovider/txpool/pool.go:511 +0x7c7
  github.com/erigontech/erigon/txnprovider/txpool.(*TxPool).Run()
      /home/ubuntu/erigon/txnprovider/txpool/pool.go:1887 +0x9b2
  github.com/erigontech/erigon/eth.(*Ethereum).Start.func3()
      /home/ubuntu/erigon/eth/backend.go:1594 +0x7a
  golang.org/x/sync/errgroup.(*Group).Go.func1()
      /home/ubuntu/go/pkg/mod/golang.org/x/sync@v0.10.0/errgroup/errgroup.go:78 +0x91

Previous read at 0x00c029988f40 by goroutine 324:
  github.com/erigontech/erigon/txnprovider/txpool.(*TxPool).OnNewBlock()
      /home/ubuntu/erigon/txnprovider/txpool/pool.go:310 +0x3c4
  github.com/erigontech/erigon/txnprovider/txpool.(*Fetch).handleStateChangesRequest()
      /home/ubuntu/erigon/txnprovider/txpool/fetch.go:516 +0x47a
  github.com/erigontech/erigon/txnprovider/txpool.(*Fetch).handleStateChanges()
      /home/ubuntu/erigon/txnprovider/txpool/fetch.go:468 +0x2ad
  github.com/erigontech/erigon/txnprovider/txpool.(*Fetch).ConnectCore.func1()
      /home/ubuntu/erigon/txnprovider/txpool/fetch.go:124 +0xa4
```

also renamed `coreDB` to `chainDB`
relates to erigontech#13480
fully fixes erigontech#13474

Seems like `Header.hash` memoization changes from a few months ago have
made `Block.Header()` (which makes a copy by calling `CopyHeader`) show
race warnings. This is because we are copying over `atomic.Pointer` in
`CopyHeader`.

This PR fixes this by manually copy-ing all fields and skipping copy of
the `hash atomic.Pointer` attribute (instead it will get set using its
zero value - correct behaviour) and then the hash will be memoized using
it as part of the copy later on.

This is a bit clumsier as we can miss updating the `CopyHeader` function
when adding new attributes to the `Header` struct - to address that I've
added a reflection based test which will generate random values for all
struct fields and as a result capture this pitfall and remind developers
to update.

Example race detected:
```
==================
WARNING: DATA RACE
Read at 0x00c03acc0838 by goroutine 26498:
  github.com/erigontech/erigon/core/types.CopyHeader()
      /home/ubuntu/erigon/core/types/block.go:1098 +0x5c
  github.com/erigontech/erigon/core/types.(*Block).Header()
      /home/ubuntu/erigon/core/types/block.go:1278 +0x54
  github.com/erigontech/erigon/turbo/execution/eth1/eth1_utils.ConvertBlockToRPC()
      /home/ubuntu/erigon/turbo/execution/eth1/eth1_utils/grpc.go:108 +0x55
  github.com/erigontech/erigon/turbo/execution/eth1/eth1_utils.ConvertBlocksToRPC()
      /home/ubuntu/erigon/turbo/execution/eth1/eth1_utils/grpc.go:102 +0xc4
  github.com/erigontech/erigon/polygon/sync.(*executionClient).InsertBlocks()
      /home/ubuntu/erigon/polygon/sync/execution_client.go:71 +0x4f
  github.com/erigontech/erigon/polygon/sync.(*ExecutionClientStore).insertBlocks()
      /home/ubuntu/erigon/polygon/sync/store.go:162 +0x14f
  github.com/erigontech/erigon/polygon/sync.(*ExecutionClientStore).Run()
      /home/ubuntu/erigon/polygon/sync/store.go:143 +0x1ab
  github.com/erigontech/erigon/polygon/sync.(*Service).Run.func2()
      /home/ubuntu/erigon/polygon/sync/service.go:125 +0x69
  golang.org/x/sync/errgroup.(*Group).Go.func1()
==================
```
added circuit breaker if too many iterations are performed
Disabling pectra assertoor - they are neither complete nor stable for
devnet-5

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
…#13499)

Removed web download client stats calculation as it isn't used at all.
# Conflicts:
#	.github/workflows/release.yml
#	.github/workflows/test-integration-erigon.yml
#	cmd/state/exec3/state.go
#	consensus/merge/merge.go
#	consensus/misc/eip4844.go
#	core/blockchain.go
#	core/chain_makers.go
#	core/types/block_test.go
#	erigon-lib/chain/chain_config.go
#	params/version.go
#	tests/state_test_util.go
@blxdyx blxdyx merged commit 800e532 into node-real:main Jan 23, 2025
9 checks passed
@blxdyx blxdyx deleted the beta_1 branch January 23, 2025 02:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.