Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

eth_gasPrice ERROR Could not retrieve block #7287

Closed
siladu opened this issue Jul 2, 2024 · 3 comments · Fixed by #7482
Closed

eth_gasPrice ERROR Could not retrieve block #7287

siladu opened this issue Jul 2, 2024 · 3 comments · Fixed by #7482
Assignees
Labels
bug Something isn't working good first issue Good for newcomers P2 High (ex: Degrading performance issues, unexpected behavior of core features (DevP2P, syncing, etc))

Comments

@siladu
Copy link
Contributor

siladu commented Jul 2, 2024

While retesting #6965, which involved running a QBFT node and throwing https://github.com/MariusVanDerWijden/tx-fuzz at it. I received a new error:

2024-07-01 16:08:56.279+10:00 | vert.x-worker-thread-8 | ERROR | BaseJsonRpcProcessor | Error processing method: eth_gasPrice null
java.lang.IllegalStateException: Could not retrieve block #10
	at org.hyperledger.besu.ethereum.api.query.BlockchainQueries.lambda$gasPrice$25(BlockchainQueries.java:994)
	at java.base/java.util.Optional.orElseThrow(Optional.java:403)
	at org.hyperledger.besu.ethereum.api.query.BlockchainQueries.lambda$gasPrice$26(BlockchainQueries.java:993)
	at java.base/java.util.stream.LongPipeline$1$1.accept(LongPipeline.java:177)
	at java.base/java.util.stream.Streams$RangeLongSpliterator.forEachRemaining(Streams.java:232)
	at java.base/java.util.Spliterator$OfLong.forEachRemaining(Spliterator.java:777)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575)
	at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
	at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616)
	at org.hyperledger.besu.ethereum.api.query.BlockchainQueries.gasPrice(BlockchainQueries.java:999)
	at org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.EthGasPrice.calculateGasPrice(EthGasPrice.java:49)
	at org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.EthGasPrice.response(EthGasPrice.java:45)
	at org.hyperledger.besu.ethereum.api.jsonrpc.execution.BaseJsonRpcProcessor.process(BaseJsonRpcProcessor.java:44)
	at org.hyperledger.besu.ethereum.api.jsonrpc.execution.TracedJsonRpcProcessor.process(TracedJsonRpcProcessor.java:54)
	at org.hyperledger.besu.ethereum.api.jsonrpc.execution.TimedJsonRpcProcessor.process(TimedJsonRpcProcessor.java:45)
	at org.hyperledger.besu.ethereum.api.jsonrpc.execution.JsonRpcExecutor.execute(JsonRpcExecutor.java:92)
	at org.hyperledger.besu.ethereum.api.handlers.AbstractJsonRpcExecutor.executeRequest(AbstractJsonRpcExecutor.java:87)
	at org.hyperledger.besu.ethereum.api.handlers.JsonRpcObjectExecutor.execute(JsonRpcObjectExecutor.java:57)
	at org.hyperledger.besu.ethereum.api.handlers.JsonRpcExecutorHandler.lambda$handler$0(JsonRpcExecutorHandler.java:57)
	at java.base/java.util.Optional.ifPresentOrElse(Optional.java:196)
	at org.hyperledger.besu.ethereum.api.handlers.JsonRpcExecutorHandler.lambda$handler$2(JsonRpcExecutorHandler.java:54)
	at io.vertx.ext.web.impl.BlockingHandlerDecorator.lambda$handle$0(BlockingHandlerDecorator.java:48)
	at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:191)
	at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:279)
	at io.vertx.core.impl.ContextImpl.lambda$internalExecuteBlocking$2(ContextImpl.java:210)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:1583)

This PR had main pulled in recently. Previous test was based on May 21st main, that area of code had changed around May 2nd, in 8df6bca#diff-2ef41086175ba1eea041526cfd1d92a80688eeeb8e4e1f17566b2f3d65ed7cae

Still trying to recreate it consistently.

@siladu siladu added the bug Something isn't working label Jul 2, 2024
@jframe jframe added the P4 Low (ex: Node doesn't start up when the configuration file has unexpected "end-of-line" character) label Jul 9, 2024
@siladu
Copy link
Contributor Author

siladu commented Jul 10, 2024

I can reliably recreate this - seems to happen the first time the tx-fuzz runs on a new network. Why this is the case is unknown, would need to study the pattern of requests it makes.

Having tested older versions of Besu, I am confident 8df6bca introduced this error.

24.5.4 - works
24.6.0 - breaks
8df6bca - Make eth_gasPrice aware of the base fee market - breaks
bd12986 (the commit before "Make eth_gasPrice aware...") - works.

Attaching a terminal log showing the various errors/lack of errors for each version
7287.log


The impact isn't fully clear as only QBFT has been tested. It sometimes kills the tx-fuzz tool...

➜  tx-fuzz git:(master) ./cmd/livefuzzer/livefuzzer spam
No seed provided, creating one
Spamming 3758 transactions per account on 10 accounts with seed: 0x-5f2e05649bf1dd5
panic: invalid chain id for signer

goroutine 83 [running]:
main.SendBaikalTransactions(0x1400010e180, 0x20?, 0x40?, {0x102c24d95, 0x2a}, 0xeae, 0x0?)
	/Users/simon.dudley/IdeaProjects/tx-fuzz/cmd/livefuzzer/main.go:193 +0x438
main.SpamTransactions.func1({0x102c2e13f?, 0x0?}, {0x102c24d95, 0x2a}, 0x0?)
	/Users/simon.dudley/IdeaProjects/tx-fuzz/cmd/livefuzzer/main.go:163 +0xa4
created by main.SpamTransactions
	/Users/simon.dudley/IdeaProjects/tx-fuzz/cmd/livefuzzer/main.go:160 +0x3a0

Restarting the tool allows more transactions to be submitted though so impact seems limited.


To recreate (this might not be limited to QBFT or tx-fuzz though):

  1. Setup a fresh single-node QBFT network (I used attached genesis-with-transition.json which includes the necessary account for tx-fuzz to work)
  2. besu --config-file besu1.conf (config file attached, you need to tweak data path, key path and genesis file path)
  3. Run tx-fuzz against it
    a. run go build .
    b. ./cmd/livefuzzer/livefuzzer spam

7287-config.tar.gz

@siladu siladu added P3 Medium (ex: JSON-RPC request not working with a specific client library due to loose spec assumtion) and removed P4 Low (ex: Node doesn't start up when the configuration file has unexpected "end-of-line" character) labels Jul 10, 2024
@non-fungible-nelson non-fungible-nelson added the good first issue Good for newcomers label Jul 22, 2024
@non-fungible-nelson
Copy link
Contributor

@matthew1001 flagging this for you

@matthew1001
Copy link
Contributor

Could this be similar to the issues fixed in #6143 and #6345?

Those stemmed from issues around chainHead being updated before blockchain storage had been committed. DefaultBlockchain.appendBlockHelper(...) updates the chainHead before committing the stored block to the DB. This means that any non-synchronized method on DefaultBlockchain (of which DefaultBlockchain.getBlockHashByNumber(...) is one) chain head could be new but storage could be yet to update.

If you look at BlockchainQueries.gasPrice(...) it calls blockchain.getChainHeadHeader() which gets the new chain head via an unsynchronized call to DefaultBlockchain.getChainHeadHash(...). It then calls blockchain.getBlockByNumber(...) which ends up at the un-synchronized blockchainStorage.getBlockHash(...). There's a timing window therefore where chain head has been updated but storage hasn't been committed.

@fab-10 fab-10 self-assigned this Aug 13, 2024
@fab-10 fab-10 added P2 High (ex: Degrading performance issues, unexpected behavior of core features (DevP2P, syncing, etc)) and removed P3 Medium (ex: JSON-RPC request not working with a specific client library due to loose spec assumtion) labels Aug 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers P2 High (ex: Degrading performance issues, unexpected behavior of core features (DevP2P, syncing, etc))
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants