Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

client: allow past block numbers in RPC queries #1598

Merged
merged 4 commits into from
Dec 14, 2021

Conversation

ryanio
Copy link
Contributor

@ryanio ryanio commented Dec 8, 2021

This PR enables the RPC layer to use past block numbers in call, getBalance, getCode, getStorageAt, getTransactionCount, and estimateGas.

This is possible by setting the vm's stateManager's stateRoot to the block's stateRoot. Thanks @jochem-brouwer for the tip :) The logic is fully tested in getBalance.spec.ts by comparing a past block to the latest.

The second commit (unrelated) simplifies some of the tape result comparisons.

@codecov
Copy link

codecov bot commented Dec 8, 2021

Codecov Report

Merging #1598 (140542c) into master (22bc187) will increase coverage by 0.12%.
The diff coverage is 100.00%.

Impacted file tree graph

Flag Coverage Δ
block 84.96% <ø> (ø)
blockchain 85.08% <ø> (ø)
client 71.27% <100.00%> (+0.37%) ⬆️
common 100.00% <ø> (ø)
devp2p 83.03% <ø> (ø)
ethash ∅ <ø> (∅)
trie 86.09% <ø> (ø)
tx 88.62% <ø> (ø)
util 91.81% <ø> (ø)
vm 79.79% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

@jochem-brouwer
Copy link
Member

Have given this a quick glance, will give it a more thorough review later, but this looks good - and also much more simple than I expected 😄

@jochem-brouwer
Copy link
Member

Could we wait merging this until #1590 is merged? Then we can also include these changes for the getProof RPC call :)

@ryanio ryanio mentioned this pull request Dec 9, 2021
4 tasks
ryanio added a commit that referenced this pull request Dec 9, 2021
  * already have array validator (usage: `validators.array(validators.hex)`)
  * add typedoc for getProof (thanks @gabrocheleau)
  * update getProof.spec.ts to match future test setup from PR #1598 (this slightly modifies the returned accountProof)
jochem-brouwer added a commit that referenced this pull request Dec 9, 2021
* vm: start on getProof

* vm: stateManager: add getProof EIP-1186

* vm: stateManager: add verifyProof

* vm: add getProof tests

* vm: start on getProof

* vm: stateManager: add getProof EIP-1186

* vm: stateManager: add verifyProof

* vm: move changes of old state manager into new one

* vm: fix proof test

* vm: make getProof better readable

* vm: stateManager cleanup verifyProof

* vm: make proof/verifyproof optional

* stateManager: add ropsten stateManager tests
stateManager: fix getProof EIP1178 field

* stateManager: more getProof tests / ensure geth compatibility

* stateManager: partially fix verifyProof

* stateManager: fix empty account proofs
stateManager: fix storage slot proofs

* stateManager: add tests for tampered proofs

* stateManager: use Proof type of stateManage not MPT

* client: add getProof endpoint
client: add tests for getProof

* vm: state: update interface

* review - vm:
  * bolster invalid proof error messages
  * extract ProofStateManager tests to own file
  * move testdata files to own folder and use import syntax for improved type info

* review - client:
  * already have array validator (usage: `validators.array(validators.hex)`)
  * add typedoc for getProof (thanks @gabrocheleau)
  * update getProof.spec.ts to match future test setup from PR #1598 (this slightly modifies the returned accountProof)

Co-authored-by: Ryan Ghods <ryan@ryanio.com>
@ryanio ryanio force-pushed the client-rpc-allow-block-number-arg branch from 328f22a to 4af298d Compare December 10, 2021 04:22
@@ -10,48 +11,65 @@ import { checkError } from '../util'
const method = 'eth_call'

tape(`${method}: call with valid arguments`, async (t) => {
const blockchain = await Blockchain.create()
const blockchain = await Blockchain.create({ validateBlocks: false, validateConsensus: false })
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How come this passed before? It should throw on invalid PoW blocks right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

before we were just using vm.runTx, but now are creating blocks and adding them to the blockchain (so our source code can setStateRoot properly), so needed these additional overrides


// call with height that exceeds chain height
req = params(method, [address.toString(), '0x1'])
expectRes = checkError(t, INVALID_PARAMS, 'specified block greater than current height')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test in general looks great, but it does not explicitly check that when we pass a specific block, it actually uses that block. Probably just create 2 blocks (dont have to be valid) so we have blocks 0 (earliest), 1 and 2 (latest). Now you can check explicitly for earliest/latest (these are two different blocks), explicit 0x0, explicit 0x2, explicit 0x3 (throws) and explicit 0x1. Just check the balance of coinbase of all blocks should be fine.

Also 2 questions: (1) what happens if we pass a negative number over RPC? and (2) in general what happens if the RPC throws? Does it then still return a request with an error code, or does it fail to reply to the request?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea, will add another block, thanks.

  1. numbers are only passed as hex encoded over the RPC layer, so something like -0x10?

Trying with toBuffer which we mostly use internally to format, I get:

Error: Cannot convert string to buffer. toBuffer only supports 0x-prefixed hex strings and this string was given: -0x10

  1. the RPC needs to throw an error of shape { code, message } to properly transmit as an JSON-RPC formatted error object. otherwise if something throws deeper, sometimes it may return as Internal Error with no error object/msg displayed, which is annoying and challenging to debug :) so you can see in some delicate places in the engine API I do:
} catch (error: any) {
      throw {
        code: INTERNAL_ERROR,
        message: error.message.toString(),
      }
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah great thanks.

For (2) I think we should log the entire error message in the console? In Geth, for instance, if you do an RPC call and there is some internal error (e.g. trying to mine without etherbase/coinbase set) then it puts [WARN] <error message> in the console.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(This is not in the scope of this PR though)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I agree with you, haven't found a straightforward way to forward all errors like this through jayson yet, definitely worth more investigation to see if it is possible (i assume it should be, we have to place a catch at a higher level and throw with the right obj shape)

Copy link
Member

@jochem-brouwer jochem-brouwer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR looks very great! I love also the improvements in the tests so that these are more compact.

I have 1 request to expand the tests and some not super-related questions.

@jochem-brouwer
Copy link
Member

jochem-brouwer commented Dec 10, 2021

BTW, this PR also shows that we should really start creating a standalone package for StateManager soon. It does not make a lot of sense to use the VM interface when we have "simple" state lookups such as getCode/getBalance and alike. It only makes sense when we actually simulate txs, such as call or estimateGas. See #1599

@acolytec3 acolytec3 self-requested a review December 10, 2021 20:26
Copy link
Contributor

@acolytec3 acolytec3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, especially the test clean up. I experimented with some rpc calls to a couple of the eth methods and got good results.

@holgerd77
Copy link
Member

Will merge here.

@holgerd77 holgerd77 merged commit fe50f4e into master Dec 14, 2021
@holgerd77 holgerd77 deleted the client-rpc-allow-block-number-arg branch December 14, 2021 19:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants