testeth via RPC requirements

[winsvega]

!!! THIS PAGE IS OUTDATED !!!
NEW LINK: https://github.com/ethereum/retesteth/wiki/RPC-Methods

The idea is to just ask a client to init a chain with the given genesis config and then calculate a transaction and tell the result state. So we need to come up with the following test rpc methods:

0. Support of commandline options

   • --ipcpath /home/user/ (geth.ipc added by the client)
   • --db-path /home/user/tempdir (datadir)
   • --test (tells the client to work in test mode)

   Discussion: Support of commandline options for ethereum clients #16
   Implemented in: CPP

1. void test_setChainParams(string _config)
   init genesis block and chain configuration
   Discussion: void test_setChainParams(string _config) #4 <----- More specs here
   Implemented in: CPP

2. string eth_sendRawTransaction(string _transactionRlpWithSig)
   accept a transaction for this test configuration (could be called multiple times)
   Discussion: string eth_sendRawTransaction(string _trRLPWithSig) #6 <----- More specs here
   Implemented in: CPP

3. void test_mineBlocks(int _number)
   mine a block (mine a block with the accepted transactions.) mining could be without PoW enabled. Function should return only after block has been imported or errorer. The status of a block should be returned.
   Discussion: void test_mineBlocks(int _number) #8 <----- More specs here
   Implemented in: CPP

4. string test_getLogHash(string _txHash)
   get pre-calculated log has of the _txHash. hash of emptyRLPList if transaction not found.
   Discussion: string test_getLogHash(string _txHash) #7 <----- More specs here
   Implemented in: CPP

5. void test_rewindToBlock(int number)
   set chain state as of blocknumber 'number'
   Discussion: void test_rewindToBlock(int number) #9 <----- More specs here
   Implemented in: CPP

6. void test_modifyTimestamp(int _timestamp)
   set the timestamp for the current block
   Discussion: void test_modifyTimestamp(int _timestamp) #10 <----- More specs here
   Implemented in: CPP

7. Export a block (rlp) by its number or hash.

8. string web3_clientVersion()
   get client version and build information to put it into the filled test
   Discussion: string web3_clientVersion() #13 <----- More specs here
   Implemented in: CPP, GETH, PYTHON..

9. Json::Array debug_accountRangeAt(string _blockHashOrNumber, int _txIndex, string _address, int _maxResults)
   get the list of accounts in the given block after given transaction index with provided account mask.
   Discussion: Json::Array debug_accountRangeAt(string _blockHashOrNumber, int _txIndex, string _address, int _maxResults) #18 <----- More specs here
   Implemented in: CPP

10. void test_importRawBlock(std::string const& _blockRLP)
    Attempt to import a block from rlp data. RLP data might be corrupted intentionally. Function should return only after block has been imported or errorer. The status of a block should be returned.
    if PoW is disabled in the chain config, a block without PoW should be accepted. Discussion: void test_importRawBlock(std::string const& _blockRLP) #19

The methods formats are to be decided. Perhaps new methods would be added. Lets come up with a tes RPC protocol here.

[carver]

If I understand this Issue correctly, we have a version of this in py-evm. The tests are currently running in Travis, against the BlockchainTests fixtures. We use two custom RPC calls, which we called:

• evm_resetToGenesisFixture(complete_json_fixture)
• evm_applyBlockFixture(json_block)

There are still many more validations we'd like to run, as we continue implementing the json-rpc methods in py-evm. Maybe we'll discover a need for new setup/testing methods. So far, I've been pleasantly surprised that those two setup calls have been sufficient.

An example of the running tests can be found in Travis: all of the jobs with the word rpc are run over the json-rpc connection.

Testing other nodes

These py-evm tests do not require any direct node access, everything is done over rpc. So the existing python tests could be run against any compliant node that implements evm_resetToGenesisFixture and evm_applyBlockFixture.

Collaboration

I'm happy to help design, implement and test the rpc testing standard, as it evolves. If you like, I can help extract the python RPC tests from py-evm to another repo (like this one, if that makes sense).

Examples

Below are examples of what these two methods take as arguments.

(From the /BlockchainTests/GeneralStateTests/stArgsZeroOneBalance/addNonConst_d0g0v0 fixture)

evm_resetToGenesisFixture

The one and only parameter is the entire text from the fixture file.

evm_applyBlockFixture

The one and only parameter is an element from the blocks value. The method is called once for each block in the list, in the order specified in the fixture. The state should be validated after each block.

In this example, the fixture defines only one block. So the method is only called once, with this parameter:

            {
                "blockHeader" : {
                    "bloom" : "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
                    "coinbase" : "0x2adc25665018aa1fe0e6bc666dac8fc2697ff9ba",
                    "difficulty" : "0x020000",
                    "extraData" : "",
                    "gasLimit" : "0x0f4240",
                    "gasUsed" : "0x68bc",
                    "hash" : "0x112ea9e0fc2a09b16e50c9a8c78ba31cf458ae5f7fcffa1e7dabc44fb17b2ecc",
                    "mixHash" : "0xda817ff0c5cfd32cc27a3741ea7bb494057ab9bbe8e1dc8a1fff2bc676b08ef6",
                    "nonce" : "0xd6ac8386f28dced8",
                    "number" : "0x01",
                    "parentHash" : "0x7e05ddccec4c8340f534f595ef1d8bea16c4e7780128ec466416c6001cd91341",
                    "receiptTrie" : "0x65432e748f3cf563b142479c8af10a76d8c470da84ca9f02e1fd563907705b99",
                    "stateRoot" : "0xe6c331f5d075f62b27e42ff0b8415b110867c792570e641badaf1616222ae794",
                    "timestamp" : "0x03e8",
                    "transactionsTrie" : "0x61276ef1c9b908482c8e8c08db0e17904a5358a160db4eb2d88c325d397443b8",
                    "uncleHash" : "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347"
                },
                "rlp" : "0xf9025ff901f7a07e05ddccec4c8340f534f595ef1d8bea16c4e7780128ec466416c6001cd91341a01dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347942adc25665018aa1fe0e6bc666dac8fc2697ff9baa0e6c331f5d075f62b27e42ff0b8415b110867c792570e641badaf1616222ae794a061276ef1c9b908482c8e8c08db0e17904a5358a160db4eb2d88c325d397443b8a065432e748f3cf563b142479c8af10a76d8c470da84ca9f02e1fd563907705b99b90100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008302000001830f42408268bc8203e880a0da817ff0c5cfd32cc27a3741ea7bb494057ab9bbe8e1dc8a1fff2bc676b08ef688d6ac8386f28dced8f862f860800183061a8094095e7baea6a6c7c4c2dfeb977efac326af552d8780801ca0c1760cf96cfbbc2756850ffd2ff8078543d42da5fd0327619c642b2c8055d5a6a066715427a6eca6698235668ae3ee160dbe4f96df97ce96f2d1193a112a15ebb6c0",
                "transactions" : [
                    {
                        "data" : "0x",
                        "gasLimit" : "0x061a80",
                        "gasPrice" : "0x01",
                        "nonce" : "0x00",
                        "r" : "0xc1760cf96cfbbc2756850ffd2ff8078543d42da5fd0327619c642b2c8055d5a6",
                        "s" : "0x66715427a6eca6698235668ae3ee160dbe4f96df97ce96f2d1193a112a15ebb6",
                        "to" : "0x095e7baea6a6c7c4c2dfeb977efac326af552d87",
                        "v" : "0x1c",
                        "value" : "0x00"
                    }
                ],
                "uncleHeaders" : [
                ]
            }

[winsvega]

I am currently investigate what methods do I need in order to run a state test via rpc.
State tests have a simplier genesis. Lets see what methods I would come up with in cpp.
The issus is that this time not only I need just to run a test via rpc, but also to generate it.

[ehildenb]

Paging @dwightguth, could we take advantage of this given our recent introduction of RPC with KEVM? If so, do you have input about how these methods should be?

[dwightguth]

@ehildenb in order to take advantage of this, we would need an ethereum client that implemented these methods and used kevm as its vm implementation. It's possible down the road but I don't think it's something we will be doing in the near future.

[carver]

Just linking to this discussion of an expanded set of APIs: ethereum/interfaces#4 (comment)

[cdetrio]

The test_getPostState will be useful more generally, especially for block explorers. Currently there is no RPC method to get a list of accounts in the state trie, and getting this basic info for a block explorer requires a lot of workaround (such as indexing and tracing every tx in the chain). But as specified, test_getPostState won't be useable on the mainnet (where the state is so large that it takes many hours to fully export).

I'd favor a simpler primitive that would also be useable on the mainnet: debug_accountRangeAt(blockNum, startKey, limit), inspired by storageRangeAt. The method would iterate over the leaves of the state trie (starting at startKey), and return a list of address hashes (up to the limit). Address hashes are returned because most clients (fast sync'd clients in particular) do not have the hash preimages, and it is not much effort to maintain the hash preimages separately (if the client has them, they can be queried with debug_preimage. They can also be imported/exported separately). The storage for each account can then be dumped with storageRangeAt.

[winsvega]

Are those debug method described and accepted for implementation already?

[Matthalp-zz]

@winsvega A few questions:

1. Is it also expected that some existing JSON-RPC methods could be needed? For example debug_trace* could be useful for finding out why a test did not pass correctly (or comparing with a correct test case)?

2. Are there plans to provide more granular testing abilities besides at the block importing/mining level? Such as what the GeneralStateTests and TransactionTest currently do?

[winsvega]

GeneralStateTests are basically block importing/mining
Transaction tests could theoretically be converted into GeneralStateTests with incorrect transactions.
VMTests are convertrable into GeneralStateTests.
So a uniform test source format is a good idea.

yes. debug methods would be needed.

[winsvega]

what do you think about this idea?

instead of test_setChainParams()

test_addAccountToState()
test_removeAccountFromState()
test_setForkRule()

[winsvega]

https://github.com/ethereum/retesteth/wiki/RPC-Methods