Add trace_ methods to JSON RPC

[AlexeyAkhunov]

https://openethereum.github.io/wiki/JSONRPC-trace-module#trace_call

https://openethereum.github.io/wiki/JSONRPC-trace-module#trace_callmany

[tjayrush]

A list of all OpenEthereum trace-related routines:

Transaction-Trace Filtering (focusing on these first)

• trace_block
  • implemented, not fully tested
• trace_filter
  • implemented, known issues, not fully tested
• trace_get
  • implemented, not fully tested
• trace_transaction
  • implemented, not fully tested

Ad-hoc Tracing

• trace_call
• trace_callMany
• trace_rawTransaction
• trace_replayBlockTransactions
• trace_replayTransaction

[tjayrush]

Just a note for anyone listening. I'm going to try to reproduce Parity's traces. One immediate thing I see is that TurboGeth's traces do not include the 0th trace of Parity. Parity's 0th trace is identical to the transaction itself other than the 'gasUsed' is different if there is more than one trace. When there is more than one trace, Parity's 0th trace's gasUsed is the sum of all the other traces subtracted from the initial transaction. (So, the sum of all traces including the 0th trace is the gasUsed in the receipt). [Take what I just wrote with a small grain of salt, I'm working from memory here.]

I'll make other notes here as I move forward.

[AskAlexSharov]

Somebody listening ;-)

[koen84]

This i've found in my logs. Seems most of the time it goes well, but occasionally it borks :

WARN Trying again after trace_filter RPC call failed (attempt #10) with result Err(Decoder("Error(\"unknown variant `CALL`, expected one
of `call`, `create`, `suicide`, `reward`\", line: 0, column: 0)")), subgraph_id: QmZ6noN8Ek4dPpyvmSv4AdAtefVWtUsopBc8hFKFiWo1Uw, component: SubgraphInstanceManager > BlockStream

I first thought it might be a capitalisation issue (request side), but plenty potentially similar requests get processed well.

[koen84]

As well as these in debug :

2020-09-17T18:02:37+02:00 DBG Request: {"jsonrpc":"2.0","method":"trace_filter","params":[{"fromBlock":"0xa606a3","toBlock":"0xa606a3"}],"id":3464005}
2020-09-17T18:02:37+02:00 DBG Forwarding requests: [{"id":3464005,"jsonrpc":"2.0","method":"trace_filter","params":[{"fromBlock":"0xa606a3","toBlock":"0xa606a3"}]}] reqs=1 url=http://127.0.0.1:8545/
2020-09-17T18:02:37+02:00 DBG Received response: [{"jsonrpc":"2.0","id":3464005,"error":{"code":-32000,"message":"too many traces found"}}]
 status="200 OK" url=http://127.0.0.1:8545/

and series of :

2020-09-17T18:06:56+02:00 DBG Request: {"jsonrpc":"2.0","method":"trace_filter","params":[{"fromBlock":"0xa606ba","toBlock":"0xa606ba"}],"id":3469877}
2020-09-17T18:06:56+02:00 DBG Forwarding requests: [{"id":3469877,"jsonrpc":"2.0","method":"trace_filter","params":[{"fromBlock":"0xa606ba","toBlock":"0xa606ba"}]}] reqs=1 url=http://127.0.0.1:8545/
2020-09-17T18:06:56+02:00 DBG Received response: [{"jsonrpc":"2.0","id":3469877,"error":{"code":-32000,"message":"method handler crashed"}}]
 status="200 OK" url=http://127.0.0.1:8545/

[tjayrush]

The message above seem to say that there's too many traces. There's an option when starting the rpcdaemon called --trace.maxtraces I'm not sure that will help, but you can try it.

[tjayrush]

A very large list of remaining open issues with TurboGeth tracing. There are some of the current known issues with TurboGeth tracing. As each issue is more clearly understood and closed, we will create a separate issues and strike it from this list.

General Comments:

• The goal of TurboGeth tracing is to emulate Parity's trace structure exactly (with possible backwards-compatible enhancements where needed).
• Parity returns a 'flat' array of traces presenting the call structure of the transaction in a field called traceAddress. Geth presents a single trace per transaction representing the same call structure. I prefer the flat format of Parity, as it makes treating the data as a one-dimensional array much easier. We use Parity's flat structure.
• Parity's traces provide enough data to do a full 18-decimal-place-accurate accounting on any address history (we've shown this using TrueBlocks). Geth's existing traces do not provide this same information (for example, balance transfers on self-destructs). We will try to produce any data we need to do 18-decimal place accurate accounting.
• We may add a command line flag to the RPC daemon thus:
  • --tracing:geth return already existing Geth traces
  • --tracing:parity return duplicates of parity traces
  • --tracing:enhanced return improved traces that add things parity doesn't provide but should. For example, block reward traces when address is present in eth_filter (to allow miners to recover their own rewards)

Comments about the existing tracing code (random order):

• In the callTraces.js code, DELEGATECALL does not return balance, but Parity does, For STATICCALL callTraces.js does not return a balance, but Parity does (I think).
• When I run 'go generate' (which I need to do sometimes when I change the file callTracer.js), many other files are generated. - Is there a way to generate code for a single file? - Should we try to figure out why other files are being generated (that is, are they out of data and should they be generated)?
• The need for --trace.maxtraces on the command line (for eth_filter routine) reveals the fact that we accumulate the results of an RPC call before delivering it. Might not a better way be to generate the value into a stream so we don't build a big memory usage? If it works this way already, then why the need for --trace.maxtraces?
• The file cmd/utils/flags.go is an example of getting global data from the command line flags (I think). Do we prefer using this type of global data or passing this data into the constructor of the API?
• It is not at all clear to me when to use uint256.Int vs. bit.Int vs. hexutil.Big. My preference would be to use the native Go types, but not sure.

Specific issues:

• For some calls to eth_filter, Parity returns traces showing block (and uncle) rewards(if there is no toAddress or fromAddress field in the filter). In other cases, Parity does not return traces showing block rewards (when an address option is not present). This seems odd. In some future --traces.enhanced mode, perhaps this is included.
• The after and count options of trace_filter are very poorly tested.
• Testing in general is all but non-existent for the RPC.
• There are nolint comments in the rpcdaemon code. Remove them.
• If a single transaction creates a contract and then, in that same transaction, calls self-destruct--and--if the contract has a non-zero balance at the time of destruction, we need to return that balance (from callTracer.js) in the self-destruct trace. If not there, we only have balances at the start of the previous block. Is there a call I can make somewhere in the callTracer.js to get balance of the destructed account at time of destruction? [Geth recently added this code to the tracer.]
• trace_call uses types.Transaction in its interface which requires more fields than the RPC command requires.
• Need to revisit every tracing interface (every RPC interface for that matter) for optional fields, etc. I was sloppy when adding new interfaces
• If a function exists against an API, the end user can call that function even if it's not in the interface definition. This is probably not what we want. To see this, comment out one of the interface routines and the notice the endpoint is still available.

Very large hack -- please note:

• callTracer.js has this comment. In order return the data I needed to match Parity, I currently send 0xdeadbeef as the amount of gasUsed. (YES -- this is a stunning hack!!) In the calling code, I then look for gasUsed == '0xdeadbeef which for some unknown reason makes the returned value of gasUsed as the difference between that value and 0xdeadbeef. This DOES NOT work in every case -- which is why Peter wrote this comment. The data we need is there somewhere, but I do not know how to get it in callTracer.js. This data is required in order to do full 18-decimal-place accurate accounting.

Differences between the JSON data from Parity and TurboGeth:

• If result is empty (for example, in block reward traces):
  • TG......... { ... result: {} ...}
  • Parity..... { ... result: null ...}
• if traceAddress is empty (for example, if the trace has no subtraces):
  • TG......... { ... traceAddress: null ...}
  • Parity..... { ... traceAddress: [] ...}
• if transactionHash is empty (for example, block reward traces):
  • TG......... { ... transactionHash: "0x0000...0000" ...}
  • Parity..... { ... transactionHash: null ...}
• if transactionPosition is empty (for example, block reward traces):
  • TG......... { ... transactionPosition: 0 ...}
  • Parity..... { ... transactionPosition: null ...}
• if the traces are for a self-destruct trace:
  • TG......... { ... type: selfdestruct ...}
  • Parity..... { ... type: suicide ...}
• if value or gasUsed is zero:
  • TG......... { ... xxx: 0x0 ...}
  • Parity..... field not included

Different error messages from Parity and TurboGeth:

• • TG......... invalid argument 1: json: cannot unmarshal hex string "0x" into Go value of type hexutil.Uint64
  • Parity..... Invalid params: Invalid index: cannot parse integer from empty string.
• • TG......... invalid argument 1: json: cannot unmarshal number into Go value of type []hexutil.Uint64
  • Parity..... Invalid params: invalid type: integer 0, expected a sequence.
• • TG......... missing value for required argument 1
  • Parity..... Invalid params: invalid length 1, expected a tuple of size 2.
• • TG......... invalid argument 1: json: cannot unmarshal string into Go value of type []hexutil.Uint64
  • Parity..... Invalid params: invalid type: string "0x0", expected a sequence.
• • TG......... invalid argument 0: hex number > 64 bits
  • Parity..... Invalid params: Invalid block number: number too large to fit in target type.
• • TG......... the method xxx does not exist/is not available
  • Parity..... Method not found
• • Error messages for trace_call, trace_callMany, trace_replay\* are different

Incorrect behaviour from TurboGeth:

• For eth_getTransactionReceipt the documentation says it returns:
  • root : DATA 32 bytes of post-transaction stateroot (pre Byzantium)
  • status: QUANTITY either 1 (success) or 0 (failure)
• TG returns status for every block - it does not return root

[d-xo]

Geth's existing traces do not provide this same information (for example, balance transfers on selfdestructs).

Just as a note, geth added support for self destruct in their tracer in the new release today: https://github.com/ethereum/go-ethereum/releases/tag/v1.9.22

[tjayrush]

@xwvvvvwx Thank you so much. That solves the missing balance test case and a few others. Saved me huge amount of time figuring it out.

[Edit]: I incorporated that change into TG branch 1119-tracing-3. Thanks again.