Cannot get traces for block 21030627

[artstar]

System information

Erigon version 2.60.8

Actual behaviour

Execute

curl -X POST http://node.url/ -H 'Content-Type: application/json' -d '{"method":"trace_block","params":["0x140e6e3"],"id":1,"jsonrpc":"2.0"}'

Result:

{"jsonrpc":"2.0","id":1,"error":{"code":-32000,"message":"first run for txIndex 85 error: insufficient funds for gas * price + value: address 0x0b92619DdE55C0cbf828d32993a7fB004E00c84B have 1414044108792614043 want 1414044806692614043"}}

Expected behaviour

Expected response is a list of traces without error.

[phyyou]

This issue seems to be related to #12092, #12242 reported earlier.

[MarcinZakrzewski13]

Are there any predictions as to when this problem will be fixed?

[zakirgl]

any updates here? or workarounds guys?

[jamesgithub1]

Hello. I am encountering the same issue. Is there any ETA for a fix? Does it affect all release versions?

[aliaksei-a]

try to use

curl --location '<node_ip>:<node_port>' \
--header 'Content-Type: application/json' \
--data '{
	"jsonrpc":"2.0",
	"method":"debug_traceBlockByNumber",
	"params":[
		"0x140E6E3",
        { "tracer": "callTracer", "timeout": "300000ms" }
	],
	"id":"1"
}'

instead of trace_block
use erigon version 2.60.x, because for 2.59.x debug_traceBlockByNumber also returns error for tx 0x03d15392f4f774d0dbf8dff287991896cf39b171fb0b1da6c51d9045657a0cc9:
tracing failed: insufficient funds for gas * price + value:
the point is that you need to handle traces in new way, so, you have to make remapping

[zeevo]

I am also experiencing this on version 2.60.6-d24e5d45

[MarcinZakrzewski13]

try to use

curl --location '<node_ip>:<node_port>' \
--header 'Content-Type: application/json' \
--data '{
	"jsonrpc":"2.0",
	"method":"debug_traceBlockByNumber",
	"params":[
		"0x140E6E3",
        { "tracer": "callTracer", "timeout": "300000ms" }
	],
	"id":"1"
}'

instead of trace_block use erigon version 2.60.x, because for 2.59.x debug_traceBlockByNumber also returns error for tx 0x03d15392f4f774d0dbf8dff287991896cf39b171fb0b1da6c51d9045657a0cc9: tracing failed: insufficient funds for gas * price + value: the point is that you need to handle traces in new way, so, you have to make remapping

Or trace_transaction if you can skip individual transactions and are dependent on the data model.

[nate-staked]

also hitting this, would love a fix, happy to test

[racket2000]

+1

I am encountering this exact issue as well, and would greatly appreciate a fix or workaround.

[tjayrush]

the point is that you need to handle traces in new way, so, you have to make remapping

I'm curious about this. First, are you one of the devs on Erigon?

In many cases, "handling traces in a new way" is not possible.

Perhaps I'm just misunderstanding what you're saying. Sorry if so.

[aliaksei-a]

the point is that you need to handle traces in new way, so, you have to make remapping

I'm curious about this. First, are you one of the devs on Erigon?

In many cases, "handling traces in a new way" is not possible.

Perhaps I'm just misunderstanding what you're saying. Sorry if so.

I am not an Erigon dev. I've just faced an issue as others in this thread. I wanted to get traces for block 21030627 but
trace_block threw and error and I suggested to use debug_traceBlockByNumber instead because looks like all data from trace_block can be fetched from debug_traceBlockByNumber (at least in my case), but it is structured in another way.

[keithchew]

For those who have access to update their code, here is an example on how to interchange between trace_replayBlockTransactions and debug_traceBlockByNumber:

curl -X POST -H "Content-Type: application/json" --data '{"jsonrpc": "2.0", "method": "trace_replayBlockTransactions", "params": ["0x140E6F2",["trace"]], "id":1}' <node_url>

vs

curl -X POST -H "Content-Type: application/json" --data '{"jsonrpc": "2.0", "method": "debug_traceBlockByNumber", "params": ["0x140E6F2",{ "tracer": "callTracer" }], "id":1}' <node_url>

Results (just a transaction for this example) are:

{
      "output": "0x0000000000000000000000000000000000000000000000000000000000000001",
      "stateDiff": null,
      "trace": [
        {
          "action": {
            "from": "0x5cf78da3a7c0f96eb6723b0dad4ab1aa5b0da78e",
            "callType": "call",
            "gas": "0x76be",
            "input": "0xa9059cbb0000000000000000000000009642b23ed1e01df1092b92641051881a322f5d4e000000000000000000000000000000000000000000000000c7845769a8be67a4",
            "to": "0x1ceb5cb57c4d4e2b2433641b95dd330a33185a44",
            "value": "0x0"
          },
          "result": {
            "gasUsed": "0x32e7",
            "output": "0x0000000000000000000000000000000000000000000000000000000000000001"
          },
          "subtraces": 0,
          "traceAddress": [],
          "type": "call"
        }
      ],
      "vmTrace": null,
      "transactionHash": "0x6767170763fefd92ed83fc9fdb16952df91df1da10ee992c89776ad2e541b385"
    }

vs

{
      "txHash": "0x6767170763fefd92ed83fc9fdb16952df91df1da10ee992c89776ad2e541b385",
      "result": {
        "from": "0x5cf78da3a7c0f96eb6723b0dad4ab1aa5b0da78e",
        "gas": "0xcb56",
        "gasUsed": "0x74bf",
        "to": "0x1ceb5cb57c4d4e2b2433641b95dd330a33185a44",
        "input": "0xa9059cbb0000000000000000000000009642b23ed1e01df1092b92641051881a322f5d4e000000000000000000000000000000000000000000000000c7845769a8be67a4",
        "output": "0x0000000000000000000000000000000000000000000000000000000000000001",
        "value": "0x0",
        "type": "CALL"
      }
    }

As seen above and mentioned by @aliaksei-a, the values from trace_replayBlockTransactions can be found in debug_traceBlockByNumber.

This might also give the Erigon devs some pointers on how to fix the bug, since it is not breaking on debug_traceBlockByNumber.

[keithchew]

Also worth pointing out that each result in debug_traceBlockByNumber has a "calls" value to show subtraces, but in trace_replayBlockTransactions you have one flat array for all traces/subtraces. So, you will also need to restructure how you process subtraces.

Here is an example in Javascript on how to process the results:

    for (let i = 0; i < tracesResults1.result.length; i++) {
        const traces = tracesResults1.result[i].trace;
        for (let j = 0; j < traces.length; j++) {
            const trace = traces[j];
            <process trace>
        }
    }

vs

    for (let i = 0; i < tracesResults2.result.length; i++) {
        const trace = tracesResults2.result[i].result;
        processTraceRecursive(trace);
    }
    const processTraceRecursive = (trace) => {
        <process trace>
        if (trace.calls) {
            for (let i = 0; i < trace.calls.length; i++) {
                const subTrace = trace.calls[i];
                processTraceRecursive(subTrace);
            }
        }
    };

[racytech]

Hi everyone, this issue is being investigated. Since we are moving towards erigon3 (which has more optimized state history and faster sync time then version 2.x.x), we made many changes and some parts of the code in versions 2.x.x may left without necessary attention. As soon as the problem is detected it will be fixed. Thank you for your patience

[tjayrush]

Thanks for letting us know. I've been meaning to upgrade, so this will be impetus. Question for you though, first: I'm certain moving to 3.x requires a full re-sync, but does anyone have any idea how long that takes for a full archive node? Just need to plan ahead.

[racytech]

In the worst case scenario we estimate around 72 hours, but normally it should take around a day or less

[AskAlexSharov]

Thanks for letting us know. I've been meaning to upgrade, so this will be impetus. Question for you though, first: I'm certain moving to 3.x requires a full re-sync, but does anyone have any idea how long that takes for a full archive node? Just need to plan ahead.

Erigon3 does downloading most of the data. So speed depends on your network speed and —torrent.download.rate, we did manage sync ethmInnet in 8hours (and polygon mainnet in 2 days): https://erigon.substack.com/p/erigon-3-alpha-2-introducing-blazingly

[yperbasis]

Fixed by PR #12559 and will be released in v2.60.10.