Parity crashes when executing trace_replayBlockTransactions on blocks related to the 2016 DDoS attack.

[begelundmuller]

I'm running:

• Which Parity version?: Parity/v1.11.3-beta-a66e36b-20180605/x86_64-linux-gnu/rustc1.26.1
• Which operating system?: Ubuntu 16.04
• How installed?: curl (get.parity.io)
• Are you fully synchronized?: yes
• Which network are you connected to?: ethereum
• Did you try to restart the node?: yes

Issue: Parity crashes when I try to replay the transactions in block 2283417 (and a range of other blocks related to the Fall 2016 DDoS attack).

I am replaying transactions as follows:

const payload = {
  "method": "trace_replayBlockTransactions",
  "params": [Web3.utils.toHex(blockNumber), ["trace", "stateDiff", "vmTrace"]],
  "id": 1,
  "jsonrpc": "2.0"
};

rpcProvider = new Web3.providers.HttpProvider(rpcHost);
rpcProvider.send(payload, function (error, result) {
  if (error) {
    reject(error);
  } else {
    resolve(result);
  }
});

I start Parity as follows:

parity --base-path=/home2/.local/share/io.parity.ethereum --pruning archive --tracing on --jsonrpc-hosts all --jsonrpc-interface all --jsonrpc-apis eth,traces --no-ui --no-ws --no-dapps --cache-size=24576 --jsonrpc-threads=4 --jsonrpc-server-threads=4 --max-peers=200 --min-peers=100

It's running on an 8-core, 32GB RAM box with a 2TB SSD disk.

How can I debug this issue?

[Tbaut]

Hey, Thanks for the report. Could you monitor your CPU, RAM and tell if one of them goes through the roof?
Does it print anything when it crashes?

[begelundmuller]

Executing on block 2379718 (also contains DDoS transactions), I see Parity halt with the following message:

thread '<unknown>' has overflowed its stack
fatal runtime error: stack overflow
Aborted (core dumped)

I'll see if I can get some CPU/RAM stats. Meanwhile, a) how should I run Parity to have it output more useful logs, and b) is there any configuration change I can make that would make it easier for Parity to replay these blocks (after all, it was able to run the transactions when I synced)?

[Tbaut]

Given your beefy configuration, I'd try with a bigger cache size. Regarding the logs, https://wiki.parity.io/FAQ#how-can-i-make-parity-to-write-logs I don't think there is any useful flag to give you a better trace.

[tzapu]

seems we are getting the same on 2288034
--cache-size was 22000, trying with larger now

--auto-update=none --base-path=/paritydb --mode=active --tracing=on --pruning=archive --db-compaction=ssd --scale-verifiers --num-verifiers=6 --jsonrpc-server-threads=6 --jsonrpc-threads=6 --cache-size=22000 --min-peers=100 --max-peers=1000

Parity/v1.11.1-beta-6654d02-20180515/x86_64-linux-gnu/rustc1.26.0

[tzapu]

with the following settings

command: ["/bin/bash"]
    args:
      - "-c"
      - |-
        ulimit -a;
        ulimit -S -s 32000;
        ulimit -a;
        exec /parity/parity \
          --auto-update=none \
          --base-path=/paritydb \
          --mode=active \
          --tracing=on \
          --pruning=archive \
          --db-compaction=ssd \
          --scale-verifiers \
          --num-verifiers=6 \
          --jsonrpc-server-threads=6 \
          --jsonrpc-threads=6 \
          --cache-size=32000 \
          --min-peers=100 \
          --max-peers=1000 \

    env:
      - name: "RUST_MIN_STACK"
        value: "127108864"

using latest docker image and running only one trace_replayBlockTransactions it did the trace

[Tbaut]

@tzapu Thanks for the feedback. Please upgrade to the latest release as it contains a security fix (since v1.11.3)

@bem7 could you please try with the proposed configuration?

[tzapu]

@Tbaut thanks, forgot to mention, the last test was made with the latest docker image

[debris]

#9360 will resolve this issue

[folsen]

#9360 is now merged, can anyone confirm that it did indeed fix the issue?