GetBlocks may have a returned data ratio of 1:31250

[obscuren]

Assuming a block can be as large as 1Mbyte (this is not specified in the code, but it's a reasonable value) then MsgGetBlocksTy has an return data ratio of 1:31250, which means that a peer sending 32 bytes can get 1 Mbyte of data in return. Two attacks are possible:

1. It's possible that a node sends a MsgGetBlocksTy packet and then closes silently the local socket, but does not send the CLOSE/FIN packet. The victim will send up to a maximum of the TCP buffer size, which depending on the operation system default and the negotiated connection parameters, can be between 8Kb to 1Gb (but normally is in the 8 - 64 Kbyte range). Then the attacker may perform multiple DoS attacks simultaneously to different peers, without even processing the incoming data.
2. The attacker can ask for 300 blocks but never read anything from the incoming TCP endpoint. Assuming each block is 1Mbyte in size, this will force the victim node to buffer 300 Mbytes of information. The attacker can continuously send MsgGetBlocksTy messages until the victim node consumes all available process memory or begins trashing. The current ping/pong keep-alive method is ineffective because it does not include a random nonce, so the attacker may send MsgGetBlocksTy messages interleaved with MsgPongTy messages. This attack also works if the block is much smaller in size (e.g. 1 Kbyte), because the attacker can request the same block multiple times.
   In both cases the Also the attacker can re-connect to the victim node from the same or another IP address when the victim node closes the connection, since a free incoming connection slot will become available.

Recommendations

• A node should not accept MsgGetBlocksTy messages until the the output queue is near empty.
• The output queue should have a maximum size.
• A node should not accept any message if the output queue is near full.
• The ping and pong messages should have a random nonce value to detect if pong messages are related to ping messages or not.
• The random nonce sent in the ping message should not be taken from the same random pool as the ECDSA signing or private key generation random pool.
• A node requesting the same block in the same MsgGetBlocksTy message should be punished.
• A node should force a delay in processing messages from a node that is requesting the same block over and over, or too many blocks. Also a peer should ban a node that asks for a high amount of information and then stops responding to ping messages.

[fjl]

Package p2p currently maintains a 64k output buffer. Message writes block until they are delivered (ack'd on the other side). The full data for outgoing messages is never buffered.

[obscuren]

What happens if the output buffer is full? Do we kick off peers that are stalling?

[fjl]

All protocols will block until the data has been read. The buffer is per connection.

[fjl]

OK, wait, I mixed it up with incoming message reads. If the protocol encodes 300 blocks, they will all sit in memory (encoded) until read. It would be good to limit the number of blocks being sent in a single message.

I'm also mistaken about the buffer size, the used (default) buffer size is 4k.

[fjl]

The concern about the TCP buffer remains. We can limit the size of the buffer using net.TCPConn.SetWriteBuffer. Please note that the write timeout for any message is currently 5 seconds.

When a write error (e.g. timeout) occurs, the protocol will usually return from its main loop and the peer will be disconnected. It can also ignore the error, but shouldn't. I have a TODO in the code to also handle (serious) write errors directly on the connection level without relying on the protocol to report them.

[fjl]

All in all, I think we are prepared for this scenario, mostly because we do not maintain an unbounded output queue and because we limit the number of active peers.

[obscuren]

👍 Do limit the output buffer.

[fjl]

I will also add a wrapper for p2p.MsgReadWriter that limits the incoming/outgoing message size.

func LimitRW(maxWriteSize, maxReadSize uint32, rw MsgReadWriter) MsgReadWriter

[fjl]

This issue is very old (it was raised during an audit of the previous p2p layer). The situation has improved as follows:

• We send at most 64 blocks
• Message sends must finish within 5 seconds
• We do not maintain an unbounded output queue