Proposal for Mempool Congestion #358
Comments
|
#1 is already coded in by ranking by fee/size?
…On Sat, 25 Aug 2018, 15:42 decentralisedkev, ***@***.***> wrote:
Below I propose three policies to help guard against the mempool being
flooded. The three would be used in conjunction with each other.
*Problem 1: A user can send large transactions instead of multiple small
ones, which also hogs memory.*
Solution: The bigger a transaction is, the more in fees it should cost to
send. This can be done using the txsize.
*Problem2: When the network is being spammed, nodes will make it worse by
propagating the malicious transactions.*
Solution: Each node will have a minRelayFee. This fee will be based on the
size of the mempool and the rate at which transactions are being added. If
a lot of transactions are added in a small amount of time and they all have
low fees, they will not pass the minRelayFee threshold and will not be
propagated. If the nodes mempool is full, then it will take a large fee for
that node to relay the transaction. Importantly, he will not discard it.
*Problem3: When the mempool is full, it stays full for a long period of
time.*
Solution3: Transactions upon being added can be stamped with a time, If a
transaction has been in the mempool for longer than 5/10 minutes, it should
be flushed, and the user would need to re-submit their transaction.
*Counter Arguments*
-
During the time that the attacker is attacking the network, fees will
be high and so good transactions trying to get in, will be delayed or will
need to pay high fees.
-
One issue brought up by Igor, is that some attackers would be willing
to pay millions to take down promising blockchains.
With the above model it would make it expensive for an attacker to do so,
the fees would go back into the economy, however it still stands that the
more money the attacker has to waste the longer the attack would last for.
- Nodes may find it hard to agree on a set of transactions.
With the model, the subset of transactions that would be agreed upon would
be the ones with the highest fees, as these would be the ones that are
being propagated.
*Implementation*
Implementation would not require a fork.
*Notes*
In order for a malicious actor to spam the network, he will need to keep
increasing the fee, the faster he spams, the more he has to pay, in order
for his transaction to propagate. The attacker would need to adjust the fee
for each node as node mempool sizes are different, so they will have
different minRelayFees. The attacker could mitigate this by attaching a
large than normal fee, and increasing it by some factor.
*Improvements*
The amount being sent could also be added as a factor, when deciding on
whether to relay. A case would be where the txsize is small, however the
amount is large because there are not many inputs/outputs.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#358>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/Amt4jl5Pb-h4NZA2jWRxGZegDK2iWtVtks5uUWJrgaJpZM4WMaOD>
.
|
|
@f27d I thought that only ranked them? My proposal was to not accept them at all, If a solution is made to problem3 then either way would work though. |
|
Yes you are right - thanks. I was thinking of when we get to above 500tx in
mempool on a normal day then the ranking effectively does this
…On Sat, 25 Aug 2018, 15:57 decentralisedkev, ***@***.***> wrote:
@f27d <https://github.com/f27d> I thought that only ranked them? My
proposal was to not accept them at all, If a solution is made to problem3
then either way would work though.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#358 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/Amt4jmhgNb78vViRH4lU-hlwtk5KnqHVks5uUWXlgaJpZM4WMaOD>
.
|
|
Problem 1 Like this a lot. We had some spam a while back where single transactions had huge sizes. If we implement a minimum transaction fee, I'd like to see it scale based on transaction size. Problem 2 How would the proposed minRelayFee affect services that use many small transactions, e.g Effect.ai? Problem 3 I like the idea of a TTL for transactions in the mempool, but I think it needs to be fine-tuned to ensure the average user experience is not harmed, even during spam. I think TTL should scale based on the number of transactions that address has in the mempool. People are going to get very frustrated if they send one transaction that goes nowhere for 5-10 minutes and find out it has been dropped. |
For this, if they are sending so many that it could be seen as spam, then the node will not relay the transaction without a high enough fee attached. It can be the case that the relay fee needed is higher than the amount that is being sent for that node. The minRelayFee would be recalculated as the mempool drains, so if it is not spam then eventually it will get propagated and confirmed. If Effect.ai were to keep sending large amounts of small transactions with no fees in a short amount of time, then they will not be propagated and the majority thrown away.
My thinking with this is that since we have 30 second blocks, if your tx has not been approved within 10 - 20 blocks, you may as well re-send. You are right in that the most important thing is the TTL, so if we can find a TTL solution with better UX that would be ideal |
|
@f27d I was proposing a more dynamic approach to it. The 500tx approach is simpler and could be the more optimal way, I don't have any benchmarks so I cannot make heads or tail, which is better under stress |
|
Good ideas Kev, I guess we need to move from a static decision making chain to a self adaptive dynamic one... and all involved elements must contribute to an overall good experience on the network. |
erikzhang
added
the
Discussion
|
Hi @decentralisedkev, I am interested in the method of calculation you had in mind regarding ratcheting:
You use the imprecise English "if a lot of transactions... small amount of time... all have low fees". How do you envisage this mathematically? That's three separate variables, I'm interested in how the relay fee or threshold evolves as a function of x, y and z. Thanks for your contribution, I think your three ideas have merit. |
|
Hey @penlite , I think that another solution will be implemented. I will hash out something for your entertainment though. Assumptions:
From these assumptions, I will define spamming as a period of time where
At this point, you could do the calculations in either number of transactions per second using 135 transactions or bytes per second using 40kb. I will use the bytes per second as it will lead to more accurate results, plus it will complement the mempool calculation being done over some dynamic time t, that each node would be able to set based on their cpu usage, memory and or amount of transactions they have received in a period of time. Let ms = The size of the mempool in bytes. Let minRelayFee = some initial small agreed upon Over some delta t the minRelayFee can be calculated can be recalculated with: Where dms/dt is the change in the mempool size over change in time. As noted above, if delta t needs to be calculated faster, it can be adjusted. An example follows: If mempool size = 5MB at time t=0, and grows to 10MB at time t=5. So the mempool has grown by 1MB in a second. Dividing the number by 40000 = 25. The min relay fee would then be 25 satoshi + m satoshi, so maybe 30 satoshi. If you are spamming, I believe that this adds up. The punishment can be increased from 1 satoshi to 10 or more, that was just an arbitrary number that I plugged for simplicity of calculations. As spam goes down dms/dt would become negative, lowering the minRelayFee. A floor would be set at Notes:
Edit: Here is the other solution I mentioned from my friend Igor: #357 |
|
This solution could actually work; I have one other proposal that has some similarities to #357 but doesn't require sending any special stake transaction and would be backward compatible for all existing accounts. I've started working on a potential NEP proposal. I'll add a short description of the idea as a comment on #357. |
|
Good ideas Kev, sorry for the short time these days, I'll comment here as soon as work stack is a little lighter. Anyway, we need multiple integrated solutions to achieve more robust network. |
|
Hi Guys, Did any of the suggested fixes go in with 2.9 as the network is being spammed again and is borderline unusable right now? |
|
@bpetridis for now, the safest thing for users when mempool is full, in my opinion is to attach a time limit script: CityOfZion/neon-js#311 |
|
This is less relevant after #500 since a full mempool won't be a big issue, however I think this one actually shouldn't be closed by 500; maybe the title could be changed to something like prevent spam from bad actors |
|
Since we have high and low priority fee transactions now though it is less of an issue. I think possibly throttling bad actors should be a new issue and we close this one by #500. |
Below I propose three policies to help guard against the mempool being flooded. The three would be used in conjunction with each other.
Problem 1: A user can send large transactions instead of multiple small ones, which also hogs memory.
Solution: The bigger a transaction is, the more in fees it should cost to send. This can be done using the txsize.
Problem2: When the network is being spammed, nodes will make it worse by propagating the malicious transactions.
Solution: Each node will have a minRelayFee. This fee will be based on the size of the mempool and the rate at which transactions are being added. If a lot of transactions are added in a small amount of time and they all have low fees, they will not pass the minRelayFee threshold and will not be propagated. If the nodes mempool is full, then it will take a large fee for that node to relay the transaction. Importantly, he will not discard it.
Problem3: When the mempool is full, it stays full for a long period of time.
Solution3: Transactions upon being added can be stamped with a time, If a transaction has been in the mempool for longer than 5/10 minutes, it should be flushed, and the user would need to re-submit their transaction.
Counter Arguments
During the time that the attacker is attacking the network, fees will be high and so good transactions trying to get in, will be delayed or will need to pay high fees.
One issue brought up by Igor, is that some attackers would be willing to pay millions to take down promising blockchains.
With the above model it would make it expensive for an attacker to do so, the fees would go back into the economy, however it still stands that the more money the attacker has to waste the longer the attack would last for.
With the model, the subset of transactions that would be agreed upon would be the ones with the highest fees, as these would be the ones that are being propagated.
Implementation
Implementation would not require a fork.
Notes
In order for a malicious actor to spam the network, he will need to keep increasing the fee, the faster he spams, the more he has to pay, in order for his transaction to propagate. The attacker would need to adjust the fee for each node as node mempool sizes are different, so they will have different minRelayFees. The attacker could mitigate this by attaching a large than normal fee, and increasing it by some factor.
Improvements
The amount being sent could also be added as a factor, when deciding on whether to relay. A case would be where the txsize is small, however the amount is large because there are not many inputs/outputs.
The text was updated successfully, but these errors were encountered: