Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix taskManager #2176

Closed
wants to merge 8 commits into from
Closed

Fix taskManager #2176

wants to merge 8 commits into from

Conversation

Tommo-L
Copy link
Contributor

@Tommo-L Tommo-L commented Dec 23, 2020

No description provided.

@erikzhang
Copy link
Member

What is this?

@Tommo-L
Copy link
Contributor Author

Tommo-L commented Dec 23, 2020

What is this?

The last line will block nodes to relay transactions, even if the node is only one block behind.

private void OnNewTasks(InvPayload payload)
{
if (!sessions.TryGetValue(Sender, out TaskSession session))
return;
// Do not accept payload of type InventoryType.TX if not synced on best known HeaderHeight
if (payload.Type == InventoryType.TX && Blockchain.Singleton.Height < sessions.Values.Max(p => p.LastBlockIndex))
return;

@@ -95,7 +95,7 @@ private void OnNewTasks(InvPayload payload)
if (!sessions.TryGetValue(Sender, out TaskSession session))
return;
// Do not accept payload of type InventoryType.TX if not synced on best known HeaderHeight
if (payload.Type == InventoryType.TX && Blockchain.Singleton.Height < sessions.Values.Max(p => p.LastBlockIndex))
if (payload.Type == InventoryType.TX && sessions.Values.Where(p => p.LastBlockIndex > Blockchain.Singleton.Height + 12).Count() > sessions.Count / 2)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (payload.Type == InventoryType.TX && sessions.Values.Where(p => p.LastBlockIndex > Blockchain.Singleton.Height + 12).Count() > sessions.Count / 2)
if (payload.Type == InventoryType.TX && sessions.Values.Any(p => p.LastBlockIndex > Blockchain.Singleton.Height + 12))

What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it can be easily attacked.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think sessions.Count / 2 is too large.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adjusted to sessions.Count/3.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So as it is now with +12 it will accept more transactions? Also based on the number of peers it has?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before this, some seed nodes can't relay transactions to consensus nodes in preview4 testnet.

@roman-khimov
Copy link
Contributor

FYI, related: https://github.com/nspcc-dev/neo-go/blob/df6cb1101780c1acded30fc2befdd9d6d2048611/pkg/network/server.go#L414-L445

Copy link
Member

@shargon shargon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if I connect to you some nodes that never sync?

@superboyiii
Copy link
Member

Current testnet has this issue that Preview3 nodes are influencing Preview4 nodes because they're higher in peers list which makes tx can't be relayed.

@@ -95,7 +95,7 @@ private void OnNewTasks(InvPayload payload)
if (!sessions.TryGetValue(Sender, out TaskSession session))
return;
// Do not accept payload of type InventoryType.TX if not synced on best known HeaderHeight
if (payload.Type == InventoryType.TX && Blockchain.Singleton.Height < sessions.Values.Max(p => p.LastBlockIndex))
if (payload.Type == InventoryType.TX && sessions.Values.Where(p => p.LastBlockIndex > Blockchain.Singleton.Height + 12).Count() > sessions.Count / 3)
return;
Copy link
Member

@shargon shargon Dec 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that it's better to remove this lines, the mempool will work as expected, if it's filled, will be trimmed, so why we need to avoid this messages during sync based on untrusted information? Please check #2180

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the way neo-go currently works (IsInSync logic mentioned earlier is only used to determine when we should start dbft). The only problem with this is that the node can try relaying invalid transactions which can be interpreted as suspicious behaviour by its peers. But these transactions should at the same time fall within MaxValidUntilBlockIncrement range from current node's height which is not really likely to happen.

@superboyiii
Copy link
Member

Let's move on?

@superboyiii
Copy link
Member

superboyiii commented Dec 31, 2020

Already updated testnet with this fix, works well now. @erikzhang

@superboyiii
Copy link
Member

@erikzhang Merge?

@superboyiii
Copy link
Member

@erikzhang @shargon Merge?

@erikzhang
Copy link
Member

@Tommo-L I think maybe this is not the best solution. In NEO2, we sync the headers first, we can easily detect whether we are on the best known height. Now we sync the blocks directly, we can't do that.

@Tommo-L
Copy link
Contributor Author

Tommo-L commented Jan 14, 2021

Then we'd better merge this #2180 first, and we can add #1811 later.

@erikzhang
Copy link
Member

#1811 doesn't solve the problem.

@superboyiii
Copy link
Member

superboyiii commented Jan 18, 2021

Can we make out a good solution before Preview5 release? If not, maybe we could merge this first, although not the best way but at least solve the issue. Otherwise we'll meet the same issue on Preview5 testnet.

@Tommo-L
Copy link
Contributor Author

Tommo-L commented Jan 20, 2021

@Tommo-L I think maybe this is not the best solution. In NEO2, we sync the headers first, we can easily detect whether we are on the best known height. Now we sync the blocks directly, we can't do that.

If a node is too far behind, every time a block is synchronized, the transaction in the memory pool will be re-verified. In fact, this part is not very time-consuming, because most of the verification time is spent on verifying the signature.

In addition, we can optimize in the future. If multiple blocks are being synchronized, we can only use the last block(the highest height block) to verify transactions in the memory pool.

@erikzhang
Copy link
Member

In fact, this part is not very time-consuming, because most of the verification time is spent on verifying the signature.

This is uncertain, and we cannot rely on this assumption. Otherwise this will become a weakness.

@Tommo-L
Copy link
Contributor Author

Tommo-L commented Jan 20, 2021

Maybe we can derive the "best" header from UnverifiedBlock, then we can avoid excessive re-verify.

private VerifyResult OnNewBlock(Block block)
{
if (block.Index <= Height)
return VerifyResult.AlreadyExists;
if (block.Index - 1 > Height)
{
AddUnverifiedBlockToCache(block);

@Tommo-L
Copy link
Contributor Author

Tommo-L commented Jan 21, 2021

We'll add header back.

@Tommo-L Tommo-L closed this Jan 21, 2021
@@ -95,7 +95,7 @@ private void OnNewTasks(InvPayload payload)
if (!sessions.TryGetValue(Sender, out TaskSession session))
return;
// Do not accept payload of type InventoryType.TX if not synced on best known HeaderHeight
if (payload.Type == InventoryType.TX && Blockchain.Singleton.Height < sessions.Values.Max(p => p.LastBlockIndex))
if (payload.Type == InventoryType.TX && sessions.Values.Where(p => p.LastBlockIndex > Blockchain.Singleton.Height + 12).Count() > sessions.Count / 3)
Copy link
Contributor Author

@Tommo-L Tommo-L Jan 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if (payload.Type == InventoryType.TX && !sessions.Values.Any(p => p.LastBlockIndex > Blockchain.Singleton.Height  - 12 && p.LastBlockIndex < Blockchain.Singleton.Height + 12))

Will it be ok? I think one node must connect a correct node.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants