Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Throttle validation of incoming messages #46

Closed
keks opened this issue Nov 16, 2017 · 2 comments
Closed

Throttle validation of incoming messages #46

keks opened this issue Nov 16, 2017 · 2 comments
Assignees

Comments

@keks
Copy link
Contributor

keks commented Nov 16, 2017

When #45 is in, we'll need to find a way to throttle parallel validator executions, so if they take a long time we don't run out of memory.

I plan on doing this by passed them all a buffered chan struct{} that they write to before they are started and read from when they finish. Question is: Should we just block the event loop or should we drop a message when the maximum number of validators is running? I think we already drop messages in a similar situation, so I'd propose going the same way here.

This leads us to the next problem, which is that validators that block very long need to be cancelled because otherwise they'll start piling up until either the event loop blocks or we drop all packets. So, how large should the timeout be? 200ms?

ping @vyzo @whyrusleeping

@keks keks self-assigned this Nov 16, 2017
@vyzo
Copy link
Collaborator

vyzo commented Nov 17, 2017

It makes sense to drop messages when the validation queue is full, so that it doesn't grow unbounded.
Even a simple tail drop strategy might be sufficient, if it works for the Internet it works for us too :)

@Stebalien
Copy link
Member

We should also disconnect from peers that send us bad data (data that fails to validate). It would be kind of nice to get a blacklist mechanism in libp2p as well but that's a different issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants