Network benchmarking

[ggwpez]

The idea of adding a network benchmark came up in the chat. The reason is that validators have a required minimum network speed of 500 Mbit/s.
It seems that multiple hosters promise one thing and then deliver another. Adding a network benchmark will help with sieving out subpar hardware quicker.

How?

Some infrastructure would be required for this. There are three ways that I can think of (feel free to extend 😆):

• A. Provide own infrastructure: some GCP servers in multiple locations with fast internet.
• B. Rent a service that already offers A.
• C. (Ab)use existing infrastructure, like GDrive or free services that provide A.

cc @koute

[bkchr]

Adding some networking benchmark sounds nice first. However, we would need to ensure that we don't ddos someone. And 1000 other things that we need to ensure.

IMO this should be solved by recommending some external tool in the validator setup to verify their download/upload speed.

[koute]

Yeah, this is tricky. As Basti said, there are many potential problems here that need to be handled to make sure this is reliable and doesn't DoS any third-party.

The way I see it, if we want to do this and have the benchmark be fully automatic and do it at least semi-reliably then I don't really see any other way than having our own infrastructure that we control.

Basically, something like this (just writing out anything which comes to my mind):

• Have multiple servers in multiple geographical regions in multiple different datacenters.
• When benchmarking make sure we pick a server which has enough spare bandwidth. (Other people currently benchmarking shouldn't affect your results.)
• When benchmarking make sure to not pick a server in the same datacenter. (e.g. if someone's running their node in the same GCP datacenter they'll have blazingly fast connection to anyone else who does, but this might not be true on a global scale when looking at the whole swarm of nodes, and that is what we care about)
• When benchmarking make sure we pick multiple servers across the globe. (Only a single server might have a temporary issue with its connection which might not be indicative of the user's network connection.)
• Deal with possible inherent bandwidth limitations of some of the servers. (e.g. if we'll have a server somewhere in Siberia where the connectivity is poor to essentially everyone else this shouldn't tank everyone's benchmarks, because the problem here is not the user's connection, but the server's; so we'd have to potentially benchmark the servers themselves and use that to weight their contribution)

All in all, a lot of potential problems, probably way too many. Trying to implement this would be a fun technical challenge, but I'm not entirely convinced it'd be worth it; perhaps like Basti said it'd be better to just YOLO it and delegate it to an external tool and expect it to be just a very rough guideline? I'm not sure.

[ggwpez]

Trying to implement this would be a fun technical challenge, but I'm not entirely convinced it'd be worth it; perhaps like Basti said it'd be better to just YOLO it and delegate it to an external tool and expect it to be just a very rough guideline? I'm not sure.

Yea also from a cost-benefit perspective this is nothing that we have to do.
Then I will look for some external tooling that already offers this and recommend it.

Leaving the issue open until this is done.