[Feature] Implement a "blocked" servers list

[alexjustesen]

📃 Description

In some cases "most" servers available locally are ok but sometimes they're not preferred and shouldn't be used when running a speedtest.

• Add "Exclude Speedtest Servers" configuration option #1438
• [idea] server blacklist #17

This issue will track the research needed to implement a blocked server list as the cli doesn't support this use-case.

🤔 Research

Since the cli doesn't support --exclude anymore we'll need to get a list of local servers first and intentionally choose one while excluding and "blocked" servers.

Options

1. A list of local servers can be found by running speedtest --servers --format=json which returns a list of servers using Ookla's opinion of which is "best for you".
2. A list of servers can also be found by hitting the api endpoint https://www.speedtest.net/api/js/servers. This looks close to the "best for you" cli list but does have some variance with my testing.

In my option 1 is likely the best source as it's "probably" the method the cli uses to choose a server anyways.

🚶‍♂️ Steps

If a blocked server list exists take the following steps:

1. Get a list of servers.
2. Remove the blocked servers from the list.
3. Choose the "best" server by using the first one in the list.
4. Run the speedtest.

[nwh3365]

This is great news, Alex! The steps look spot-on, and I like Option 1 for generating a list of servers to choose from for the reason you stated.

This method is essentially the same as that used by another script I use from time to time and it has been very reliable. It allows the closest servers list to change dynamically as servers come and go over time, while still being able to filter out the slow ones that are identified.

Also, --selection-details (show server selection details) seems to produce the same list of servers as --servers (aka -L), but includes what I believe is the idle latency and in a slightly different format. I only mention this because if you specify --selection-details, it picks the first server in the list to run the test, so it does appear that the list is an ordered list, although it doesn't appear that the order is based strictly on latency.

speedtest --servers
Closest servers:

    ID  Name                           Location             Country
==============================================================================
  8862  CenturyLink                    Denver, CO           United States
 15597  BAM Broadband                  Littleton, CO        United States
 16968  Spectrum                       Centennial, CO       United States
 17687  Hitron Technologies Americas   Centennial, CO       United States
 47683  NextLight                      Denver, CO           United States
 16797  Nitel                          Denver, CO           United States
 24079  ALLO - Denver                  Denver, CO           United States
 51010  Highline                       Denver, CO           United States
 23628  Whitesky Communications LLC    Denver, CO           United States
 10051  Comcast                        Denver, CO           United States
  9912  fdcservers.net                 Denver, CO           United States


speedtest --selection-details

   Speedtest by Ookla

Selecting server:
       8862:   1.93 ms; CenturyLink - Denver, CO
      15597:   3.35 ms; BAM Broadband - Littleton, CO
      16968:  41.97 ms; Spectrum - Centennial, CO
      17687:   2.93 ms; Hitron Technologies Americas - Centennial, CO
      47683:   2.61 ms; NextLight - Denver, CO
      16797:   2.56 ms; Nitel - Denver, CO
      24079:   2.94 ms; ALLO - Denver - Denver, CO
      51010:   2.65 ms; Highline - Denver, CO
      23628:   2.99 ms; Whitesky Communications LLC - Denver, CO
      10051:   3.03 ms; Comcast - Denver, CO
       9912:   2.75 ms; fdcservers.net - Denver, CO
      Server: CenturyLink - Denver, CO (id: 8862)
         ISP: CenturyLink

[alexjustesen]

When dumping to json looks like the latency is missing but as long as the order is maintained I suppose that doesn't matter.

{
  "id": 56485,
  "host": "secaucus.nj.speedtest.frontier.com",
  "port": 8080,
  "name": "Frontier",
  "location": "Secaucus, NJ",
  "country": "United States"
},

[nwh3365]

The latency is outside of the "server" structure:

{"latency":1.1259999999999999,
"server":{
  "id":8862,
  "host":"denver.speedtest.centurylink.net",
  "port":8080,
  "name":"CenturyLink",
  "location":"Denver, CO",
  "country":"United States"
}
}

BUT...I think you can use the simpler speedtest -L to generate the list as you originally suggested; it appears to be the same list of servers in the same order according to my testing. The only thing I was pointing out is that --selection-details generates the list and runs a test using the first entry, which would indicate that the list is in order of preference, and since that order matches -L, the -L list is likely also in order of preference. Interestingly, the list is not in order of increasing idle latency as illustrated in my earlier post which indicates that there are likely other factors that go into the ordering process.

[hcgatewood]

Looking forward to this, thanks so much! Also, if there's any chance of being able to configure this via the UI over time, that would feel like a big win as well