The "-l" option of start_all.sh is undiscoverable and better be automatic

[nyh]

In prometheus/scylla_servers.yml I had

- targets:
       - 127.0.0.1:9180
       - 127.0.0.2:9180
       - 127.0.0.3:9180
  labels:
       cluster: cluster1
       dc: dc1

And it didn't work - start_all.sh didn't complain, but the resulting monitor did not see any of these nodes. Lucikly, @psarna who had exactly the same problem and read through the start_all.sh code, discovered that unless the "-l" option is given, prometheus (which is run inside docker, or in my case podman) cannot contact Scylla running on the host's loopback interface (127.0.0.1).

The problem is that this "-l" option is completely undiscoverable. After I was told of its existence, I found it is described in the README, but it's not something I noticed when I first read this README.

I think that instead of "-l" being an option, it should be completely automatic. I see two approaches to do that:

1. One approach is to always enable "-l". Is there is some performance penalty or other problem in always enabling it?
2. The other approach is for start_all.sh to check if prometheus/scylla_servers.yml has 127.*.*.* addresses, and if it does, turn on the "-l" option or at least print a message recommending to do so.

[amnonh]

using the host network comes with some limitations (port mapping for example) I did think about adding a warning if someone used the localhost, we decided against it because it it's not only the 127/24 range, but also localhost and any local mapping to localhost.

[psarna]

Still, checking for 127.* and localhost would probably help 99% of the cases - maybe the idea about adding a warning should be revisited.

[amnonh]

@psarna that's why I keep this issue open

[nyh]

Thanks @amnonh. I think this is a very important problem to fix. I know you like docker and how Prometheus and Grafana runs inside it, but the users - like myself - either aren't aware of this, or aren't aware exactly how it means. If your script takes a user's "external" (outside docker) configuration file, any file name or IP address or whatever inside this configuration file should refer to the user's "external" setup, not to hidden stuff inside docker. This is why I labeled this issue a "bug" and not a "feature request". It is a bug that IP addresses specified in the configuration file do not mean what they are expected to mean.