[Security] Document StatelessProcessGroup security concerns

A recent PR, vllm-project#15988, improved StatelessProcessGroup to ensure the
torch.distributed TCPStore uses the specified IP address unless of
binding to all interfaces. Upon closer inspection, this is quite
important, as the way vllm is using this TCPStore includes pickled data,
so malicious access to the TCPStore would allow remote code execution on
a vllm host.

Update some places throughout the code base to reflect the importance of
specifying a secured IP addres for use with this interface.

Finally, fix a couple places in tests to explicitly use localhost
instead of the IP we find that's (probably) the one used for the host's
default route. Otherwise, a host running these tests is briefly
vulnerable on the IP address chosen.

Signed-off-by: Russell Bryant <rbryant@redhat.com>

Author: russellb

examples/offline_inference/rlhf.py

@@ -26,6 +26,12 @@
 supports multi-node clusters. vLLM expects the GPUs are only used for vLLM
 workloads. Residual GPU activity interferes with vLLM memory profiling and
 causes unexpected behavior.
+
+It is important to set `VLLM_HOST_IP` to an address on a secure network when
+using this example. Unsecured communications between components will be used
+over this IP address and should NOT be exposed to untrusted networks. For more
+information, see:
+https://docs.vllm.ai/en/latest/deployment/security.html
 """
 
 import os

examples/offline_inference/rlhf_utils.py

@@ -27,8 +27,14 @@ class WorkerExtension:
     By defining an extension class, the code can work no matter what is
     the underlying worker class. This way, the code can be compatible
     with both vLLM V0 and V1.
+
     NOTE: we define this class in a separate module, and the main module
     should pass the full qualified name as `worker_extension_cls` argument.
+
+    The `master_address` parameter should be an address on a secure network that
+    is ideally completely isolated. Services used on this network are insecure
+    and will make the system vulnerable to remote code execution if exposed to
+    malicious parties.
     """
 
     def init_weight_update_group(

tests/distributed/test_same_node.py

@@ -7,20 +7,20 @@
 
 from vllm.distributed.parallel_state import in_the_same_node_as
 from vllm.distributed.utils import StatelessProcessGroup
-from vllm.utils import get_ip, get_open_port
+from vllm.utils import get_open_port
 
 if __name__ == "__main__":
     dist.init_process_group(backend="gloo")
 
     rank = dist.get_rank()
     if rank == 0:
         port = get_open_port()
-        ip = get_ip()
+        ip = "127.0.0.1"
         dist.broadcast_object_list([ip, port], src=0)
     else:
-        recv = [None, None]
+        recv = [None, None]  # type: ignore
         dist.broadcast_object_list(recv, src=0)
-        ip, port = recv
+        ip, port = recv  # type: ignore
 
     stateless_pg = StatelessProcessGroup.create(ip, port, rank,
                                                 dist.get_world_size())

vllm/distributed/utils.py

@@ -388,6 +388,10 @@ def create(
         used for exchanging metadata. With this function, process A and process B
         can call `StatelessProcessGroup.create` to form a group, and then process A, B,
         C, and D can call `StatelessProcessGroup.create` to form another group.
+
+        The `host` parameter should be an address on a secure network that is ideally
+        completely isolated. Services used on this network are insecure and will make
+        the system vulnerable to remote code execution if exposed to malicious parties.
         """ # noqa
         launch_server = rank == 0
         if launch_server: