[bugfix][distributed] fix multi-node bug for shared memory #6597
Conversation
|
👋 Hi! Thank you for contributing to the vLLM project. Once the PR is approved and ready to go, please make sure to run full CI as it is required to merge (or just use auto-merge). To run full CI, you can do one of these:
🚀 |
| assert self.shared_memory.size == self.total_bytes_of_buffer | ||
| try: | ||
| self.shared_memory = shared_memory.SharedMemory(name=name) | ||
| assert self.shared_memory.size == self.total_bytes_of_buffer # noqa |
There was a problem hiding this comment.
qq: what is this noqa for?
There was a problem hiding this comment.
This is super minor, but then you can parentheses
| assert self.shared_memory.size == self.total_bytes_of_buffer # noqa | |
| assert (self.shared_memory.size == self.total_bytes_of_buffer) |
There was a problem hiding this comment.
let's do it later, too busy rn :(
|
@youkaichao wouldn't the right thing to be to not attempt to use |
|
@njhill even for a group that spans nodes, we use shm for the ranks inside the node, and zmq for ranks outside the node. this is, actually, in preparation for the future, the alternative to ray dag implementation. |
…ect#6597) Signed-off-by: Alvant <alvasian@yandex.ru>
fixes #6564