Skip to content

Leaderworkerset v0.4.0

Compare
Choose a tag to compare
@liurupeng liurupeng released this 14 Sep 00:09
· 75 commits to main since this release
v0.4.0
edc9eac

Features:

  • Support specify NetworkConfig
  • Support leader elect for lws controller
  • Add group size as an environment variable
  • Add an example for using llama.cpp to deploy a distributed inference service
  • Update the GPU multi-node inference with vLLM example to serve Llama3.1-405b model

What's Changed

New Contributors

Full Changelog: v0.3.0...v0.4.0