-
Notifications
You must be signed in to change notification settings - Fork 259
Open
Description
This issue organizes the remaining tasks to bring SGLang feature support to parity with vLLM.
Features to support:
- Support
use_conversation_multi_turn=true. [Dependencies] Upgrade to torch 2.7 #73 observed issues with SGLang version0.4.8.post1returning an empty response when enabled. - Support SGLang colocated with training with TP==1 (pending in [Generator] Support non-remote (e.g. colocated) SGLang engine #68)
- Support SGLang colocated with training with TP>1
- Need to fix remote server sleep for SGLang, which currently uses vllm API
SkyRL/skyrl-train/skyrl_train/inference_engines/remote_inference_engine.py
Lines 144 to 147 in c88a678
async def sleep(self, *args: Any, **kwargs: Any): async with aiohttp.ClientSession() as session: resp = await session.post(f"{self.url}/sleep", json={"level": kwargs.get("level", 1)}) return await resp.json() - ...
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels