[Feature]: Support for returning a value when using wait_for_save in v1 #21702
Description
🚀 The feature, motivation and pitch
Motivation
When attempting to use the v1 connector, communication issues between the scheduler and worker sides can be observed. Specifically, the scheduler-side connector struggles to detect which blocks the worker-side connector has successfully saved. We noticed that finished_sending defined in ModelRunnerOutput retrieve requests that have already sent the KV cache, but it is difficult to perceive whether there are blocks that can be successfully dumped. Currently, there is no common method to achieve this purpose.
Proposed Change
By adding new features in the Request and ModelRunnerOutput structures and returning the corresponding features through the wait_for_save function, we can achieve this purpose.
Code
Alternatives
No response
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.