[Bug][Benchmark] Fix duplicate req in oversampling

[ekagra-ranjan]

There are duplicate requests ids during oversampling in vllm bench serve which causes issue. The server throws these logs

(APIServer pid=27557) ERROR 10-02 13:53:06 [serving_chat.py:1145] Error in chat completion stream generator.                                        
(APIServer pid=27557) ERROR 10-02 13:53:06 [serving_chat.py:1145] Traceback (most recent call last):                                                
(APIServer pid=27557) ERROR 10-02 13:53:06 [serving_chat.py:1145]   File "/host/vllm-ekagra/vllm/vllm/entrypoints/openai/serving_chat.py", line 574,
 in chat_completion_stream_generator                                                                                                                
(APIServer pid=27557) ERROR 10-02 13:53:06 [serving_chat.py:1145]     async for res in result_generator:                                            
(APIServer pid=27557) ERROR 10-02 13:53:06 [serving_chat.py:1145]   File "/host/vllm-ekagra/vllm/vllm/v1/engine/async_llm.py", line 376, in generate
(APIServer pid=27557) ERROR 10-02 13:53:06 [serving_chat.py:1145]     q = await self.add_request(                                                   
(APIServer pid=27557) ERROR 10-02 13:53:06 [serving_chat.py:1145]         ^^^^^^^^^^^^^^^^^^^^^^^                                                   
(APIServer pid=27557) ERROR 10-02 13:53:06 [serving_chat.py:1145]   File "/host/vllm-ekagra/vllm/vllm/v1/engine/async_llm.py", line 289, in add_requ
est                                                                                                                                                 
(APIServer pid=27557) ERROR 10-02 13:53:06 [serving_chat.py:1145]     await self._add_request(request, prompt_str, None, 0, queue)                  
(APIServer pid=27557) ERROR 10-02 13:53:06 [serving_chat.py:1145]   File "/host/vllm-ekagra/vllm/vllm/v1/engine/async_llm.py", line 315, in _add_req
uest                                                                                                                                                
(APIServer pid=27557) ERROR 10-02 13:53:06 [serving_chat.py:1145]     self.output_processor.add_request(request, prompt, parent_req, index,         
(APIServer pid=27557) ERROR 10-02 13:53:06 [serving_chat.py:1145]   File "/host/vllm-ekagra/vllm/vllm/v1/engine/output_processor.py", line 366, in a
dd_request                                                                                                                                          
(APIServer pid=27557) ERROR 10-02 13:53:06 [serving_chat.py:1145]     raise ValueError(f"Request id {request_id} already running.")                 
(APIServer pid=27557) ERROR 10-02 13:53:06 [serving_chat.py:1145] ValueError: Request id chatcmpl-benchmark-serving146 already running.

This is because the deepcopy on multiple elements selected by random.choices still share the same base address. If the random.choice samples 1 request multiple times then the request id set during the last iteration will be shared by all the instances of the same request. This PR fixes it by deepcopying it one at a time. The PR adds a assertion to confirm that no duplicate req are found. The same assertion fails when the changes in this PR are not made.

[gemini-code-assist]

Code Review

This pull request correctly fixes a bug in the request oversampling logic that caused duplicate request IDs. The original implementation with deepcopy(random.choices(...)) did not create unique objects for requests sampled multiple times. The fix, which involves deep-copying requests individually within a loop, effectively resolves this issue. Additionally, a validation check has been added to detect duplicate request IDs. However, I've found a flaw in this new validation logic that could allow duplicates to go undetected under certain conditions and have suggested a correction.