[Feature]: MiniCPM-O support beam_search #15694
Description
🚀 The feature, motivation and pitch
When running MiniCPM-O, I noticed some logs indicating that the model does not support beam search.
The log message states:
This model supports multiple tasks: {'reward', 'classify', 'generate', 'embed', 'score'}. Defaulting to 'generate'.
Additionally, using the following code results in an incorrect response:
llm = LLM(
model=minicpm_o_2_6_path,
trust_remote_code=True,
# gpu_memory_utilization=0.7,
# max_model_len=19456,
# max_num_seqs=5,
limit_mm_per_prompt={"audio": 1, "image":0, "video": 0}, # max to 40 audios ...
)
sampling_params = SamplingParams(temperature=0.,
max_tokens=250,
stop_token_ids=stop_token_ids,
repetition_penalty=1.05)
beam_params = BeamSearchParams(beam_width=3, max_tokens=200)
self.beam_params = beam_params
batch_outputs = self.llm.beam_search(batch_inputs, self.beam_params)
Alternatives
No response
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.