[Bug]: : ERROR 07-31 11:57:33 async_llm_engine.py:658] Engine iteration timed out. This should never happen!

[lucasjinreal]

Your current environment

: ERROR 07-31 11:57:33 async_llm_engine.py:658] Engine iteration timed out. This should never happen!```


### 🐛 Describe the bug

This code not work:

sampling_params = SamplingParams(temperature=0.2, max_tokens=1024, stop=["<|im_start|>", "<|im_end|>"], skip_special_tokens=False)

while True:
    try:
        inp = input(f"{roles[0]}: ")
    except EOFError:
        inp = ""
    if inp == "\\d":
        print("exit...")
        break

    if is_image(inp):
        images = load_multi_images_maybe(inp)
        # clear conv history
        conv.messages = []
        print("Updated image, start new chat session.")
        continue

    print(f"{roles[1]}: ", end="")

    if images is not None:
        # first message
        inp = f"{DEFAULT_IMAGE_TOKEN} " * len(images) + "\n" + inp
        if len(images) > 1:
            inp = convert_image_tags(inp)
        conv.append_message(conv.roles[0], inp)
    else:
        # later messages
        conv.append_message(conv.roles[0], inp)
    conv.append_message(conv.roles[1], None)
    prompt = conv.get_prompt()

    if args.debug:
        print(f"prompt_real: {prompt}")

    if images is not None:
        inputs = {
            "prompt": prompt,
            "multi_modal_data": {
                "image": images[0]
            },
        }
        # next don't need it
        images = None
    else:
        inputs = {
            "prompt": prompt,
        }

    results_generator = model.generate(inputs, sampling_params, str(time.monotonic()))
    outputs = ""
    async for request_output in results_generator:
        prompt = request_output.prompt
        if request_output.finished:
            print()
        else:
            out = request_output.outputs[-1].text
            if len(out) == 0:
                continue
            out_delta = out[len(outputs) :]
            print(out_delta, end="", flush=True)
            outputs = out

[DarkLight1337]

Please report this issue to #5901 so we can better investigate this.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!

[github-actions]

This issue has been automatically closed due to inactivity. Please feel free to reopen if you feel it is still relevant. Thank you!