LLaVA image_token_index is not 64000 but 64002

[XuweiyiChen]

System Info

LLaVA image_token_index is not 64000 but 64002 in the latest version of the code. (main)

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

1. pip install the latest transformer from main branch
2. run llava 34b model.

Expected behavior

You can do inference freely but now an error will pop out saying there is 0 image token.

[NielsRogge]

Hi,

Thanks for reporting, this is also reported here: https://huggingface.co/llava-hf/llava-v1.6-34b-hf/discussions/2 and will be resolved by #29797