fix: Fix tensor shape error, during llava inference.

[SeanCraven314]

Hi, thanks for your great work.

As in issue #39, I also encountered the same error: a small tensor dimension error. I added some logic to perform broadcasting, which solved the issue for me.

I haven't spent much time on this, and it hasn't been tested with all the model weight permutations. I am happy to do this if needed!

Regards,

Sean

[joebradly]

Thanks for your commit. I add the lines you changed. I think line 282 is redundant. And I still encounter the keyword tensor shape error.
`['./demo_images/av.png']

Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 25%|██▌ | 1/4 [01:04<03:14, 64.96s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [02:11<02:12, 66.08s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [03:19<01:06, 66.63s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [03:26<00:00, 43.39s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [03:26<00:00, 51.72s/it]
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results.
Setting pad_token_id to eos_token_id:128001 for open-end generation.
input: \n Please describe the traffic condition.
[WARNING] the auto inferred conversation mode is llava_v0, while --conv-mode is vicuna_v1, using vicuna_v1
torch.Size([1, 3, 384, 384])
Traceback (most recent call last):
File "/home/deping.zhang/code/llm/VILA/run_vila.py", line 153, in
eval_model(args)
File "/home/deping.zhang/code/llm/VILA/run_vila.py", line 115, in eval_model
output_ids = model.generate(
File "/home/deping.zhang/.conda/envs/vila/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/deping.zhang/code/llm/VILA/llava/model/language_model/llava_llama.py", line 171, in generate
outputs = self.llm.generate(
File "/home/deping.zhang/.conda/envs/vila/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/deping.zhang/.conda/envs/vila/lib/python3.10/site-packages/transformers/generation/utils.py", line 1764, in generate
return self.sample(
File "/home/deping.zhang/.conda/envs/vila/lib/python3.10/site-packages/transformers/generation/utils.py", line 2924, in sample
if stopping_criteria(input_ids, scores):
File "/home/deping.zhang/.conda/envs/vila/lib/python3.10/site-packages/transformers/generation/stopping_criteria.py", line 132, in call
return any(criteria(input_ids, scores) for criteria in self)
File "/home/deping.zhang/.conda/envs/vila/lib/python3.10/site-packages/transformers/generation/stopping_criteria.py", line 132, in
return any(criteria(input_ids, scores) for criteria in self)
File "/home/deping.zhang/code/llm/VILA/llava/mm_utils.py", line 299, in call
outputs.append(self.call_for_batch(output_ids[i].unsqueeze(0), scores))
File "/home/deping.zhang/code/llm/VILA/llava/mm_utils.py", line 281, in call_for_batch
raise ValueError(
ValueError: Keyword tensor should have 2 or 3 dimensions, got 1`