problem when reproducing Mistral-based InfLLM

[KangcongLi]

Traceback (most recent call last): File "/root/kcli/InfLLM/benchmark/pred.py", line 289, in <module> model, tokenizer = get_model_and_tokenizer(args.model) File "/root/kcli/InfLLM/benchmark/pred.py", line 56, in get_model_and_tokenizer model = patch_hf(model, config.type, **config) File "/root/kcli/InfLLM/inf_llm/utils/patch.py", line 152, in patch_hf hf_rope = model.model.layers[0].self_attn.rotary_emb File "/root/anaconda3/envs/lkc/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1729, in __getattr__ raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'") AttributeError: 'MistralAttention' object has no attribute 'rotary_emb'