Is there any reason not to use the tokenizer after applying LORA?

[seona21]

The official guide for HuggingFace uses the tokenizer of the base model (before LORA is applied) for inference.
Is there any reason not to use the tokenizer after applying LORA?

https://github.com/huggingface/peft/blob/main/examples/int8_training/Finetune_opt_bnb_peft.ipynb

[BenjaminBossan]

It is the same tokenizer. If you check the config that's loaded in this example, you'll see that base_model_name_or_path is "facebook/opt-6.7b":

https://huggingface.co/ybelkada/opt-6.7b-lora/blob/main/adapter_config.json#L2

[seona21]

in this case, why not using "ybelkada/opt-6.7b-lora" tokenizer, but using “facebook/opt-6.7b” tokenizer?

[BenjaminBossan]

There is no "ybelkada/opt-6.7b-lora" tokenizer, the LoRA model uses the same tokenizer as the original model.