How to generate config.json for HFForCausalLM?

[CHNtentes]

I checked https://huggingface.co/fwtan/llama-1.1b-mobilequant-w4a8-s1024-e60-sym-hf, and found the config.json in this repo contains a lot of attributes which are used in HFForCausalLM.
I tried running convert_ckpt.py but it did not generate a new config.json. What should I do?

[fwtan]

I checked https://huggingface.co/fwtan/llama-1.1b-mobilequant-w4a8-s1024-e60-sym-hf, and found the config.json in this repo contains a lot of attributes which are used in HFForCausalLM. I tried running convert_ckpt.py but it did not generate a new config.json. What should I do?

Hi, you don‘t need to convert https://huggingface.co/fwtan/llama-1.1b-mobilequant-w4a8-s1024-e60-sym-hf using convert_ckpt.py. convert_ckpt.py is only required when the model is not in HFForCausalLM.

[CHNtentes]

I know that. I'm actually converting a model based on llama architecture (a little different), and convert_ckpt.py only generates a new model.safetensors. Won't it convert my llamaconfig to hfconfig? Or do I need to manually write a config.json based on your examples?

[CHNtentes]

I also tried downloading TinyLlama-1.1B-Chat-v1.0 from HuggingFace, and convert_ckpt.py still does not give me a config.json for HFForCausalLM...

[fwtan]

I also tried downloading TinyLlama-1.1B-Chat-v1.0 from HuggingFace, and convert_ckpt.py still does not give me a config.json for HFForCausalLM...

I manually prepared all the config.json files. You can re-use them for the supported models, e.g. TinyLlama, Gemma-2B, StableLM2-1.6B. Unfortunately, there was no script to convert the config.json of any new huggingface model.