How to run for gemma3

[ghac101]

Hi,

maybe it's just me, but when I try to run a gemma3 model, I get the following error: "unknown model architecture: 'gemma3'".

Is this the expected behaviour, or should this work?

The cli I use is the following:
.\llamafile-0.9.1.exe -m .\gemma-3-1b-it-BF16.gguf --verbose

The full output it the following:

██╗ ██╗ █████╗ ███╗ ███╗ █████╗ ███████╗██╗██╗ ███████╗
██║ ██║ ██╔══██╗████╗ ████║██╔══██╗██╔════╝██║██║ ██╔════╝
██║ ██║ ███████║██╔████╔██║███████║█████╗ ██║██║ █████╗
██║ ██║ ██╔══██║██║╚██╔╝██║██╔══██║██╔══╝ ██║██║ ██╔══╝
███████╗███████╗██║ ██║██║ ╚═╝ ██║██║ ██║██║ ██║███████╗███████╗
╚══════╝╚══════╝╚═╝ ╚═╝╚═╝ ╚═╝╚═╝ ╚═╝╚═╝ ╚═╝╚══════╝╚══════╝
note: if you have an AMD or NVIDIA GPU then you need to pass -ngl 9999 to enable GPU offloading
llama_model_loader: loaded meta data with 32 key-value pairs and 340 tensors from ./gemma-3-1b-it-BF16.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = gemma3
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.name str = Gemma-3-1B-It
llama_model_loader: - kv 3: general.quantized_by str = Unsloth
llama_model_loader: - kv 4: general.size_label str = 1000M
llama_model_loader: - kv 5: general.repo_url str = https://huggingface.co/unsloth
llama_model_loader: - kv 6: gemma3.context_length u32 = 32768
llama_model_loader: - kv 7: gemma3.embedding_length u32 = 1152
llama_model_loader: - kv 8: gemma3.block_count u32 = 26
llama_model_loader: - kv 9: gemma3.feed_forward_length u32 = 6912
llama_model_loader: - kv 10: gemma3.attention.head_count u32 = 4
llama_model_loader: - kv 11: gemma3.attention.layer_norm_rms_epsilon f32 = 0.000001
llama_model_loader: - kv 12: gemma3.attention.key_length u32 = 256
llama_model_loader: - kv 13: gemma3.attention.value_length u32 = 256
llama_model_loader: - kv 14: general.file_type u32 = 32
llama_model_loader: - kv 15: gemma3.rope.freq_base f32 = 1000000.000000
llama_model_loader: - kv 16: gemma3.attention.sliding_window u32 = 512
llama_model_loader: - kv 17: gemma3.attention.head_count_kv u32 = 1
llama_model_loader: - kv 18: tokenizer.ggml.model str = llama
llama_model_loader: - kv 19: tokenizer.ggml.pre str = default
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,262144] = ["", "", "", "", ...
llama_model_loader: - kv 21: tokenizer.ggml.scores arr[f32,262144] = [-1000.000000, -1000.000000, -1000.00...
llama_model_loader: - kv 22: tokenizer.ggml.token_type arr[i32,262144] = [3, 3, 3, 3, 3, 4, 3, 3, 3, 3, 3, 3, ...
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 2
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 106
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 3
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 0
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = true
llama_model_loader: - kv 28: tokenizer.ggml.add_eos_token bool = false
llama_model_loader: - kv 29: tokenizer.chat_template str = {{ bos_token }}\n{%- if messages[0]['r...
llama_model_loader: - kv 30: tokenizer.ggml.add_space_prefix bool = false
llama_model_loader: - kv 31: general.quantization_version u32 = 2
llama_model_loader: - type f32: 157 tensors
llama_model_loader: - type bf16: 183 tensors
llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'gemma3'
llama_load_model_from_file: failed to load model
./gemma-3-1b-it-BF16.gguf: failed to load model

[cjpais]

Currently you have to compile llamafile from source and run it. A release will be coming out in the next week or two which natively supports it

[ghac101]

Thanks a lot, this has helped a lot ❤️