Value of llama2 n_kv_heads?

[dnatarajan00]

I see that the value for "n_kv_heads" for llama2_7b in sku_list.py is set to 8.

        Model(
            
            core_model_id=CoreModelId.llama2_7b,
            description="Llama 2 7b model",
            huggingface_repo="meta-llama/Llama-2-7b",
            recommended_sampling_params=recommended_sampling_params(),
            arch_args={
                "dim": 4096,
                "n_layers": 32,
                "n_heads": 32,
                "n_kv_heads": 8,
                "vocab_size": LLAMA2_VOCAB_SIZE,
                "ffn_dim_multiplier": 1.3,
                "multiple_of": 256,
                "norm_eps": 1e-05,
                "rope_theta": 500000.0,
                "use_scaled_rope": False,
            },
            pth_file_count=1,
        ),

Is this correct? I thought that the original version used a setting of None for n_kv_heads (which eventually meant n_kv_heads was set to the value of n_heads which was 32). Similar question for llama2_13b.

[dnatarajan00]

Any help with this would be much appreciated!