doc: graph: add document for sdpa with compressed key and value

[wzt1997]

General

1. Add document for SDPA with compressed Key and Value

[mgouicem]

(open question) Do we want to have separate document for non-quantized and quantized patterns?
My understanding of existing SDPA page was that it would regroup different data-types given that it currently has a subsection for floating-point pattern.

[TaoLv]

(open question) Do we want to have separate document for non-quantized and quantized patterns?

Thank you @mgouicem ! That's also my question. Initially when i added the floating-point section, i was thinking to include all these fp, int8 quantized, and only kv quantized patterns together in a single page. But with this PR, it seems there will be too much information. Maybe we need @ranukund 's input for which is a better format.

[wzt1997]

Do we want to have separate document for non-quantized and quantized patterns?

Thanks for the question! From my perspective, I think it's better for us to put the quantized sdpa patterns in a separate document as it requires much more information compared with pure floating-point patterns, such as fpmath mode setting, group quantization and adding extra dynamic quantization ops.

It's also worth noting that this PR only includes the document for quantization sdpa patterns with compressed KV, but not the pure quantized sdpa. We may need to think about fusing them together in the future.

[TaoLv]

Document folder has been changed on main branch. Please rebase the PR. Thanks.

[TaoLv]

what's the problem if we directly expand dt_fp as f32, bf16, f16 in the table above?

[wzt1997]

If we directly use f32, bf16, f16 in the table, it would result in an excessively lengthy table. Furthermore, it could lead users to misinterpret that these data types can be mixed, such as using f16 for Query with bf16 for Key scale. However, all sections involving dt_fp should utilize the same data type.