SDPA: scaled_dot_product_attention to use this attetino donot need to retain ?

[ganliqiang]

when use new sdpa,why does it dont need to be trained,now paras is added ?

[supriyar]

cc @drisspg @cpuhrsch

[drisspg]

SDPA is a purely functional op so in if you pass in the existing parameters from the trained model you should not have to retrain.

[HDCharles]

I'm going to close the issue since its not related to the torchao repository feel free to continue the discussion in the pytorch forums or in an issue in the pytorch repo