[Tracker] WIP features for torchao 0.5

[supriyar]

Branch cut: Sept 5th
Release: Sept 10th

Low-precision inference

• Make IntX dtypes work with AffineQuantizedTensor @jerryzh168 (Use torch.uint1 to torch.uint7 for Uintx tensor subclass #672)
• Performance benchmarks for intX dtypes on relevant shapes, identify potential improvements to torch.compile @HDCharles @jerryzh168
• Make quant_llm (quant/dequant/pack/unpack) available through AffineQuantizedTensor @jerryzh168 ([reland] Refactor quant_llm to work with affine quantized tensor (#696) #772)
• QAT Tensor subclass based implementation Refactor QAT to use tensor subclasses #585 @andrewor14
• TBD new feature enablement on quantized training/QAT/quantized fine-tuning @andrewor14 @gau-nernst
• KV Cache quantization for llama3.1 w/ long context @HDCharles
• Float8 inference flow @jainapurva @drisspg
• Float8 gemm benchmarks on relevant shapes @vkuzo @jainapurva

Low-precision training

• Float8 training recipes @vkuzo
• Make AQT trainable, i.e. compose with DTensor and FSDP2 @andrewor14 @jerryzh168

Sparsity

• Hopper + FP8 support in core @jcaip
• 2:4 sparsity + BSR benchmarks for ViT model @jcaip
• int8 BSR support

Benchmarking

• setup torchao benchmarking in torchbench @HDCharles

Blog

• README revamp @msaroufim
• torchao hard launch blogpost @msaroufim
• Low-bit quanti w/ HQQ @HDCharles @mobicham