flip mx inference scaling setting to RCEIL

[vkuzo]

Summary:

Overall we know RCEIL is better from industry knowledge, the benchmarks
below are very light just to validate we can measure the increase. Switching to RCEIL for inference.

Flipping the overall default is left as a TODO because we need to update our dim1 triton and c++ kernels accordingly, and we can do that after the next branch cut.

Accuracy

• before

wikitext: {'alias': 'wikitext', 'word_perplexity,none':
7.609070006132819, 'word_perplexity_stderr,none': 'N/A',
'byte_perplexity,none': 1.4615491037668933,
'byte_perplexity_stderr,none': 'N/A', 'bits_per_byte,none':
0.5474983002838458, 'bits_per_byte_stderr,none': 'N/A'}
winogrande: {'alias': 'winogrande', 'acc,none': 0.7292817679558011,
'acc_stderr,none': 0.012487904760626407}

• after

wikitext: {'alias': 'wikitext', 'word_perplexity,none':
7.605192917647689, 'word_perplexity_stderr,none': 'N/A',
'byte_perplexity,none': 1.4614098103053235,
'byte_perplexity_stderr,none': 'N/A', 'bits_per_byte,none':
0.547360797163005, 'bits_per_byte_stderr,none': 'N/A'}
winogrande: {'alias': 'winogrande', 'acc,none': 0.7355958958168903,
'acc_stderr,none': 0.012394724896983764}

nice lift in perplexity and winogrande accuracy score

Performance on norm -> linear benchmarks

• before: https://gist.github.com/vkuzo/e4eab53fc9a23c007585c2235a7c7088
• after: https://gist.github.com/vkuzo/4ac7cde8a3ec1cd8f4d66847df091f7e

a slight performance regression, but we have not optimized RCEIL
performance at all and we aren't using the intrinsics yet, so room to
optimize

Test Plan:

pytest test/prototype/mx_formats/ -s -x

Reviewers:

Subscribers:

Tasks:

Tags:

[vkuzo]

Stack from ghstack (oldest at bottom):

• -> flip mx inference scaling setting to RCEIL #3428

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/3428

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.