Commit a9efa91
committed
flip mx scaling enum default to RCEIL
Summary:
Overall we know RCEIL is better from industry knowledge, the benchmarks
below are very light just to validate we can measure the increase.
Accuracy
* before
```
wikitext: {'alias': 'wikitext', 'word_perplexity,none':
7.609070006132819, 'word_perplexity_stderr,none': 'N/A',
'byte_perplexity,none': 1.4615491037668933,
'byte_perplexity_stderr,none': 'N/A', 'bits_per_byte,none':
0.5474983002838458, 'bits_per_byte_stderr,none': 'N/A'}
winogrande: {'alias': 'winogrande', 'acc,none': 0.7292817679558011,
'acc_stderr,none': 0.012487904760626407}
```
* after
```
wikitext: {'alias': 'wikitext', 'word_perplexity,none':
7.605192917647689, 'word_perplexity_stderr,none': 'N/A',
'byte_perplexity,none': 1.4614098103053235,
'byte_perplexity_stderr,none': 'N/A', 'bits_per_byte,none':
0.547360797163005, 'bits_per_byte_stderr,none': 'N/A'}
winogrande: {'alias': 'winogrande', 'acc,none': 0.7355958958168903,
'acc_stderr,none': 0.012394724896983764}
```
nice lift in perplexity and winogrande accuracy score
Performance on norm -> linear benchmarks
* before: https://gist.github.com/vkuzo/e4eab53fc9a23c007585c2235a7c7088
* after: https://gist.github.com/vkuzo/4ac7cde8a3ec1cd8f4d66847df091f7e
a slight performance regression, but we have not optimized RCEIL
performance at all and we aren't using the intrinsics yet, so room to
optimize
Test Plan:
```
pytest test/prototype/mx_formats/ -s -x
```
Reviewers:
Subscribers:
Tasks:
Tags:
ghstack-source-id: f47b33f
ghstack-comment-id: 3608933956
Pull-Request: #34281 parent 1e3558c commit a9efa91
2 files changed
+4
-4
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
230 | 230 | | |
231 | 231 | | |
232 | 232 | | |
233 | | - | |
| 233 | + | |
234 | 234 | | |
235 | 235 | | |
236 | 236 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
87 | 87 | | |
88 | 88 | | |
89 | 89 | | |
90 | | - | |
| 90 | + | |
91 | 91 | | |
92 | 92 | | |
93 | 93 | | |
| |||
144 | 144 | | |
145 | 145 | | |
146 | 146 | | |
147 | | - | |
| 147 | + | |
148 | 148 | | |
149 | 149 | | |
150 | 150 | | |
| |||
533 | 533 | | |
534 | 534 | | |
535 | 535 | | |
536 | | - | |
| 536 | + | |
537 | 537 | | |
538 | 538 | | |
539 | 539 | | |
| |||
0 commit comments