Skip to content

Conversation

@Rohan138
Copy link
Contributor

Motivation

The expected src input tensor shapes for shuffle_weight_a16w4 (e, n, k) and shuffle_scale_a16w4 (e * n, k) are internally inconsistent: https://github.com/ROCm/aiter/blob/main/aiter/ops/shuffle.py

This requires the caller in vllm to add an extra view function for the scales to reshape them to match the function calls: https://github.com/ROCm/vllm/pull/808/files#diff-c73528091f2176f6547e24074950962deec4311315fa5e99ef2ca7682680708c

In aiter/355_wip, the shuffle_mxfp4_weight and shuffle_mxfp4_scale functions had matching src input shapes: https://github.com/ROCm/aiter/blob/355_wip/aiter/ops/shuffle.py

Technical Details

Test Plan

Test Result

Submission Checklist

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants