Skip to content

Add mx_fp4_kernel #1661

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Feb 12, 2025
Merged

Add mx_fp4_kernel #1661

merged 1 commit into from
Feb 12, 2025

Conversation

@drisspg
Copy link
Contributor

@drisspg drisspg commented Feb 4, 2025 •
edited
Loading

Stacked PRs:

Add mx_fp4_kernel

drisspg added a commit that referenced this pull request Feb 4, 2025
@drisspg
Add mx_fp4_kernel
8e05144 
stack-info: PR: #1661, branch: drisspg/stack/34
@drisspg drisspg mentioned this pull request Feb 4, 2025
Merged
@drisspg drisspg mentioned this pull request Feb 4, 2025
Merged
@pytorch-bot
Copy link

pytorch-bot bot commented Feb 4, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/1661

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 4 Pending

As of commit 50a8a2a with merge base 682ffd5 (image):

NEW FAILURE - The following job has failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 4, 2025
@drisspg drisspg changed the base branch from drisspg/stack/31 to main February 4, 2025 19:55
drisspg added a commit that referenced this pull request Feb 4, 2025
@drisspg
Add mx_fp4_kernel
c18dde0 
stack-info: PR: #1661, branch: drisspg/stack/34
@drisspg drisspg changed the base branch from main to drisspg/stack/31 February 4, 2025 19:57
@drisspg drisspg changed the base branch from drisspg/stack/31 to main February 4, 2025 20:00
drisspg added a commit that referenced this pull request Feb 4, 2025
@drisspg
Add mx_fp4_kernel
a4ddbce 
stack-info: PR: #1661, branch: drisspg/stack/34
@drisspg drisspg added mx topic: new feature Use this tag if this PR adds a new feature labels Feb 4, 2025
@drisspg drisspg changed the base branch from main to drisspg/stack/31 February 4, 2025 20:00
@drisspg drisspg changed the base branch from drisspg/stack/31 to main February 7, 2025 22:32
@drisspg drisspg changed the base branch from main to drisspg/stack/31 February 7, 2025 22:32
@drisspg drisspg changed the base branch from drisspg/stack/31 to main February 10, 2025 21:44
drisspg added a commit that referenced this pull request Feb 10, 2025
@drisspg
Add mx_fp4_kernel
3c6f769 
stack-info: PR: #1661, branch: drisspg/stack/34
@drisspg drisspg changed the base branch from main to drisspg/stack/31 February 10, 2025 21:44
@drisspg drisspg changed the base branch from drisspg/stack/31 to main February 10, 2025 22:27
drisspg added a commit that referenced this pull request Feb 10, 2025
@drisspg
Add mx_fp4_kernel
44c99ed 
stack-info: PR: #1661, branch: drisspg/stack/34
drisspg added a commit that referenced this pull request Feb 10, 2025
@drisspg
Add mx_fp4_kernel
90ee0ff 
stack-info: PR: #1661, branch: drisspg/stack/34
@drisspg drisspg mentioned this pull request Feb 10, 2025
Merged
@drisspg drisspg changed the base branch from main to drisspg/stack/31 February 10, 2025 22:27
@drisspg drisspg changed the base branch from drisspg/stack/31 to main February 10, 2025 22:36
drisspg added a commit that referenced this pull request Feb 10, 2025
@drisspg
Add mx_fp4_kernel
59ed96a 
stack-info: PR: #1661, branch: drisspg/stack/34
@drisspg drisspg changed the base branch from main to drisspg/stack/31 February 10, 2025 22:36
@drisspg drisspg changed the base branch from drisspg/stack/31 to main February 10, 2025 22:38
@drisspg drisspg changed the base branch from main to drisspg/stack/31 February 10, 2025 22:38
@drisspg drisspg changed the base branch from drisspg/stack/31 to main February 11, 2025 23:32
drisspg added a commit that referenced this pull request Feb 11, 2025
@drisspg
Add mx_fp4_kernel
c4c3650 
stack-info: PR: #1661, branch: drisspg/stack/34
@drisspg drisspg changed the base branch from main to drisspg/stack/31 February 11, 2025 23:32
@drisspg drisspg changed the base branch from drisspg/stack/31 to main February 11, 2025 23:36
drisspg added a commit that referenced this pull request Feb 11, 2025
@drisspg
Add mx_fp4_kernel
204d06c 
stack-info: PR: #1661, branch: drisspg/stack/34
@drisspg drisspg changed the base branch from main to drisspg/stack/31 February 11, 2025 23:37
@drisspg drisspg requested a review from vkuzo February 12, 2025 00:57
@drisspg drisspg changed the base branch from drisspg/stack/31 to main February 12, 2025 01:11
drisspg added a commit that referenced this pull request Feb 12, 2025
@drisspg
Add mx_fp4_kernel
42395c2 
stack-info: PR: #1661, branch: drisspg/stack/34
@drisspg drisspg changed the base branch from main to drisspg/stack/31 February 12, 2025 01:11
vkuzo
vkuzo reviewed Feb 12, 2025
View reviewed changes
torchao/ops.py


def mx_fp4_bf16(A: Tensor, B: Tensor, A_scale: Tensor, B_scale: Tensor):
"""Defines a matmul between two fp4 tensors w/ MX scales in E8MO and returns a bf16 tensor.
Copy link
Contributor

@vkuzo vkuzo Feb 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: clarify that it's fp4_e2m1? Maybe link to https://www.opencompute.org/documents/ocp-microscaling-formats-mx-v1-0-spec-final-pdf section 5.3.3?

@drisspg
Add mx_fp4_kernel
50a8a2a 
stack-info: PR: #1661, branch: drisspg/stack/34
@drisspg drisspg changed the base branch from drisspg/stack/31 to main February 12, 2025 18:32
@drisspg drisspg changed the base branch from main to drisspg/stack/31 February 12, 2025 18:33
@drisspg drisspg merged commit f013189 into drisspg/stack/31 Feb 12, 2025
49 of 50 checks passed
drisspg added a commit that referenced this pull request Feb 12, 2025
@drisspg
Add mx_fp8_bf16 kernel (#1637)
d3306b2 
* Add mx_fp8_bf16 kernel

stack-info: PR: #1637, branch: drisspg/stack/31

* Add mx_fp4_kernel (#1661)

stack-info: PR: #1661
@vkuzo vkuzo mentioned this pull request Feb 24, 2025
Closed
liangel-02 pushed a commit that referenced this pull request Aug 25, 2025
@drisspg @liangel-02
Add mx_fp8_bf16 kernel (#1637)
92bd1ba 
* Add mx_fp8_bf16 kernel

stack-info: PR: #1637, branch: drisspg/stack/31

* Add mx_fp4_kernel (#1661)

stack-info: PR: #1661
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vkuzo vkuzo vkuzo approved these changes

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. mx topic: new feature Use this tag if this PR adds a new feature

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@drisspg @vkuzo @facebook-github-bot