Skip to content

[WebNN EP] Support local attention feature for GQA #26565

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

[WebNN EP] Support local attention feature for GQA #26565

wants to merge 2 commits into from

Conversation

@peishenyan
Copy link
Contributor

@peishenyan peishenyan commented Nov 13, 2025

Description

Support the local_window_size attribute in GroupQueryAttention Operator, which is designed for sliding window attention and may influence the attention mask pattern.

For local window size not equal to -1, new attention mask pattern will be created as follows for applying sliding window.

     condition_1 (old attn_mask) ---> CumSum (axis=3, exclusive=true, reversed=true)
          |                             |
          |                           Lesser <--- local_window_size
          |                             |
      LogicalAnd <----------------- condition_2
          |
    new attn_mask

Motivation and Context

@peishenyan
temp local window size
6af6346 
add log info

temp

Support local_window_size for WebNN GQA
@peishenyan
decompose large constant
e9ef8ae
@peishenyan
Copy link
Contributor Author

peishenyan commented Nov 17, 2025

PTAL, thanks. @Honry

Honry
Honry approved these changes Nov 17, 2025
View reviewed changes
Copy link
Contributor

@Honry Honry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, LGTM, pls. also mention the needs to use 'expand' for mask_shape_ones_shape_constant in your commit message.

@Honry
Copy link
Contributor

Honry commented Nov 17, 2025

  • @fdwr, please take another look, thanks!

fdwr
fdwr approved these changes Nov 20, 2025
View reviewed changes
Copy link
Contributor

@fdwr fdwr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@fdwr
Copy link
Contributor

fdwr commented Nov 20, 2025

/azp run ONNX Runtime Web CI Pipeline,Windows GPU CI Pipeline,Linux Android Emulator QNN CI Pipeline,Windows GPU WebGPU CI Pipeline,Windows OpenVINO CI Pipeline

@fdwr
Copy link
Contributor

fdwr commented Nov 20, 2025

/azp run Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux OpenVINO CI Pipeline,MacOS CI Pipeline,Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline

@fdwr
Copy link
Contributor

fdwr commented Nov 20, 2025

/azp run Windows GPU CUDA CI Pipeline,Windows GPU DML CI Pipeline,Windows GPU Doc Gen CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI

@fdwr
Copy link
Contributor

fdwr commented Nov 20, 2025

/azp run Windows GPU TensorRT CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed,Windows x64 QNN CI Pipeline,Big Models

@azure-pipelines
Copy link

azure-pipelines bot commented Nov 20, 2025

Azure Pipelines successfully started running 1 pipeline(s).

@fdwr
Copy link
Contributor

fdwr commented Nov 20, 2025

/azp run Test Linux CUDA x64 Release,Test Linux TensorRT x64 Release,web_Debug / build_onnxruntime_web,web_Release / build_onnxruntime_web

@azure-pipelines
Copy link

azure-pipelines bot commented Nov 20, 2025

Azure Pipelines successfully started running 1 pipeline(s).

@fdwr
Copy link
Contributor

fdwr commented Nov 20, 2025

/azp run Linux QNN CI Pipeline

@azure-pipelines
Copy link

azure-pipelines bot commented Nov 20, 2025

No pipelines are associated with this pull request.

1 similar comment
@azure-pipelines
Copy link

azure-pipelines bot commented Nov 20, 2025

No pipelines are associated with this pull request.

@azure-pipelines
Copy link

azure-pipelines bot commented Nov 20, 2025

Azure Pipelines successfully started running 2 pipeline(s).

@azure-pipelines
Copy link

azure-pipelines bot commented Nov 20, 2025

Azure Pipelines successfully started running 1 pipeline(s).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@fdwr fdwr fdwr approved these changes

+1 more reviewer

@Honry Honry Honry approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@peishenyan @Honry @fdwr