Skip to content

Conversation

@larryliu0820
Copy link
Contributor

Add a CUDA kernel for argmax operation to support GPU-based sampling:

  • argmax.cuh: Template kernel using warp-level reductions with __shfl_xor_sync
    for efficient parallel max finding. Supports float, half, and bfloat16.
  • argmax.cu: Wrapper function argmax_cuda() that launches the kernel,
    handles device-to-host copy, and synchronization.
  • test_argmax.cu: Comprehensive unit tests covering various vocab sizes,
    data types, edge cases, and numerical precision.
  • CMakeLists.txt: Build configuration for extension_llm_sampler_cuda library
    and GoogleTest-based unit tests.

[ghstack-poisoned]
@larryliu0820
Copy link
Contributor Author

larryliu0820 commented Dec 24, 2025

@pytorch-bot
Copy link

pytorch-bot bot commented Dec 24, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/16386

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 2 Unrelated Failures

As of commit f8cd4d2 with merge base c5d66a5 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Dec 24, 2025
@larryliu0820 larryliu0820 added the release notes: desktop for desktop/laptop workstream label Dec 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. release notes: desktop for desktop/laptop workstream

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants