-
-
Notifications
You must be signed in to change notification settings - Fork 11.5k
Description
🚀 The feature, motivation and pitch
There should be an audit of all call sites of current_platform.has_device_capability() to replace them with feature-specific query APIs. has_device_capability is CUDA-centric. There is some support for ROCm but the version numbers have different semantics. Often you want to check device capability because you want to conditionally enable a new feature such as bfloat16 or fp8. If we replaced has_device_capability with a feature check, e.g., current_platform.supports_fp8(), then this would be self-documenting and allow other platforms to implement them. For example, in gpu_worker.py it's used to check for bfloat16 support. The code would improve if there was a current_platform.supports_bfloat16().
Alternatives
No response
Additional context
This request was inspired by the recent PR #14245 that added new APIs to current_platform for fp8 support.
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.