Capabilities overhaul #76
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
madamczykhabana
commented
Jan 16, 2025
madamczykhabana
commented
- 'Capabilities' are now called 'Flags' as they can include things that are selectable by the user on startup (like env flags)
- Flags can be combined with 'And' and 'Not'
- Three new flag tests added: Kernel (tests whether a custom kernel impl is loadable), EnvFlag (checks env variables) and Model (checks what model type is loaded)
- Two new supported flags: "fp32_softmax" and "fsdpa"
- Moved fsdpa and rms_norm to a new file: kernels.py
- Moved logger to utils.py
Capabilties are now known as flags as they include togglable entries that might depend on env vars and other things.
madamczykhabana
requested review from
kdamaszk,
kzawora-intel,
michalkuligowski and
afierka-intel
kdamaszk
reviewed
Jan 20, 2025
madamczykhabana
added a commit
to HabanaAI/vllm-fork
that referenced
this pull request
Jan 20, 2025
Supporting PR for HabanaAI/vllm-hpu-extension#76
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.