Skip to content

Commit

Permalink
Dependencies: Change handling of exllamav2 checks
Browse files Browse the repository at this point in the history
ExllamaV2 should check for solely exllamav2, otherwise errors don't
make sense. Migrate the combined "exl2" computed property to "inference"
since those are the required dependencies for minimal inference.

Signed-off-by: kingbri <bdashore3@proton.me>
  • Loading branch information
bdashore3 committed Sep 22, 2024
1 parent 5380b3f commit e0ffa90
Show file tree
Hide file tree
Showing 3 changed files with 3 additions and 3 deletions.
2 changes: 1 addition & 1 deletion backends/exllamav2/version.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ def check_exllama_version():
"pip install --upgrade .[amd]\n\n"
)

if not dependencies.exl2:
if not dependencies.exllamav2:
raise SystemExit(("Exllamav2 is not installed.\n" + install_message))

required_version = version.parse("0.2.2")
Expand Down
2 changes: 1 addition & 1 deletion common/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
from common.tabby_config import config
from common.optional_dependencies import dependencies

if dependencies.exl2:
if dependencies.exllamav2:
from backends.exllamav2.model import ExllamaV2Container

# Global model container
Expand Down
2 changes: 1 addition & 1 deletion common/optional_dependencies.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ def extras(self) -> bool:

@computed_field
@property
def exl2(self) -> bool:
def inference(self) -> bool:
return self.torch and self.exllamav2 and self.flash_attn


Expand Down

0 comments on commit e0ffa90

Please sign in to comment.