You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Current helm chart for TGI doesn't have much Gaudi specific parameters setting like:
OMPI_MCA_btl_vader_single_copy_mechanism: none
ENABLE_HPU_GRAPH: true
LIMIT_HPU_GRAPH: true
USE_FLASH_ATTENTION: true
FLASH_ATTENTION_RECOMPUTE: true
(https://github.com/opea-project/GenAIExamples/blob/main/VisualQnA/docker_compose/intel/hpu/gaudi/compose.yaml)
We need to evaluate the impact of the tei-gaudi parameters and have the optimized settings.
This could be part of a bigger scope of tuning/optimization.
The text was updated successfully, but these errors were encountered:
Current helm chart for TGI doesn't have much Gaudi specific parameters setting like:
OMPI_MCA_btl_vader_single_copy_mechanism: none
ENABLE_HPU_GRAPH: true
LIMIT_HPU_GRAPH: true
USE_FLASH_ATTENTION: true
FLASH_ATTENTION_RECOMPUTE: true
(https://github.com/opea-project/GenAIExamples/blob/main/VisualQnA/docker_compose/intel/hpu/gaudi/compose.yaml)
We need to evaluate the impact of the tei-gaudi parameters and have the optimized settings.
This could be part of a bigger scope of tuning/optimization.
The text was updated successfully, but these errors were encountered: