Skip to content

What's the difference between ensemble and tensorrt_llm_bls? #400

Closed Answered by Saigut
Saigut asked this question in Q&A
Discussion options

You must be logged in to vote

Triton's ensemble feature supports many use cases where multiple models are composed into a pipeline (or more generally a DAG, directed acyclic graph). However, there are many other use cases that are not supported because as part of the model pipeline they require loops, conditionals (if-then-else), data-dependent control-flow and other custom logic to be intermixed with model execution. We call this combination of custom logic and model executions Business Logic Scripting (BLS).

From: https://github.com/triton-inference-server/python_backend/blob/main/README.md#business-logic-scripting

Replies: 1 comment

Comment options

You must be logged in to vote
0 replies
Answer selected by Saigut
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
1 participant