-
Notifications
You must be signed in to change notification settings - Fork 563
fix: Propagate model and base URL in LLMCallException; improve error handling #1502
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Conversation
| # FIXME: is text and chat a good idea? | ||
| # For text mode, use text completion, we are using both text and chat as the last resort | ||
| ModelInitializer(_init_text_completion_model, ["text", "chat"]), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The NeMo Guardrails MS uses a custom chat client for main models. We often face this generic error:
nemoguardrails.llm.models.langchain_initializer.ModelInitializationError: Failed to initialize model 'meta/llama-3.3-70b-instruct' with provider 'nimchat' in 'chat' mode: Could not find LLM provider 'nimchat'
However, the relevant exception was thrown by our custom class, but gets swallowed here by the exception thrown by _init_text_completion_model.
I repositioned _init_text_completion_model to come before ModelInitializer(_init_community_chat_models, ["chat"]), to ensure the relevant exception is bubbled up. Let me know if there are any issues with this.
Greptile Summary
Confidence Score: 3/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant User
participant LLMRails
participant ActionDispatcher
participant LLMUtils as "llm/utils"
participant LangChainModel as "LangChain Model"
participant ExceptionHandler as "Exception Handler"
User->>LLMRails: "generate_async(prompt)"
LLMRails->>ActionDispatcher: "execute action"
ActionDispatcher->>LLMUtils: "llm_call(llm, prompt)"
LLMUtils->>LangChainModel: "ainvoke(prompt)"
alt Success
LangChainModel-->>LLMUtils: "response"
LLMUtils-->>ActionDispatcher: "response"
else Failure
LangChainModel-->>LLMUtils: "Exception"
LLMUtils->>ExceptionHandler: "_raise_llm_call_exception(e, llm)"
ExceptionHandler->>ExceptionHandler: "extract model_name from llm"
ExceptionHandler->>ExceptionHandler: "extract endpoint_url from llm attributes"
ExceptionHandler->>ExceptionHandler: "build context message"
ExceptionHandler-->>LLMUtils: "raise LLMCallException(e, context)"
LLMUtils-->>ActionDispatcher: "LLMCallException"
ActionDispatcher-->>User: "Error with model and endpoint context"
end
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Additional Comments (2)
8 files reviewed, 2 comments
Edit Code Review Agent Settings | Greptile
React with 👍 or 👎 to share your feedback on this new summary format
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
10 files reviewed, 1 comment
Edit Code Review Agent Settings | Greptile
React with 👍 or 👎 to share your feedback on this new summary format
| # FIXME: is text and chat a good idea? | ||
| # For text mode, use text completion, we are using both text and chat as the last resort | ||
| ModelInitializer(_init_text_completion_model, ["text", "chat"]), | ||
| # For chat mode, first try the standard chat completion API | ||
| ModelInitializer(_init_chat_completion_model, ["chat"]), | ||
| # For chat mode, fall back to community chat models | ||
| ModelInitializer(_init_community_chat_models, ["chat"]), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
style: reordering changes fallback behavior - text completion now runs before chat/community models. FIXME comment suggests uncertainty. verify this doesn't break existing configurations expecting chat completion first
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
12 files reviewed, 1 comment
Edit Code Review Agent Settings | Greptile
React with 👍 or 👎 to share your feedback on this new summary format
| # FIXME: is text and chat a good idea? | ||
| # For text mode, use text completion, we are using both text and chat as the last resort | ||
| ModelInitializer(_init_text_completion_model, ["text", "chat"]), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
style: text completion now runs before chat models - verify this doesn't break existing chat configurations that expect chat completion to be tried first. the FIXME suggests uncertainty about this ordering.
Description
Related Issue(s)
Checklist