Skip to content

Conversation

Pouyanpi
Copy link
Collaborator

This PR implements tool call extraction and passthrough functionality in LLMRails, enabling the framework to capture and forward tool calls from LLM responses. Key changes include:

  • Add tool_calls_var context variable for storing LLM tool calls
  • Refactor llm_call utils to extract and store tool calls from responses
  • Support tool calls in both GenerationResponse and dict message formats
  • Add ToolMessage support for langchain message conversion
  • Comprehensive test coverage for tool calling integration scenarios

The implementation maintains backward compatibility while adding new tool calling capabilities to the passthrough mode.

@codecov-commenter
Copy link

codecov-commenter commented Aug 29, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.73%. Comparing base (eb29437) to head (8902ddb).

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1364      +/-   ##
===========================================
+ Coverage    71.66%   71.73%   +0.07%     
===========================================
  Files          171      171              
  Lines        17015    17045      +30     
===========================================
+ Hits         12193    12227      +34     
+ Misses        4822     4818       -4     
Flag Coverage Δ
python 71.73% <100.00%> (+0.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
nemoguardrails/actions/llm/utils.py 82.31% <100.00%> (+2.31%) ⬆️
nemoguardrails/context.py 100.00% <100.00%> (ø)
nemoguardrails/rails/llm/llmrails.py 90.52% <100.00%> (+0.06%) ⬆️
nemoguardrails/rails/llm/options.py 63.39% <100.00%> (+0.32%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Pouyanpi Pouyanpi added this to the v0.17.0 milestone Sep 2, 2025
@Pouyanpi Pouyanpi self-assigned this Sep 2, 2025
@Pouyanpi Pouyanpi added the enhancement New feature or request label Sep 2, 2025
@Pouyanpi Pouyanpi force-pushed the feat/tool-calling branch 3 times, most recently from 0c2a65e to d809788 Compare September 15, 2025 09:34
Copy link
Contributor

Documentation preview

https://nvidia.github.io/NeMo-Guardrails/review/pr-1364

@Pouyanpi Pouyanpi requested a review from tgasser-nv September 15, 2025 09:48
Copy link
Collaborator

@tgasser-nv tgasser-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, left some comments to address before merging. This is a huge PR (4k LOC) could you split this type of change into multiple smaller PRs in future?

Also could you add some local integration testing to check this works when interfacing with real Nemoguard / LLMs which support tool-calling?

@Pouyanpi
Copy link
Collaborator Author

Pouyanpi commented Sep 22, 2025

Looks good, left some comments to address before merging. This is a huge PR (4k LOC) could you split this type of change into multiple smaller PRs in future?

Also could you add some local integration testing to check this works when interfacing with real Nemoguard / LLMs which support tool-calling?

You are absolutely right, sorry for that. I can see that commits of another PR has gotten here probably during rebase.

image

Now it is around 890

Implements tool call extraction and passthrough functionality in LLMRails:
- Add tool_calls_var context variable for storing LLM tool calls
- Refactor llm_call utils to extract and store tool calls from responses
- Support tool calls in both GenerationResponse and dict message formats
- Add ToolMessage support for langchain message conversion
- Comprehensive test coverage for tool calling integration

feat(tool-calling): add tool call passthrough support in LLMRails

Implements tool call extraction and passthrough functionality in LLMRails:
- Add tool_calls_var context variable for storing LLM tool calls
- Refactor llm_call utils to extract and store tool calls from responses
- Support tool calls in both GenerationResponse and dict message formats
- Add ToolMessage support for langchain message conversion
- Comprehensive test coverage for tool calling integration
@codecov-commenter
Copy link

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

ℹ️ You can also turn on project coverage checks and project coverage reporting on Pull Request comment

Thanks for integrating Codecov - We've got you covered ☂️

@Pouyanpi Pouyanpi merged commit 51b1308 into develop Sep 22, 2025
19 of 21 checks passed
@Pouyanpi Pouyanpi deleted the feat/tool-calling branch September 22, 2025 08:19
tgasser-nv pushed a commit that referenced this pull request Sep 22, 2025
)

Implements tool call extraction and passthrough functionality in LLMRails:
- Add tool_calls_var context variable for storing LLM tool calls
- Refactor llm_call utils to extract and store tool calls from responses
- Support tool calls in both GenerationResponse and dict message formats
- Add ToolMessage support for langchain message conversion
- Comprehensive test coverage for tool calling integration
tgasser-nv pushed a commit that referenced this pull request Sep 22, 2025
)

Implements tool call extraction and passthrough functionality in LLMRails:
- Add tool_calls_var context variable for storing LLM tool calls
- Refactor llm_call utils to extract and store tool calls from responses
- Support tool calls in both GenerationResponse and dict message formats
- Add ToolMessage support for langchain message conversion
- Comprehensive test coverage for tool calling integration
Pouyanpi added a commit that referenced this pull request Oct 1, 2025
)

Implements tool call extraction and passthrough functionality in LLMRails:
- Add tool_calls_var context variable for storing LLM tool calls
- Refactor llm_call utils to extract and store tool calls from responses
- Support tool calls in both GenerationResponse and dict message formats
- Add ToolMessage support for langchain message conversion
- Comprehensive test coverage for tool calling integration
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants