Skip to content

Conversation

Pouyanpi
Copy link
Collaborator

@Pouyanpi Pouyanpi commented Sep 2, 2025

Description

  • Complete rewrite of RunnableRails implementing full LangChain Runnable protocol
  • Added comprehensive async/sync support for invoke, batch, and streaming operations
  • Enhanced input/output handling for all LangChain message formats and chaining scenarios
  • Extensive test suite with 14 test files covering batching, streaming, tool calling, and composition

requires #1364 and #1343 and #1289

@Pouyanpi Pouyanpi added this to the v0.17.0 milestone Sep 2, 2025
@Pouyanpi Pouyanpi self-assigned this Sep 2, 2025
@Pouyanpi Pouyanpi added the enhancement New feature or request label Sep 2, 2025
@Pouyanpi Pouyanpi changed the title Feat/runnable rails feat(runnable_rails): complete rewrite of RunnableRails with full LangChain Runnable protocol support Sep 2, 2025
Copy link
Contributor

github-actions bot commented Sep 2, 2025

Documentation preview

https://nvidia.github.io/NeMo-Guardrails/review/pr-1366

@codecov-commenter
Copy link

codecov-commenter commented Sep 2, 2025

Codecov Report

❌ Patch coverage is 77.25000% with 91 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.65%. Comparing base (eb29437) to head (0c2a65e).

Files with missing lines Patch % Lines
...uardrails/integrations/langchain/runnable_rails.py 72.80% 90 Missing ⚠️
nemoguardrails/actions/llm/utils.py 98.18% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1366      +/-   ##
===========================================
- Coverage    71.66%   71.65%   -0.01%     
===========================================
  Files          171      171              
  Lines        17015    17313     +298     
===========================================
+ Hits         12193    12406     +213     
- Misses        4822     4907      +85     
Flag Coverage Δ
python 71.65% <77.25%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
nemoguardrails/context.py 100.00% <100.00%> (ø)
nemoguardrails/logging/verbose.py 90.90% <100.00%> (+0.39%) ⬆️
nemoguardrails/rails/llm/llmrails.py 90.52% <100.00%> (+0.06%) ⬆️
nemoguardrails/rails/llm/options.py 63.39% <100.00%> (+0.32%) ⬆️
nemoguardrails/actions/llm/utils.py 82.31% <98.18%> (+2.31%) ⬆️
...uardrails/integrations/langchain/runnable_rails.py 75.52% <72.80%> (-14.57%) ⬇️

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Pouyanpi Pouyanpi force-pushed the feat/runnable-rails branch from 34dba2b to 7ae2f3a Compare September 3, 2025 09:10
@Pouyanpi Pouyanpi force-pushed the feat/runnable-rails branch from 7ae2f3a to 68f438e Compare September 9, 2025 09:22
@Pouyanpi Pouyanpi marked this pull request as ready for review September 9, 2025 09:41
@Pouyanpi Pouyanpi requested a review from Copilot September 9, 2025 09:42
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR completes a full rewrite of RunnableRails with comprehensive LangChain Runnable protocol support, providing async/sync operations, streaming capabilities, tool calling functionality, and enhanced input/output handling.

  • Implemented complete LangChain Runnable protocol including invoke, batch, stream, and async variants
  • Added tool calling support with proper context variable management across the pipeline
  • Enhanced streaming functionality with proper chunk formatting and state management

Reviewed Changes

Copilot reviewed 26 out of 26 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
tests/utils.py Added streaming support to FakeLLM with _stream and _astream methods
tests/test_tool_calls_context.py New test file for tool calls context variable functionality
tests/test_tool_calling_utils.py New comprehensive tests for tool calling utility functions
tests/test_tool_calling_passthrough_integration.py Integration tests for tool calling in passthrough mode
tests/runnable_rails/*.py 14 new test files covering streaming, batching, composition, and tool calling
nemoguardrails/rails/llm/options.py Added tool_calls field to GenerationResponse model
nemoguardrails/rails/llm/llmrails.py Enhanced to extract and include tool calls in responses
nemoguardrails/logging/verbose.py Fixed potential AttributeError with missing record attributes
nemoguardrails/integrations/langchain/runnable_rails.py Complete rewrite implementing full Runnable protocol
nemoguardrails/context.py Added tool_calls_var context variable
nemoguardrails/actions/llm/utils.py Refactored llm_call with tool calling support and improved message handling
examples/configs/nemoguards/* New example configuration demonstrating NeMoGuard safety rails

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@Pouyanpi Pouyanpi force-pushed the feat/runnable-rails branch 3 times, most recently from feb4447 to 0c2a65e Compare September 15, 2025 09:32
@Pouyanpi Pouyanpi requested a review from tgasser-nv September 15, 2025 10:34
@Pouyanpi Pouyanpi changed the title feat(runnable_rails): complete rewrite of RunnableRails with full LangChain Runnable protocol support feat(runnable-rails): update RunnableRails with full LangChain Runnable protocol support Sep 16, 2025
Copy link
Collaborator

@tgasser-nv tgasser-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a few comments:

  • ~4k LOC is too big for a single PR. Is there any way to stack multiple 100 - 400 line PRs or include something in the description link them?
  • Can you run some tool calls locally to check all this works when calling production APIs?

… Runnable protocol support

- Implement comprehensive async/sync invoke, batch, and streaming
support
- Add robust input/output transformation for all LangChain formats
(ChatPromptValue, BaseMessage,
dict, string)
- Enhance chaining behavior with intelligent __or__ method handling
RunnableBinding and complex
chains
- Add concurrency controls, error handling, and configurable blocking
messages
- Implement proper tool calling support with tool call passthrough
- Add extensive test suite (14 test files, 2800+ lines) covering all
major functionality including
batching, streaming, composition, piping, and tool calling
- Reorganize and expand test structure for better maintainability

apply review suggestions
@codecov-commenter
Copy link

Codecov Report

❌ Patch coverage is 72.80967% with 90 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...uardrails/integrations/langchain/runnable_rails.py 72.80% 90 Missing ⚠️

📢 Thoughts on this report? Let us know!

@Pouyanpi Pouyanpi merged commit 93083d1 into develop Sep 22, 2025
18 of 20 checks passed
@Pouyanpi Pouyanpi deleted the feat/runnable-rails branch September 22, 2025 08:32
tgasser-nv pushed a commit that referenced this pull request Sep 22, 2025
… Runnable protocol support (#1366)

- Implement comprehensive async/sync invoke, batch, and streaming
support
- Add robust input/output transformation for all LangChain formats
(ChatPromptValue, BaseMessage,
dict, string)
- Enhance chaining behavior with intelligent __or__ method handling
RunnableBinding and complex
chains
- Add concurrency controls, error handling, and configurable blocking
messages
- Implement proper tool calling support with tool call passthrough
- Add extensive test suite (14 test files, 2800+ lines) covering all
major functionality including
batching, streaming, composition, piping, and tool calling
- Reorganize and expand test structure for better maintainability

apply review suggestions
tgasser-nv pushed a commit that referenced this pull request Sep 22, 2025
… Runnable protocol support (#1366)

- Implement comprehensive async/sync invoke, batch, and streaming
support
- Add robust input/output transformation for all LangChain formats
(ChatPromptValue, BaseMessage,
dict, string)
- Enhance chaining behavior with intelligent __or__ method handling
RunnableBinding and complex
chains
- Add concurrency controls, error handling, and configurable blocking
messages
- Implement proper tool calling support with tool call passthrough
- Add extensive test suite (14 test files, 2800+ lines) covering all
major functionality including
batching, streaming, composition, piping, and tool calling
- Reorganize and expand test structure for better maintainability

apply review suggestions
Pouyanpi added a commit that referenced this pull request Oct 1, 2025
… Runnable protocol support (#1366)

- Implement comprehensive async/sync invoke, batch, and streaming
support
- Add robust input/output transformation for all LangChain formats
(ChatPromptValue, BaseMessage,
dict, string)
- Enhance chaining behavior with intelligent __or__ method handling
RunnableBinding and complex
chains
- Add concurrency controls, error handling, and configurable blocking
messages
- Implement proper tool calling support with tool call passthrough
- Add extensive test suite (14 test files, 2800+ lines) covering all
major functionality including
batching, streaming, composition, piping, and tool calling
- Reorganize and expand test structure for better maintainability

apply review suggestions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request runnable
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants