Skip to content

Conversation

tgasser-nv
Copy link
Collaborator

@tgasser-nv tgasser-nv commented Sep 2, 2025

Description

This top-level PR combines all the Sub-PRs at the module-level into a single PR that can be merged and rebased against.

Sub-PRs

  • actions : 146 errors, 0 warnings, 0 informations. PR1361
  • actions_server : 2 errors, 0 warnings, 0 informations: PR1379
  • cli : 38 errors, 0 warnings, 0 informations: PR1380
  • colang : 113 errors, 0 warnings, 0 informations: PR1381
  • embeddings : 31 errors, 0 warnings, 0 informations: PR1383
  • eval : 91 errors, 0 warnings, 0 informations (P1 as it's an offline evaluation tool)
  • evaluate : 52 errors, 4 warnings, 0 informations (P1 as it's an offline evaluation tool)
  • integrations : 21 errors, 0 warnings, 0 informations: PR1384
  • kb : 2 errors, 0 warnings, 0 informations: PR1385
  • library : 142 errors, 0 warnings, 0 informations: PR1389
  • llm : 49 errors, 1 warning, 0 informations: PR1394
  • logging : 28 errors, 0 warnings, 0 informations: PR1395
  • rails : 82 errors, 0 warnings, 0 informations: PR1396
  • server : 20 errors, 0 warnings, 0 informations PR1397
  • tracing : 5 errors, 1 warning, 0 informations: PR1388

Related Issue(s)

Checklist

  • I've read the CONTRIBUTING guidelines.
  • I've updated the documentation if applicable.
  • I've added tests if applicable.
  • @mentions of the person or team responsible for reviewing proposed changes.

@codecov-commenter
Copy link

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.62%. Comparing base (960e0b8) to head (71d00f0).

Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #1367   +/-   ##
========================================
  Coverage    71.62%   71.62%           
========================================
  Files          171      171           
  Lines        17020    17020           
========================================
  Hits         12191    12191           
  Misses        4829     4829           
Flag Coverage Δ
python 71.62% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
nemoguardrails/actions/llm/generation.py 85.96% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@tgasser-nv
Copy link
Collaborator Author

tgasser-nv commented Sep 9, 2025

Note to self before merging:

  • Need to replace # pyright: ignore with # type: ignore which is a PEP standard and generalizes to other type-checkers other than Pyright
  • Check for any if type(obj) == ClassName and replace with the more general if isinstance(obj, ClassName) which is inheritance-aware

tgasser-nv and others added 9 commits September 15, 2025 10:19
Signed-off-by: Mike McKiernan <mmckiernan@nvidia.com>
* Updated notebook 1 with VDR feedback

* Updated notebook 2 (Jaeger) with VDR comments

* Replace Llama 3.1 8B with Llama 4 scout as main model

* Correct type in cell 10 notebook 2
* chore(release): prepare for v0.16.0

------

Co-authored-by: tgasser-nv <tgasser-nv@users.noreply.github.com>
Co-authored-by: tgasser-nv <200644301+tgasser-nv@users.noreply.github.com>
Co-authored-by: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com>
Signed-off-by: Tim Gasser <200644301+tgasser-nv@users.noreply.github.com>
Pouyanpi and others added 16 commits September 22, 2025 13:14
…1.0 (#1365)

Add example configuration and documentation for using NVIDIA NeMoGuard NIMs,
including content moderation, topic control, and jailbreak
detection.
…1399)

* Checkin of demo script link

* Update getting-started.md

Signed-off-by: Tim Gasser <200644301+tgasser-nv@users.noreply.github.com>

* docs: edit 1399 (#1400)

* edit more

* more fixes

---------

Signed-off-by: Tim Gasser <200644301+tgasser-nv@users.noreply.github.com>
Co-authored-by: Miyoung Choi <miyoungc@nvidia.com>
Update verbose logging to safely handle cases where log records may not
have 'id' or 'task' attributes. Prevents potential AttributeError and
improves robustness of LLM and prompt log output formatting.
Remove branch restriction from pull_request trigger to allow codecov
coverage reporting on PRs targeting any branch, not just develop.
)

Implements tool call extraction and passthrough functionality in LLMRails:
- Add tool_calls_var context variable for storing LLM tool calls
- Refactor llm_call utils to extract and store tool calls from responses
- Support tool calls in both GenerationResponse and dict message formats
- Add ToolMessage support for langchain message conversion
- Comprehensive test coverage for tool calling integration
… Runnable protocol support (#1366)

- Implement comprehensive async/sync invoke, batch, and streaming
support
- Add robust input/output transformation for all LangChain formats
(ChatPromptValue, BaseMessage,
dict, string)
- Enhance chaining behavior with intelligent __or__ method handling
RunnableBinding and complex
chains
- Add concurrency controls, error handling, and configurable blocking
messages
- Implement proper tool calling support with tool call passthrough
- Add extensive test suite (14 test files, 2800+ lines) covering all
major functionality including
batching, streaming, composition, piping, and tool calling
- Reorganize and expand test structure for better maintainability

apply review suggestions
…Rails (#1369)

Ensure AIMessage responses from RunnableRails contain the same metadata
fields (response_metadata, usage_metadata, additional_kwargs, id) as
direct LLM calls, enabling consistent LangChain integration behavior.
Enhance streaming in RunnableRails to include generation metadata in
streamed chunks. Skips END_OF_STREAM markers and updates chunk
formatting to support metadata for AIMessageChunk outputs. This improves
compatibility with consumers expecting metadata in streaming responses.
Remove the push trigger from the PR Tests workflow to avoid duplicate
runs when a pull request is updated. The workflow will now only run on
pull_request events, ensuring a single execution per PR update.
…1382)

Introduce tool output/input rails configuration and Colang flows for
tool call validation and parameter security checks. Add support for
BotToolCall event emission in passthrough mode, enabling tool call
guardrails before execution.
…ion and processing (#1386)

- Add UserToolMessages event handling and tool input rails processing
- Fix message-to-event conversion to properly handle tool messages in
conversation history
- Preserve tool call context in passthrough mode by using full
conversation history
- Support tool_calls and tool message metadata in LangChain format
conversion
- Include comprehensive test suite for tool input rails functionality

test(runnable_rails): fix prompt format in passthrough mode

feat: support ToolMessage in message dicts

refactor: rename BotToolCall to BotToolCalls
…calling (#1405)

* fix: preserve message metadata in RunnableRails tool calling

- Extract message conversion logic to centralized message_utils module
  - Dynamically preserve all LangChain message fields (tool_calls,
additional_kwargs, etc.)
  - Fix tool calling metadata loss in passthrough mode
  - Add comprehensive unit tests for message conversions
…tegration (#1355)

---------

Co-authored-by: Trent Holmes <trent_holmes@trendmicro.com>
Co-authored-by: Karanjot Singh Saggu <karanjotsingh_saggu@trendmicro.com>
…me (#1401)

Signed-off-by: Curtis G. Northcutt <curtis.northcutt@gmail.com>
Copy link
Contributor

Documentation preview

https://nvidia-nemo.github.io/Guardrails/review/pr-1367

@codecov-commenter
Copy link

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants