-
Notifications
You must be signed in to change notification settings - Fork 559
feat: add Nemotron model support with message-based prompts #1199
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Documentation preview |
c3600f9 to
2117f0f
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #1199 +/- ##
===========================================
+ Coverage 68.43% 68.65% +0.21%
===========================================
Files 161 161
Lines 15943 15978 +35
===========================================
+ Hits 10910 10969 +59
+ Misses 5033 5009 -24
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds support for Nemotron models in NeMo Guardrails using message‐based prompt templates with a detailed thinking feature. Key changes include adding new YAML prompt definitions for Nemotron and DeepSeek, implementing system message conversion in LLMRails, and updating test suites and configuration files accordingly.
Reviewed Changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_system_message_conversion.py | Added tests to ensure system messages are converted correctly. |
| tests/test_nemotron_prompt_modes.py | Added tests verifying detailed thinking behavior for Nemotron prompts. |
| nemoguardrails/rails/llm/llmrails.py | Modified to convert system messages to SystemMessage events appropriately. |
| nemoguardrails/llm/prompts/nemotron_reasoning.yml | New prompt definitions with detailed thinking enabled in specific tasks. |
| nemoguardrails/llm/prompts/llama3.yml | Updated prompt definitions to include new model identifiers. |
| nemoguardrails/llm/prompts.py | Updated scoring logic in prompt selection for improved model matching. |
| examples/configs/nemotron/config.yml | New configuration file for Nemotron model support. |
| examples/configs/nemotron/README.md | Added documentation for using Nemotron message-based prompts. |
Comments suppressed due to low confidence (1)
nemoguardrails/llm/prompts.py:108
- [nitpick] The added 'break' after detecting a substring match may prevent subsequent models in the list from being evaluated. Consider reviewing the order of model checks or accumulating all candidate scores to ensure the best matching prompt is selected.
elif _model in model:
mikemckiernan
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm, thanks so much!
8303cb1 to
7ce964d
Compare
Pouyanpi
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Signed-off-by: Pouyan <13303554+Pouyanpi@users.noreply.github.com>
Signed-off-by: Pouyan <13303554+Pouyanpi@users.noreply.github.com>
Signed-off-by: Pouyan <13303554+Pouyanpi@users.noreply.github.com>
Signed-off-by: Pouyan <13303554+Pouyanpi@users.noreply.github.com>
Signed-off-by: Pouyan <13303554+Pouyanpi@users.noreply.github.com>
Description
This PR adds support for Nemotron models in NeMo Guardrails with a message-based prompt approach. It implements a system for controlling detailed thinking traces both for internal tasks and in final user responses, similar to how it works on build.nvidia.com.
Key Changes
nemotron.ymlwith message-based prompt templatesFeatures
Control Detailed Thinking via System Messages
As Nemotron is a hybrid reasoning model, users can toggle the "detailed thinking" feature for final responses:
Configuration Options
Control detailed thinking for internal tasks:
Documentation
Added README.md with examples and explanations of the prompt system and thinking control options.
Testing
Added tests to verify:
generate_bot_messagehave "detailed thinking on"generate_user_intentdon't have "detailed thinking on"