Skip to content

Conversation

@KimiJL
Copy link
Contributor

@KimiJL KimiJL commented Nov 21, 2024

Description

API Changes: Alignment Configuration Name Change

  • pii_fast -> pii
  • confidential_detection -> confidential_info_detection
  • text_toxicity_extraction -> toxicity_detection
  • factcheck -> groundedness_checker

Added new config + flow for AutoAlign factchecker

A new fact check guardrail that takes the user message and bot message to generate a score from AutoAlign service, and depending on user set threshold from .co file (default to 0.5), a warning log will be produced with the score if below set threshold.
Previous factcheck/groundedness check has been fully renamed to groundedness check, and the same knowledge base factual scoring remains

Added multi_language parameter support

A new optional parameter that enables support for languages beyond English

Related Issue(s)

N/A

Checklist

  • I've read the CONTRIBUTING guidelines.
  • I've updated the documentation if applicable.
  • I've added tests if applicable.
  • @mentions of the person or team responsible for reviewing proposed changes.

@KimiJL KimiJL changed the title feat:autoalign guardrail updates feat:AutoAlign guardrail updates Nov 21, 2024
Copy link
Collaborator

@Pouyanpi Pouyanpi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @KimiJL, what do you think about adding a warning in the community section that there are backward incompatible changes in v0.12.0 version of the toolkit. As the flow names and config params have changed?

I'm going to take a closer look next week.

Please use

[PLACEHOLDER]{warning}
THE WARNING TEXT
[PLACEHOLDER]

change the [PLACEHOLDER] to ```

@KimiJL
Copy link
Contributor Author

KimiJL commented Dec 2, 2024

@Pouyanpi Thanks for the suggestion, it makes sense to me. I have added the warning under the updated guardrails, please let me know if that is the expected format.

@Pouyanpi Pouyanpi self-requested a review December 3, 2024 06:09
Copy link
Collaborator

@Pouyanpi Pouyanpi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks 👍🏻

@Pouyanpi Pouyanpi merged commit 131f861 into NVIDIA-NeMo:develop Dec 3, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants