-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for all Granite Guardian risks #1576
base: main
Are you sure you want to change the base?
Conversation
I don't agree on calling 'question ' to the default |
The RAG Granite Guardian metrics are duplicated now, but not exactly. The current Granite Guardian RAG metric is defined differently in the catalog. @elronbandel let me know if this is the way I should define the metrics in the prepare file and how should I adapt the example. |
I also changed the Granite-based LLM evaluators. I removed the '0' from the version as it looked awkward. Let me know if it is to late to make that change and I will revert it. |
@arielge FYI, we are adding all the missing Granite Guardian risks. |
Signed-off-by: Martín Santillán Cooper <msantillancooper@ibm.com>
Signed-off-by: Martín Santillán Cooper <msantillancooper@ibm.com>
Signed-off-by: Martín Santillán Cooper <msantillancooper@ibm.com>
Signed-off-by: Martín Santillán Cooper <msantillancooper@ibm.com>
Signed-off-by: Martín Santillán Cooper <msantillancooper@ibm.com>
Signed-off-by: Martín Santillán Cooper <msantillancooper@ibm.com>
Signed-off-by: Martín Santillán Cooper <msantillancooper@ibm.com>
Signed-off-by: Martín Santillán Cooper <msantillancooper@ibm.com>
552fb4a
to
84e15e9
Compare
@@ -80,12 +80,10 @@ class EvaluatorNameEnum(str, Enum): | |||
O1_PREVIEW = "o1-Preview" | |||
O1_MINI = "o1-Mini" | |||
GRANITE_13B = "Granite-13b" | |||
GRANITE3_2B = "Granite3.0-2b" | |||
GRANITE3_8B = "Granite3.0-8b" | |||
GRANITE3_2B = "Granite3-2b" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@martinscooper is this breaking change or just internal renaming?
Signed-off-by: Martín Santillán Cooper <msantillancooper@ibm.com>
d95a593
to
789e9b0
Compare
This PR adds support for all available Granite Guardian risks, extending current support of RAG risks, plus bug fixes and several improvements. In summary, this PR: