Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

set up Guardrails API to check LLM inputs/outputs #70

Open
tatianab opened this issue Dec 18, 2024 · 5 comments
Open

set up Guardrails API to check LLM inputs/outputs #70

tatianab opened this issue Dec 18, 2024 · 5 comments
Assignees
@tatianab

Comments

@tatianab
Copy link
Contributor

Add a package that calls the Guardrails API (https://developers.google.com/checks/guide/ai-safety/guardrails), which will be used to check LLM inputs and outputs against policies.
@tatianab tatianab self-assigned this Dec 18, 2024
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/637977 mentions this issue: internal/llmapp: allow policy checker in overview functions

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/637976 mentions this issue: internal/gcp/checks: add implementation of llm.PolicyChecker

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/637978 mentions this issue: internal/gaby: turn on policy checks for overviews in Gaby

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/637975 mentions this issue: internal/llm: add interface PolicyChecker

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/637979 mentions this issue: internal/{llmapp,gaby}: display policy evaluation results in web UI

gopherbot pushed a commit that referenced this issue Dec 20, 2024
@tatianab
internal/llm: add interface PolicyChecker
df0382c 
A PolicyChecker is used to check the inputs and outputs
of an LLM against safety policies. We will implement this
interface with the GCP Checks Guardrails API.

For #70

Change-Id: I52a776cc94900cef4c0b56f284c56e44f5136d4b
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/637975
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
gopherbot pushed a commit that referenced this issue Dec 20, 2024
@tatianab
internal/gcp/checks: add implementation of llm.PolicyChecker
158f50b 
Use the GCP Checks Guardrails API to implement llm.PolicyChecker.
Don't use it anywhere yet.

For #70

Change-Id: I788eb138ed1f887e53ab154a8b569abe70cadf34
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/637976
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
gopherbot pushed a commit that referenced this issue Dec 20, 2024
@tatianab
internal/llmapp: allow policy checker in overview functions
a315118 
Add an optional policy checker to the overviews client. When a
policy checker is configured, all LLM inputs and outputs will be checked
for safety against the configured policy.

Not yet used by Gaby or anywhere else.

For #70

Change-Id: I8d48048eae9651499ec937a8804ab554baca2316
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/637977
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
gopherbot pushed a commit that referenced this issue Dec 20, 2024
@tatianab
internal/gaby: allow policy checks via flag for overviews in Gaby
5d12b58 
If the -enforcepolicy flag is set (default false), check all safety
categories when generating overviews. Since overviews are not yet
published anywhere, this would only affect users of the web UI.

For #70

Change-Id: I100070b5726ca0ff21cea4dec9f7f68e74018f08
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/637978
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
gopherbot pushed a commit that referenced this issue Dec 20, 2024
@tatianab
internal/{llmapp,gaby}: display policy evaluation results in web UI
b08abec 
When policies are enforced (-enforcepolicy), display policy evaluation
results on the Overview web UI page.

To support this, return the detailed results from the llmapp.Overview
functions.

For #70

Change-Id: I5642236f48d205be25f7359b540a72ff86903e4f
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/637979
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tatianab tatianab

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tatianab @gopherbot