Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docs for permission in 03 and add enable_trace in 04 #21

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -134,9 +134,11 @@
# MAGIC %md
# MAGIC ## Grant stakeholders access to the Review App
# MAGIC
# MAGIC Now, grant your stakeholders permissions to use the Review App. Your stakeholders do not Databricks accounts as long as you have [insert docs].
# MAGIC Now, grant your stakeholders permissions to use the Review App. Your stakeholders do not need to have Databricks accounts. From the documentation:
# MAGIC
# MAGIC `#TODO: add docs link`
# MAGIC > If reviewers don’t have access already, account admins can use account-level SCIM provisioning to sync users and groups automatically from your identity provider to your Databricks account. You can also manually register these users and groups as you set up identities in Databricks. This allows them to be included as eligible reviewers.
# MAGIC
# MAGIC [Refer to documentation here](https://docs.databricks.com/en/generative-ai/agent-evaluation/human-evaluation.html#set-up-permissions-to-the-review-app-workspace).

# COMMAND ----------

Expand All @@ -151,7 +153,7 @@
# MAGIC
# MAGIC If you lose this notebook's state and need to find the URL to your Review App, run this cell.
# MAGIC
# MAGIC Alternatively, you can construct the Review App URL as follows:
# MAGIC Alternatively, you can cnstruct the Review App URL as follows:
chengyineng38 marked this conversation as resolved.
Show resolved Hide resolved
# MAGIC
# MAGIC `https://<your-workspace-url>/ml/reviews/{UC_CATALOG}.{UC_SCHEMA}.{UC_MODEL_NAME}/{UC_MODEL_VERSION_NUMBER}/instructions`

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,20 @@
F.col("is_correct"),
))

# COMMAND ----------
# MAGIC %md
# MAGIC ## Ingest chats into the Review App to be reviewed

# COMMAND ----------

from databricks.agents import enable_trace_reviews

request_id_list = requests_with_feedback_df.select(F.col("request_id")).rdd.flatMap(lambda x: x).collect()
chengyineng38 marked this conversation as resolved.
Show resolved Hide resolved
enable_trace_reviews(
model_name=UC_MODEL_NAME,
request_ids=request_id_list
)

# COMMAND ----------

# MAGIC %md
Expand Down