Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

made changes to docs for prompt governance #457

Merged
merged 3 commits into from
Aug 1, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion docs/gh_pages/docs/retrieval_chain.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ The walkthrough requires Langchain, langchain-community, langchain-openai, and a
%pip install --upgrade --quiet langchain langchain-community langchain-openai qdrant_client
```

### Identity-aware Data Ingestion
## Identity-aware Data Ingestion
Raj725 marked this conversation as resolved.
Show resolved Hide resolved

In this scenario, Qdrant is being utilized as a vector database. However, the flexibility of the system allows for the use of any supported vector
databases.
Expand Down Expand Up @@ -203,3 +203,8 @@ Question: Please share the performance summary for John Smith?
Answer:
I don't know, I'm sorry.
```

## Prompt Governance
When a user sends any prompt to the LLM using PebbloRetrieveQA, we capture the findings from that prompt. This behavior is enabled by default.
Raj725 marked this conversation as resolved.
Show resolved Hide resolved

***Coming Soon***: Ability to block or anonymize prompts based on policy.
2 changes: 1 addition & 1 deletion docs/gh_pages/docs/safe_loader.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ This page consist of 4 primary tabs that provides the following details:
2. **Findings**:
The figure denotes the cumulative count of Topics and Entities identified across all applications. It will also furnish you with a comprehensive list of these Topics and Entities, along with supplementary information including the count of source files they originate from, the Datasource, and the name of the Application.

3. **Files with Findings**:
3. **Documents with Findings**:
gr8nishan marked this conversation as resolved.
Show resolved Hide resolved
The number of files that has one or more Findings over the total number of files used in document load across all the applications. This field indicates the number of files that need to be inspected to remediate any potentially text that needs to be removed and/or cleaned for Gen-AI inference.

It will also provide you with a list of these files, accompanied by additional details such as the file size, the owner's name, the count of topics & entities within each file, and the name of the Datasource.
Expand Down
34 changes: 26 additions & 8 deletions docs/gh_pages/docs/safe_retriever.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@

This section provides details about the retrievals for all the retrieval based application.

![Pebblo UI](../static/img/pebblo-saferetriever-ui.jpeg)
![Pebblo UI](../static/img/pebblo-saferetriever-ui.png)

## Overview Page

This page consist of 4 primary tabs that provides the following details:
This page consist of 5 primary tabs that provides the following details:

1. **Applications**:
The number signifies total number of retrieval type applications.
Expand All @@ -16,10 +16,13 @@ This page consist of 4 primary tabs that provides the following details:
2. **Retrievals**:
The figure denotes the cumulative count of retrievals happened for all the apps.

3. **Active Users**:
3. **Prompts with Findings**:
The number of prompts that has one or more findings over the total number of prompts used during retrieval process. This field indicates the number of prompts that need to be inspected for Gen-AI inference.

4. **Active Users**:
The number of active users for all the retrieval type applications.

4. **Violations**:
5. **Violations**:
Coming Soon!

## Application Details Page
Expand All @@ -29,20 +32,35 @@ You will be directed to the application details page by clicking on any applicat
**Instance Details**:
This section provide a quick glance of where the RAG application is physically running like in a Laptop (Mac OSX) or Linux VM and related properties like IP address, local filesystem path and Python version.

This page consist of 4 primary tabs that provides the following details:
This page consist of 5 primary tabs that provides the following details:

1. **Retrievals**: The figure denotes the cumulative count of retrievals happened for this app.
This section provides details about all retrievals with information like

**Prompt** : What was the prompt sent by the end user, Who sent the prompt and when it was sent.

**Findings**: One or more key findings extracted from the prompt.

**Context** : What was the context and which vector db was used.

**Response**: This is the final response generate by LLM for the given prompt.

**Retrieved From** : Source file of this context.

2. **Active Users**: The number of active users for this application will be provided, accompanied by a list of these users. This list will include additional details such as the retrieval count for each user and the last time they accessed the application.
2. **Prompts with Findings**: The number of prompts that has one or more findings over the total number of prompts for the given application.

3. **Active Users**: The number of active users for this application will be provided, accompanied by a list of these users. This list will include additional details such as the retrieval count for each user and the last time they accessed the application.

3. **Documents**: The number of documents accessed for retrievals will be provided, accompanied by a list of these documents. This list will include supplementary details such as the owner's name, retrieval count for each document, and the most recent access time for information from each document.
4. **Documents**: The number of documents accessed for retrievals will be provided, accompanied by a list of these documents. This list will include supplementary details such as the owner's name, retrieval count for each document, and the most recent access time for information from each document.

4. **Vector Databases**: The number of vector databases used in retrievals for this application.
5. **Vector Databases**: The number of vector databases used in retrievals for this application.
Details within this tab are coming soon.

## Prompt With Findings Details Page

This page shows all the prompt with findings for all the applications.

1. **Entity Name**: The name of the entity which is present in the prompts.
2. **Prompts**: Number of prompts in which the entity was detected.
3. **Users**: Users who have used this entity in the prompt.
4. **Apps** : Application in which the entity is present.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.