Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: clemlesne/private-gpt
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.6.0
Choose a base ref
...
head repository: clemlesne/private-gpt
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v0.7.0
Choose a head ref
1 contributor

Commits on Jul 27, 2023

  1. Feat: Ability for the chat to use tools and external data

    @clemlesne
    clemlesne committed Jul 27, 2023
    Copy the full SHA
    a6e3c22 View commit details

  2. Merge branch 'feat/langchain' into develop

    @clemlesne
    clemlesne committed Jul 27, 2023
    Copy the full SHA
    290a0ff View commit details

Commits on Jul 28, 2023

  1. Quality: Enhance readibility against OpenAI backend errors

    @clemlesne
    clemlesne committed Jul 28, 2023
    Copy the full SHA
    c9ff233 View commit details

  2. UI: Add animation on message loading

    @clemlesne
    clemlesne committed Jul 28, 2023
    Copy the full SHA
    7fa1b18 View commit details

  3. Fix: Usage store from OpenAI

    @clemlesne
    clemlesne committed Jul 28, 2023
    Copy the full SHA
    9f4b4bb View commit details

  4. Conf: Able to parse JSON from configuration env var backup

    @clemlesne
    clemlesne committed Jul 28, 2023
    Copy the full SHA
    906ac9f View commit details

Commits on Jul 29, 2023

  1. UI: Display input at bottom for small screens

    @clemlesne
    clemlesne committed Jul 29, 2023
    Copy the full SHA
    7888802 View commit details

  2. UI: Fix header margins

    @clemlesne
    clemlesne committed Jul 29, 2023
    Copy the full SHA
    65bb851 View commit details

  3. Fix: Login on iOS/iPadOS

    @clemlesne
    clemlesne committed Jul 29, 2023
    Copy the full SHA
    70e552c View commit details

  4. UI: Update header button and animate it on small screens

    @clemlesne
    clemlesne committed Jul 29, 2023
    Copy the full SHA
    8c1900e View commit details

  5. Dev: Make clearer model parsing errors

    @clemlesne
    clemlesne committed Jul 29, 2023
    Copy the full SHA
    3b1b2d6 View commit details

  6. UI: Remove un-needed margin in index screen

    @clemlesne
    clemlesne committed Jul 29, 2023
    Copy the full SHA
    ac6389e View commit details

  7. UI: Remove un-needed user icon in header

    @clemlesne
    clemlesne committed Jul 29, 2023
    Copy the full SHA
    297e2fd View commit details

  8. UI: Add background shadow for fixed bottom header section

    @clemlesne
    clemlesne committed Jul 29, 2023
    Copy the full SHA
    342aafe View commit details

  9. Security: Deps upgrade

    @clemlesne
    clemlesne committed Jul 29, 2023
    Copy the full SHA
    8dc471f View commit details

  10. Security: Upgrade Azure Application Insights SDK

    @clemlesne
    clemlesne committed Jul 29, 2023
    Copy the full SHA
    ec84474 View commit details

  11. Fix: Login for browsers blocking third party cookies 

    See: AzureAD/microsoft-authentication-library-for-js#3118 (comment)
    @clemlesne
    clemlesne committed Jul 29, 2023
    Copy the full SHA
    87be725 View commit details

Commits on Jul 31, 2023

  1. Perf: Add cache layer behind Store persistence

    @clemlesne
    clemlesne committed Jul 31, 2023
    Copy the full SHA
    bb12c39 View commit details

  2. Fix: Search results does not displays

    @clemlesne
    clemlesne committed Jul 31, 2023
    Copy the full SHA
    fe08645 View commit details

  3. Fix: Results ordering from API

    @clemlesne
    clemlesne committed Jul 31, 2023
    Copy the full SHA
    4b430a9 View commit details

  4. Fix: Custom prompts were missing from LangChain branch 

    See: 290a0ff
    @clemlesne
    clemlesne committed Jul 31, 2023
    Copy the full SHA
    16b9ffe View commit details

  5. Quality: Code quality

    @clemlesne
    clemlesne committed Jul 31, 2023
    Copy the full SHA
    48b3ad0 View commit details

  6. UI: Avoid main content to shrink on scroll required

    @clemlesne
    clemlesne committed Jul 31, 2023
    Copy the full SHA
    2ca8fc0 View commit details

  7. UI: Add a nice shadow under the input

    @clemlesne
    clemlesne committed Jul 31, 2023
    Copy the full SHA
    a5acb13 View commit details

  8. Doc: Update app screenshots

    @clemlesne
    clemlesne committed Jul 31, 2023
    Copy the full SHA
    e0bc482 View commit details
Showing with 2,076 additions and 1,373 deletions.
  1. +138 −70 README.md
  2. +15 −3 cicd/helm/Makefile
  3. +67 −27 cicd/helm/private-gpt/templates/conversation-api-config.yaml
  4. +1 −1 cicd/helm/private-gpt/templates/conversation-api-secret.yaml
  5. +1 −1 cicd/helm/private-gpt/templates/conversation-ui-config.yaml
  6. +39 −12 cicd/helm/private-gpt/values.yaml
  7. BIN docs/api.png
  8. BIN docs/doc.png
  9. BIN docs/main.png
  10. BIN docs/ui-conversation.png
  11. BIN docs/ui-index.png
  12. +3 −0 src/conversation-api/Makefile
  13. +15 −10 src/conversation-api/ai/contentsafety.py
  14. +449 −104 src/conversation-api/ai/openai.py
  15. +27 −0 src/conversation-api/devcontainer.json
  16. +209 −246 src/conversation-api/main.py
  17. +5 −5 src/conversation-api/models/conversation.py
  18. +13 −0 src/conversation-api/models/memory.py
  19. +19 −8 src/conversation-api/models/message.py
  20. +3 −3 src/conversation-api/models/prompt.py
  21. +5 −6 src/conversation-api/models/usage.py
  22. +7 −3 src/conversation-api/models/user.py
  23. +136 −0 src/conversation-api/persistence/cache.py
  24. +136 −42 src/conversation-api/persistence/cosmos.py
  25. +49 −0 src/conversation-api/persistence/icache.py
  26. +8 −3 src/conversation-api/persistence/isearch.py
  27. +24 −16 src/conversation-api/persistence/istore.py
  28. +7 −1 src/conversation-api/persistence/istream.py
  29. +42 −30 src/conversation-api/persistence/qdrant.py
  30. +80 −171 src/conversation-api/persistence/redis.py
  31. +8 −3 src/conversation-api/requirements.txt
  32. +77 −34 src/conversation-api/utils.py
  33. +196 −398 src/conversation-ui/package-lock.json
  34. +9 −9 src/conversation-ui/package.json
  35. +19 −9 src/conversation-ui/src/App.jsx
  36. +23 −9 src/conversation-ui/src/Conversation.jsx
  37. +3 −3 src/conversation-ui/src/Conversations.jsx
  38. +11 −11 src/conversation-ui/src/Header.jsx
  39. +78 −49 src/conversation-ui/src/Message.jsx
  40. +4 −3 src/conversation-ui/src/Search.jsx
  41. +8 −14 src/conversation-ui/src/Utils.jsx
  42. +1 −0 src/conversation-ui/src/app.scss
  43. +10 −5 src/conversation-ui/src/conversation.scss
  44. +61 −50 src/conversation-ui/src/header.scss
  45. +1 −0 src/conversation-ui/src/main.jsx
  46. +22 −0 src/conversation-ui/src/message.scss
  47. +47 −14 test/helm/values.yaml
208 changes: 138 additions & 70 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -10,6 +10,7 @@ Includes:
- Dark theme for better readability
- Dead simple interface
- Deployable on any Kubernetes cluster, with its Helm chart
- Every persistence layers (search, index, AI) is cached, for performance and low cost
- Manage users effortlessly with OpenID Connect
- Monitoring with Azure App Insights (logs, traces, user behaviors)
- More than 150 tones and personalities (accountant, advisor, debater, excel sheet, instructor, logistician, etc.) to better help employees in their specific daily tasks
@@ -20,7 +21,89 @@ Includes:
- Unlimited conversation history and number of users
- Usage tracking, for better understanding of your employees' usage

![Application screenshot](docs/main.png)
![Application screenshot](docs/ui-conversation.png)

## How it works

### High level

```mermaid
sequenceDiagram
autonumber
actor User
participant PWA
participant API
participant OpenAI
PWA ->> API: Ask for conversations
activate API
API ->> API: Get conversations from storage
API ->> PWA: Answer with conversations
deactivate API
User ->> PWA: Select a conversation
User ->> PWA: Insert a message
PWA ->> API: Send the message
activate API
API ->> OpenAI: Ask for a completion
activate OpenAI
OpenAI ->> API: Send completion
deactivate OpenAI
API ->> API: Save conversation and message
API ->> PWA: Answer with the message
deactivate API
User ->> PWA: See results
```

### Architecture

```mermaid
graph
user(["User"])
api["Conversation service\n(REST API)"]
ui["Conversation UI\n(PWA)"]
subgraph tools["Tools"]
subgraph "Business data"
form_recognizer["Form recognizer"]
cognitive_services["Cognitive services"]
storage_blob["Blob storage"]
mssql["SQL Server"]
end
subgraph "Public data"
tmdb["TMDB"]
news["News"]
listen_notes["Listen notes"]
bing["Bing"]
end
end
subgraph "Persistence"
cosmosdb[("Cosmos DB\n(disk)")]
qdrant[("Qdrant\n(disk)")]
redis[("Redis\n(memory)")]
end
subgraph "Azure OpenAI services"
oai_ada["ADA embedding"]
oai_gpt["GPT completions"]
safety["Content Safety"]
end
api -- Cache low-level AI results --> redis
api -- Generate completions --> oai_gpt
api -- Generate embeddings --> oai_ada
api -- Index messages --> qdrant
api -- Persist conversations --> cosmosdb
api -- Test moderation --> safety
api -- Orchestrate external capabilities --> tools
ui -- Use APIs --> api
user -- Use UI --> ui
cognitive_services -- Index data --> mssql
cognitive_services -- Index data --> storage_blob
```

## How to use

@@ -31,53 +114,74 @@ Create a local configuration file, a file named `config.toml` at the root of the
```toml
# config.toml
# /!\ All the file values are for example, you must change them
[api]
# root_path = "[api-path]"

[oidc]
algorithms = ["RS256"]
api_audience = "[aad_app_id]"
issuers = ["https://login.microsoftonline.com/[tenant_id]/v2.0"]
jwks = "https://login.microsoftonline.com/common/discovery/v2.0/keys"

[monitoring]

[monitoring.logging]
app_level = "DEBUG" # Enum: "NOSET", "DEBUG", "INFO", "WARN", "ERROR", "FATAL", "CRITICAL"
sys_level = "WARN" # Enum: "NOSET", "DEBUG", "INFO", "WARN", "ERROR", "FATAL", "CRITICAL"

[monitoring.azure_app_insights]
connection_str = "InstrumentationKey=[key];[...]"

[persistence]
# Enum: "qdrant"
search = "qdrant"
# Enum: "redis", "cosmos"
store = "cosmos"
# Enum: "redis"
stream = "redis"
cache = "redis" # Enum: "redis"
search = "qdrant" # Enum: "qdrant"
store = "cosmos" # Enum: "cache", "cosmos"
stream = "redis" # Enum: "redis"

[api]
root_path = ""
[persistence.qdrant]
host = "[host]"

[appinsights]
connection_str = "InstrumentationKey=00000000-0000-0000-0000-000000000000;IngestionEndpoint=https://westeurope-5.in.applicationinsights.azure.com/;LiveEndpoint=https://westeurope.livediagnostics.monitor.azure.com"
[persistence.redis]
db = 0
host = "[host]"

[persistence.cosmos]
# Containers "conversation" (/user_id), "message" (/conversation_id), "user" (/dummy), "usage" (/user_id) must exist
url = "https://[deployment].documents.azure.com:443"
database = "[db_name]"

[openai]
[ai]

[ai.openai]
ada_deploy_id = "ada"
ada_max_tokens = 2049
api_base = "https://moaw-dev-clesne-moaw-search-oai.openai.azure.com"
api_base = "https://[deployment].openai.azure.com"
gpt_deploy_id = "gpt"
gpt_max_tokens = 4096

[acs]
api_base = "https://moaw-dev-clesne-moaw-search-acs.cognitiveservices.azure.com"
api_token = "00000000000000000000000000000000"
[ai.azure_content_safety]
api_base = "https://[deployment].cognitiveservices.azure.com"
api_token = "[api_token]"
max_length = 1000

[logging]
app_level = "DEBUG"
sys_level = "WARN"
[tools]

[oidc]
algorithms = ["RS256"]
api_audience = "00000000-0000-0000-0000-000000000000"
issuers = ["https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/v2.0"]
jwks = "https://login.microsoftonline.com/common/discovery/v2.0/keys"
[tools.azure_form_recognizer]
api_base = "https://[deployment].cognitiveservices.azure.com"
api_token = "[api_token]"

[qd]
host = "127.0.0.1"
[tools.bing]
search_url = "https://api.bing.microsoft.com/v7.0/search"
subscription_key = "[api_token]"

[redis]
db = 0
host = "127.0.0.1"
[tools.tmdb]
bearer_token = "[jwt_token]"

[cosmos]
# Containers "conversation" (/user_id), "message" (/conversation_id), "user" (/dummy), "usage" (/user_id) must exist
url = "https://private-gpt.documents.azure.com:443"
database = "private-gpt"
[tools.news]
api_key = "[api_token]"

[tools.listen_notes]
api_key = "[api_token]"
```

Now, you can either run the application as container or with live reload. For development, it is recommended to use live reload. For demo, it is recommended to use the container.
@@ -115,43 +219,7 @@ helm upgrade --install default clemlesne-private-gpt/private-gpt

Go to [http://127.0.0.1:8081/redoc](http://127.0.0.1:8081/redoc).

![Documentation endpoint](docs/doc.png)

## How it works

### High level

```mermaid
sequenceDiagram
autonumber
actor User
participant PWA
participant API
participant OpenAI
PWA ->> API: Ask for conversations
activate API
API ->> API: Get conversations from storage
API ->> PWA: Answer with conversations
deactivate API
User ->> PWA: Select a conversation
User ->> PWA: Insert a message
PWA ->> API: Send the message
activate API
API ->> OpenAI: Ask for a completion
activate OpenAI
OpenAI ->> API: Send completion
deactivate OpenAI
API ->> API: Save conversation and message
API ->> PWA: Answer with the message
deactivate API
User ->> PWA: See results
```

### Architecture

WIP
![Documentation endpoint](docs/api.png)

## [Security](./SECURITY.md)

18 changes: 15 additions & 3 deletions cicd/helm/Makefile
View file
Original file line number Diff line number Diff line change
@@ -1,14 +1,26 @@
.PHONY: deploy
namespace ?= private-gpt
instance ?= default

deploy:
test -n "$(NAMESPACE)" # $$NAMESPACE
helm upgrade \
--atomic \
--dependency-update \
--install \
--namespace $(namespace) \
--timeout 5m \
--values .values.yaml \
--wait \
$(instance) private-gpt

dry-run:
helm upgrade \
--atomic \
--dependency-update \
--dry-run \
--install \
--namespace $(NAMESPACE) \
--namespace $(namespace) \
--timeout 5m \
--values .values.yaml \
--wait \
default private-gpt
$(instance) private-gpt
94 changes: 67 additions & 27 deletions cicd/helm/private-gpt/templates/conversation-api-config.yaml
View file
Original file line number Diff line number Diff line change
@@ -7,45 +7,85 @@ metadata:
app.kubernetes.io/component: conversation-api
data:
config.toml: |
[oidc]
algorithms = ["{{ join "\",\"" .Values.oidc.algorithms | required "A value for .Values.oidc.algorithms is required" }}"]
api_audience = {{ .Values.oidc.api_audience | quote | required "A value for .Values.oidc.api_audience is required" }}
issuers = ["{{ join "\",\"" .Values.oidc.issuers | required "A value for .Values.oidc.issuers is required" }}"]
jwks = {{ .Values.oidc.jwks | quote | required "A value for .Values.oidc.jwks is required" }}
[api]
root_path = "/{{ include "private-gpt.fullname" . }}-conversation-api"
[monitoring]
[monitoring.logging]
app_level = {{ .Values.monitoring.logging.app | quote | required "A value for .Values.monitoring.logging.app is required" }}
sys_level = {{ .Values.monitoring.logging.sys | quote | required "A value for .Values.monitoring.logging.sys is required" }}
[monitoring.azure_app_insights]
connection_str = {{ .Values.monitoring.azure_app_insights.connection_str | quote | required "A value for .Values.monitoring.azure_app_insights.connection_str is required" }}
[persistence]
cache = "redis"
search = "qdrant"
store = "cosmos"
stream = "redis"
[api]
root_path = "/{{ include "private-gpt.fullname" . }}-conversation-api"
[persistence.qdrant]
host = {{ include "qdrant.fullname" .Subcharts.qdrant | quote | required "A value for .Values.qdrant.host is required" }}
[persistence.redis]
db = {{ .Values.redis.db | int | required "A value for .Values.redis.db is required" }}
host = "{{ include "common.names.fullname" .Subcharts.redis }}-master"
[appinsights]
connection_str = {{ .Values.appinsights.connection_str | required "A value for .Values.appinsights.connection_str" | quote }}
[persistence.cosmos]
url = {{ .Values.persistence.cosmos.url | quote | required "A value for .Values.persistence.cosmos.url is required" }}
database = {{ .Values.persistence.cosmos.database | quote | required "A value for .Values.persistence.cosmos.database is required" }}
[openai]
ada_deploy_id = {{ .Values.api.openai.ada_deploy_id | quote }}
[ai]
[ai.openai]
ada_deploy_id = {{ .Values.ai.openai.ada_deploy_id | quote | required "A value for .Values.ai.openai.ada_deploy_id is required" }}
ada_max_tokens = 2049
api_base = {{ .Values.api.openai.base | quote }}
gpt_deploy_id = {{ .Values.api.openai.gpt_deploy_id | quote }}
gpt_max_tokens = 4096
api_base = {{ .Values.ai.openai.base | quote | required "A value for .Values.ai.openai.base is required" }}
gpt_deploy_id = {{ .Values.ai.openai.gpt_deploy_id | quote | required "A value for .Values.ai.openai.gpt_deploy_id is required" }}
gpt_max_tokens = 16384
[acs]
api_base = {{ .Values.api.acs.base | quote }}
[ai.azure_content_safety]
api_base = {{ .Values.ai.azure_content_safety.base | quote | required "A value for .Values.ai.azure_content_safety.base is required" }}
api_token = {{ .Values.ai.azure_content_safety.token | quote | required "A value for .Values.ai.azure_content_safety.token is required" }}
max_length = 1000
[logging]
app_level = {{ .Values.api.logging.app | quote }}
sys_level = {{ .Values.api.logging.sys | quote }}
[tools]
[oidc]
algorithms = ["{{ join "\",\"" .Values.oidc.algorithms }}"]
api_audience = {{ .Values.oidc.api_audience | quote }}
issuers = ["{{ join "\",\"" .Values.oidc.issuers }}"]
jwks = {{ .Values.oidc.jwks | quote }}
[tools.azure_form_recognizer]
api_base = {{ .Values.tools.azure_form_recognizer.api_base | quote | required "A value for .Values.tools.azure_form_recognizer.api_base is required" }}
api_token = {{ .Values.tools.azure_form_recognizer.api_token | quote | required "A value for .Values.tools.azure_form_recognizer.api_token is required" }}
[qd]
host = {{ include "qdrant.fullname" .Subcharts.qdrant | quote }}
[tools.bing]
search_url = {{ .Values.tools.bing.search_url | quote | required "A value for .Values.tools.bing.search_url is required" }}
subscription_key = {{ .Values.tools.bing.subscription_key | quote | required "A value for .Values.tools.bing.subscription_key is required" }}
[redis]
db = {{ .Values.redis.db | int }}
host = "{{ include "common.names.fullname" .Subcharts.redis }}-master"
[tools.tmdb]
bearer_token = {{ .Values.tools.tmdb.bearer_token | quote | required "A value for .Values.tools.tmdb.bearer_token is required" }}
[tools.news]
api_key = {{ .Values.tools.news.api_key | quote | required "A value for .Values.tools.news.api_key is required" }}
[tools.listen_notes]
api_key = {{ .Values.tools.listen_notes.api_key | quote | required "A value for .Values.tools.listen_notes.api_key is required is required" }}
[tools.open_weather_map]
api_key = {{ .Values.tools.open_weather_map.api_key | quote | required "A value for .Values.tools.open_weather_map.api_key is required" }}
[cosmos]
url = {{ .Values.cosmos.url | quote }}
database = {{ .Values.cosmos.database | quote }}
{{- range .Values.tools.azure_cognitive_search }}
# Tool {{ .displayed_name | required "A value for .displayed_name is required" }}
[[tools.azure_cognitive_search]]
api_key = {{ .api_key | quote | required "A value for .api_key is required" }}
content_key = {{ .content_key | quote | required "A value for .content_key is required" }}
displayed_name = {{ .displayed_name | quote | required "A value for .displayed_name is required" }}
index_name = {{ .index_name | quote | required "A value for .index_name is required" }}
service_name = {{ .service_name | quote | required "A value for .service_name is required" }}
top_k = {{ .top_k | int | required "A value for .top_k is required" }}
usage = {{ .usage | quote | required "A value for .usage is required" }}
{{- end }}
2 changes: 1 addition & 1 deletion cicd/helm/private-gpt/templates/conversation-api-secret.yaml
View file
Original file line number Diff line number Diff line change
@@ -6,4 +6,4 @@ metadata:
labels:
{{- include "private-gpt.labels" . | nindent 4 }}
stringData:
PG_ACS_API_TOKEN: {{ .Values.api.acs.token | required "A value for .Values.api.acs.token is required" }}
PG_ACS_API_TOKEN: {{ .Values.ai.azure_content_safety.token | required "A value for .Values.ai.azure_content_safety.token is required" }}
Loading