Add base API URL field for Ollama and OpenAI embedding models #1136

srdas · 2024-12-04T07:16:22Z

Description

Jupyter AI currently allows the user to call a model at a URL (location) different from the default one by specifying a selected Base API URL. This can be done for Ollama, OpenAI provider models. However, for these providers, there is no way to change the API URL for embedding models when using the /learn command in RAG mode. This PR adds an extra field to make this feasible.

Testing instructions

Testing as follows for Ollama:
[1] Start the Ollama system from port 11435 instead 11434 (the default):
OLLAMA_HOST=127.0.0.1:11435 ollama serve
[2] Set the Base API URL:

[3] Check that the new API URL works for completions and /learn:

Jupyter AI currently allows the user to call a model at a URL (location) different from the default one by specifying a selected Base API URL. This can be done for Ollama, OpenAI provider models. However, for these providers, there is no way to change the API URL for embedding models when using the `/learn` command in RAG mode. This PR adds an extra field to make this feasible. Tested as follows for Ollama: [1] Start the Ollama system from port 11435 instead 11434 (the default): `OLLAMA_HOST=127.0.0.1:11435 ollama serve` [2] Set the Base API URL: [3] Check that the new API URL works:

for more information, see https://pre-commit.ci

dlqqq · 2024-12-09T18:05:29Z

I noticed a bug on a24b436 on Friday but didn't have time to document it. If you change the OpenAI embeddings base URL to an empty value and save, all subsequent connections fail. I did some debugging and found that the OpenAIEmbeddings provider class fails if you pass it openai_api_base="", because it treats the empty string as the API base instead of ignoring it.

I'm pushing a change that alters the ConfigManager to exclude empty string fields from the keyword arguments it provides. This fixes the bug for me locally. Most providers already ignore optional keyword arguments that are set to an empty string, so this shouldn't cause any change for users.

for more information, see https://pre-commit.ci

srdas

Tested /learn and /ask with haiku-3.5 and titan-embed-v1. ✅ (No Base API URL)
Tested /learn and /ask with ollama-llama3.2 and ollama-mxbai-embed-large. ✅ (No Base API URL) -- all next tests with Ollama (to test changes in ollama.py ).
Tested again with Base API URL = 11434 (the default explicitly). ✅
Tested again after clearing out the Base API URL, still works ✅
Restarted Ollama with port=12345. Added this to the Base API URL and it all works as expected. ✅
Removed the custom Base API URL (blank field) and the /learn and /ask commands now fail, as they should, because Ollama is still running on the custom port. :gr-checkmark:
Leaving custom fields blank, restarted Ollama to return to default API URL and everything works as expected. ✅
With OpenAI embeddings, left the Base API URL blank (it works ✅), then added the URL (it works ✅) and then deleted the URL (and it still works ✅, confirms the change in config_manager.py is implemented).

Code looks good as well.

dlqqq · 2024-12-09T23:29:30Z

Kicking CI since the RTD workflow has stalled.

dlqqq · 2024-12-09T23:48:48Z

@meeseeksdev please backport to v3-dev

…enAI embedding models

…ding models (#1149) Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR #1049: Added new Anthropic Sonnet3.5 v2 models (#1050) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1051: Added Developer documentation for streaming responses (#1058) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1048: Implement streaming for `/fix` (#1059) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1057: [pre-commit.ci] pre-commit autoupdate (#1060) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Backport PR #1064: Added Ollama to the providers table in user docs (#1066) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1056: Add examples of using Fields and EnvAuthStrategy to developer documentation (#1073) Co-authored-by: Alan Meeson <alanmeeson@users.noreply.github.com> * Backport PR #1069: Merge Anthropic language model providers (#1076) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1068: Allow `$` to literally denote quantities of USD in chat (#1079) Co-authored-by: david qiu <david@qiu.dev> * Backport PR #1075: Fix magic commands when using non-chat providers w/ history (#1080) Co-authored-by: Alan Meeson <alanmeeson@users.noreply.github.com> * Backport PR #1077: Fix `/export` by including streamed agent messages (#1081) Co-authored-by: Mahmut CAVDAR <4072246+mcavdar@users.noreply.github.com> * Backport PR #1072: Reduced padding in cell around code icons in code toolbar (#1084) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1087: Improve installation documentation and clarify provider dependencies (#1091) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1092: Remove retired models and add new `Haiku-3.5` model in Anthropic (#1093) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1094: Continue to allow `$` symbols to delimit inline math in human messages (#1095) Co-authored-by: david qiu <david@qiu.dev> * Backport PR #1097: Update `faiss-cpu` version range (#1101) Co-authored-by: david qiu <david@qiu.dev> * Backport PR #1104: Fix rendering of code blocks in JupyterLab 4.3.0+ (#1105) Co-authored-by: david qiu <david@qiu.dev> * Backport PR #1106: Catch error on non plaintext files in `@file` and reply gracefully in chat (#1110) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1109: Bump LangChain minimum versions (#1112) Co-authored-by: david qiu <david@qiu.dev> * Backport PR #1119: Downgrade spurious 'error' logs (#1124) Co-authored-by: ctcjab <joshua.bronson@chicagotrading.com> * Backport PR #1127: Removes outdated OpenAI models and adds new ones (#1130) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1131: [pre-commit.ci] pre-commit autoupdate (#1132) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Backport PR #1125: Update model fields immediately on save (#1133) Co-authored-by: david qiu <david@qiu.dev> * Backport PR #1139: Fix install step in CI (#1140) Co-authored-by: david qiu <david@qiu.dev> * Backport PR #1129: Fix JSON serialization error in Ollama models (#1141) Co-authored-by: Mr.W <janus.choy@gmail.com> * Backport PR #1137: Update completion model fields immediately on save (#1142) Co-authored-by: david qiu <david@qiu.dev> * [v3-dev] Initial migration to `jupyterlab-chat` (#1043) * Very first version of the AI working in jupyterlab_collaborative_chat * Allows both collaborative and regular chat to work with AI * handle the help message in the chat too * Autocompletion (#2) * Fix handler methods' parameters * Add slash commands (autocompletion) to the chat input * Stream messages (#3) * Allow for stream messages * update jupyter collaborative chat dependency * AI settings (#4) * Add a menu option to open the AI settings * Remove the input option from the setting widget * pre-commit * linting * Homogeneize typing for optional arguments * Fix import * Showing that the bot is writing (answering) (#5) * Show that the bot is writing (answering) * Update jupyter chat dependency * Some typing * Update extension to jupyterlab_chat (0.6.0) (#8) * Fix linting * Remove try/except to import jupyterlab_chat (not optional anymore), and fix typing * linter * Python unit tests * Fix typing * lint * Fix lint and mypy all together * Fix web_app settings accessor * Fix jupyter_collaboration version Co-authored-by: david qiu <44106031+dlqqq@users.noreply.github.com> * Remove unecessary try/except * Dedicate one set of chat handlers per room (#9) * create new set of chat handlers per room * make YChat an instance attribute on BaseChatHandler * revert changes to chat handlers * pre-commit * use room_id local var Co-authored-by: Nicolas Brichet <32258950+brichet@users.noreply.github.com> --------- Co-authored-by: Nicolas Brichet <32258950+brichet@users.noreply.github.com> --------- Co-authored-by: david qiu <44106031+dlqqq@users.noreply.github.com> Co-authored-by: david qiu <david@qiu.dev> * Backport PR #1134: Improve user messaging and documentation for Cross-Region Inference on Amazon Bedrock (#1143) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1136: Add base API URL field for Ollama and OpenAI embedding models (#1149) Co-authored-by: Sanjiv Das <srdas@scu.edu> * [v3-dev] Remove `/export`, `/clear`, and `/fix` (#1148) * remove /export * remove /clear * remove /fix * Fix CI in `v3-dev` branch (#1154) * fix check release by bumping to impossible version * fix types * Update Playwright Snapshots --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * [v3-dev] Dedicate one LangChain history object per chat (#1151) * dedicate a separate LangChain history object per chat * pre-commit * fix mypy * Backport PR #1160: Trigger update snapshots based on commenter's role (#1161) Co-authored-by: david qiu <david@qiu.dev> * Backport PR #1155: Fix code output format in IPython (#1162) Co-authored-by: Divyansh Choudhary <divyanshchoudhary99@gmail.com> * Backport PR #1158: Update `/generate` to not split classes & functions across cells (#1164) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Remove v2 frontend components (#1156) * First pass to remove the front end chat * Remove code-toolbar by using a simplified markdown renderer in settings * Remove chat-message-menu (should be ported in jupyter-chat) * Remove chat handler * Follow up 'Remove chat-message-menu (should be ported in jupyter-chat)' commit * Clean package.json * Remove UI tests * Remove the generative AI menu * Remove unused components * run yarn dedupe --------- Co-authored-by: David L. Qiu <david@qiu.dev> * Upgrade to `jupyterlab-chat>=0.7.0` (#1166) * upgrade to jupyterlab-chat 0.7.0 * pre-commit * upgrade to @jupyter/chat ^0.7.0 in frontend * Remove v2 backend components (#1168) * remove v2 llm memory, implement ReplyStream * remove v2 websockets & REST handlers * remove unused v2 data models * fix slash command autocomplete * fix unit tests * remove unused _learned context provider * fix mypy * pre-commit * fix optional k arg in YChatHistory * bump jupyter chat to 0.7.1 to fix Python 3.9 tests * revert accidentally breaking /learn --------- Co-authored-by: Lumberbot (aka Jack) <39504233+meeseeksmachine@users.noreply.github.com> Co-authored-by: Sanjiv Das <srdas@scu.edu> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Alan Meeson <alanmeeson@users.noreply.github.com> Co-authored-by: Mahmut CAVDAR <4072246+mcavdar@users.noreply.github.com> Co-authored-by: ctcjab <joshua.bronson@chicagotrading.com> Co-authored-by: Mr.W <janus.choy@gmail.com> Co-authored-by: Nicolas Brichet <32258950+brichet@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Divyansh Choudhary <divyanshchoudhary99@gmail.com>

…rlab#1136) * Base API URL added for embedding models Jupyter AI currently allows the user to call a model at a URL (location) different from the default one by specifying a selected Base API URL. This can be done for Ollama, OpenAI provider models. However, for these providers, there is no way to change the API URL for embedding models when using the `/learn` command in RAG mode. This PR adds an extra field to make this feasible. Tested as follows for Ollama: [1] Start the Ollama system from port 11435 instead 11434 (the default): `OLLAMA_HOST=127.0.0.1:11435 ollama serve` [2] Set the Base API URL: [3] Check that the new API URL works: * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * allow embedding model fields to be saved * exclude empty str fields from config manager * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: David L. Qiu <david@qiu.dev>

* Backport PR jupyterlab#1049: Added new Anthropic Sonnet3.5 v2 models (jupyterlab#1050) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR jupyterlab#1051: Added Developer documentation for streaming responses (jupyterlab#1058) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR jupyterlab#1048: Implement streaming for `/fix` (jupyterlab#1059) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR jupyterlab#1057: [pre-commit.ci] pre-commit autoupdate (jupyterlab#1060) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Backport PR jupyterlab#1064: Added Ollama to the providers table in user docs (jupyterlab#1066) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR jupyterlab#1056: Add examples of using Fields and EnvAuthStrategy to developer documentation (jupyterlab#1073) Co-authored-by: Alan Meeson <alanmeeson@users.noreply.github.com> * Backport PR jupyterlab#1069: Merge Anthropic language model providers (jupyterlab#1076) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR jupyterlab#1068: Allow `$` to literally denote quantities of USD in chat (jupyterlab#1079) Co-authored-by: david qiu <david@qiu.dev> * Backport PR jupyterlab#1075: Fix magic commands when using non-chat providers w/ history (jupyterlab#1080) Co-authored-by: Alan Meeson <alanmeeson@users.noreply.github.com> * Backport PR jupyterlab#1077: Fix `/export` by including streamed agent messages (jupyterlab#1081) Co-authored-by: Mahmut CAVDAR <4072246+mcavdar@users.noreply.github.com> * Backport PR jupyterlab#1072: Reduced padding in cell around code icons in code toolbar (jupyterlab#1084) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR jupyterlab#1087: Improve installation documentation and clarify provider dependencies (jupyterlab#1091) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR jupyterlab#1092: Remove retired models and add new `Haiku-3.5` model in Anthropic (jupyterlab#1093) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR jupyterlab#1094: Continue to allow `$` symbols to delimit inline math in human messages (jupyterlab#1095) Co-authored-by: david qiu <david@qiu.dev> * Backport PR jupyterlab#1097: Update `faiss-cpu` version range (jupyterlab#1101) Co-authored-by: david qiu <david@qiu.dev> * Backport PR jupyterlab#1104: Fix rendering of code blocks in JupyterLab 4.3.0+ (jupyterlab#1105) Co-authored-by: david qiu <david@qiu.dev> * Backport PR jupyterlab#1106: Catch error on non plaintext files in `@file` and reply gracefully in chat (jupyterlab#1110) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR jupyterlab#1109: Bump LangChain minimum versions (jupyterlab#1112) Co-authored-by: david qiu <david@qiu.dev> * Backport PR jupyterlab#1119: Downgrade spurious 'error' logs (jupyterlab#1124) Co-authored-by: ctcjab <joshua.bronson@chicagotrading.com> * Backport PR jupyterlab#1127: Removes outdated OpenAI models and adds new ones (jupyterlab#1130) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR jupyterlab#1131: [pre-commit.ci] pre-commit autoupdate (jupyterlab#1132) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Backport PR jupyterlab#1125: Update model fields immediately on save (jupyterlab#1133) Co-authored-by: david qiu <david@qiu.dev> * Backport PR jupyterlab#1139: Fix install step in CI (jupyterlab#1140) Co-authored-by: david qiu <david@qiu.dev> * Backport PR jupyterlab#1129: Fix JSON serialization error in Ollama models (jupyterlab#1141) Co-authored-by: Mr.W <janus.choy@gmail.com> * Backport PR jupyterlab#1137: Update completion model fields immediately on save (jupyterlab#1142) Co-authored-by: david qiu <david@qiu.dev> * [v3-dev] Initial migration to `jupyterlab-chat` (jupyterlab#1043) * Very first version of the AI working in jupyterlab_collaborative_chat * Allows both collaborative and regular chat to work with AI * handle the help message in the chat too * Autocompletion (jupyterlab#2) * Fix handler methods' parameters * Add slash commands (autocompletion) to the chat input * Stream messages (jupyterlab#3) * Allow for stream messages * update jupyter collaborative chat dependency * AI settings (jupyterlab#4) * Add a menu option to open the AI settings * Remove the input option from the setting widget * pre-commit * linting * Homogeneize typing for optional arguments * Fix import * Showing that the bot is writing (answering) (jupyterlab#5) * Show that the bot is writing (answering) * Update jupyter chat dependency * Some typing * Update extension to jupyterlab_chat (0.6.0) (jupyterlab#8) * Fix linting * Remove try/except to import jupyterlab_chat (not optional anymore), and fix typing * linter * Python unit tests * Fix typing * lint * Fix lint and mypy all together * Fix web_app settings accessor * Fix jupyter_collaboration version Co-authored-by: david qiu <44106031+dlqqq@users.noreply.github.com> * Remove unecessary try/except * Dedicate one set of chat handlers per room (jupyterlab#9) * create new set of chat handlers per room * make YChat an instance attribute on BaseChatHandler * revert changes to chat handlers * pre-commit * use room_id local var Co-authored-by: Nicolas Brichet <32258950+brichet@users.noreply.github.com> --------- Co-authored-by: Nicolas Brichet <32258950+brichet@users.noreply.github.com> --------- Co-authored-by: david qiu <44106031+dlqqq@users.noreply.github.com> Co-authored-by: david qiu <david@qiu.dev> * Backport PR jupyterlab#1134: Improve user messaging and documentation for Cross-Region Inference on Amazon Bedrock (jupyterlab#1143) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR jupyterlab#1136: Add base API URL field for Ollama and OpenAI embedding models (jupyterlab#1149) Co-authored-by: Sanjiv Das <srdas@scu.edu> * [v3-dev] Remove `/export`, `/clear`, and `/fix` (jupyterlab#1148) * remove /export * remove /clear * remove /fix * Fix CI in `v3-dev` branch (jupyterlab#1154) * fix check release by bumping to impossible version * fix types * Update Playwright Snapshots --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * [v3-dev] Dedicate one LangChain history object per chat (jupyterlab#1151) * dedicate a separate LangChain history object per chat * pre-commit * fix mypy * Backport PR jupyterlab#1160: Trigger update snapshots based on commenter's role (jupyterlab#1161) Co-authored-by: david qiu <david@qiu.dev> * Backport PR jupyterlab#1155: Fix code output format in IPython (jupyterlab#1162) Co-authored-by: Divyansh Choudhary <divyanshchoudhary99@gmail.com> * Backport PR jupyterlab#1158: Update `/generate` to not split classes & functions across cells (jupyterlab#1164) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Remove v2 frontend components (jupyterlab#1156) * First pass to remove the front end chat * Remove code-toolbar by using a simplified markdown renderer in settings * Remove chat-message-menu (should be ported in jupyter-chat) * Remove chat handler * Follow up 'Remove chat-message-menu (should be ported in jupyter-chat)' commit * Clean package.json * Remove UI tests * Remove the generative AI menu * Remove unused components * run yarn dedupe --------- Co-authored-by: David L. Qiu <david@qiu.dev> * Upgrade to `jupyterlab-chat>=0.7.0` (jupyterlab#1166) * upgrade to jupyterlab-chat 0.7.0 * pre-commit * upgrade to @jupyter/chat ^0.7.0 in frontend * Remove v2 backend components (jupyterlab#1168) * remove v2 llm memory, implement ReplyStream * remove v2 websockets & REST handlers * remove unused v2 data models * fix slash command autocomplete * fix unit tests * remove unused _learned context provider * fix mypy * pre-commit * fix optional k arg in YChatHistory * bump jupyter chat to 0.7.1 to fix Python 3.9 tests * revert accidentally breaking /learn --------- Co-authored-by: Lumberbot (aka Jack) <39504233+meeseeksmachine@users.noreply.github.com> Co-authored-by: Sanjiv Das <srdas@scu.edu> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Alan Meeson <alanmeeson@users.noreply.github.com> Co-authored-by: Mahmut CAVDAR <4072246+mcavdar@users.noreply.github.com> Co-authored-by: ctcjab <joshua.bronson@chicagotrading.com> Co-authored-by: Mr.W <janus.choy@gmail.com> Co-authored-by: Nicolas Brichet <32258950+brichet@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Divyansh Choudhary <divyanshchoudhary99@gmail.com>

srdas added the enhancement New feature or request label Dec 4, 2024

srdas and others added 2 commits December 5, 2024 15:27

[pre-commit.ci] auto fixes from pre-commit.com hooks

6df1247

for more information, see https://pre-commit.ci

dlqqq force-pushed the base_api_url branch from 9c5b9d1 to 6df1247 Compare December 5, 2024 23:27

dlqqq and others added 3 commits December 6, 2024 12:28

allow embedding model fields to be saved

a24b436

Merge branch 'main' into base_api_url

0eb18be

Merge branch 'main' into base_api_url

dc4e3af

dlqqq and others added 2 commits December 9, 2024 10:09

exclude empty str fields from config manager

80b24b7

[pre-commit.ci] auto fixes from pre-commit.com hooks

4a69101

for more information, see https://pre-commit.ci

dlqqq changed the title ~~Base API URL added for embedding models~~ Add base API URL field for Ollama and OpenAI embedding models Dec 9, 2024

dlqqq marked this pull request as ready for review December 9, 2024 19:52

srdas commented Dec 9, 2024

View reviewed changes

dlqqq approved these changes Dec 9, 2024

View reviewed changes

dlqqq closed this Dec 9, 2024

dlqqq reopened this Dec 9, 2024

dlqqq merged commit 1cbd853 into jupyterlab:main Dec 9, 2024
20 checks passed

meeseeksmachine pushed a commit to meeseeksmachine/jupyter-ai that referenced this pull request Dec 9, 2024

Backport PR jupyterlab#1136: Add base API URL field for Ollama and Op…

802f085

…enAI embedding models

meeseeksmachine mentioned this pull request Dec 9, 2024

Backport PR #1136 on branch v3-dev (Add base API URL field for Ollama and OpenAI embedding models) #1149

Merged

dlqqq pushed a commit that referenced this pull request Dec 10, 2024

Backport PR #1136: Add base API URL field for Ollama and OpenAI embed…

b37b7be

…ding models (#1149) Co-authored-by: Sanjiv Das <srdas@scu.edu>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add base API URL field for Ollama and OpenAI embedding models #1136

Add base API URL field for Ollama and OpenAI embedding models #1136

srdas commented Dec 4, 2024 •

edited by dlqqq

Loading

dlqqq commented Dec 9, 2024

srdas left a comment

dlqqq commented Dec 9, 2024

dlqqq commented Dec 9, 2024

Add base API URL field for Ollama and OpenAI embedding models #1136

Add base API URL field for Ollama and OpenAI embedding models #1136

Conversation

srdas commented Dec 4, 2024 • edited by dlqqq Loading

Description

Testing instructions

dlqqq commented Dec 9, 2024

srdas left a comment

Choose a reason for hiding this comment

dlqqq commented Dec 9, 2024

dlqqq commented Dec 9, 2024

srdas commented Dec 4, 2024 •

edited by dlqqq

Loading