Chat participant API · Issue #199908 · microsoft/vscode

[irthomasthomas]

• Chat participant API · Issue #199908 · microsoft/vscode

Chat participant API · Issue #199908 · microsoft/vscode

Description:

• Proposal dts: microsoft/vscode
• Sample: microsoft/vscode-extension-samples
• Docs: Chat Participant API

Extension authors can subscribe to this issue to get updates about the proposed chat agent API. There may still be breaking changes coming, and I will post here whenever a breaking change is made.

I'm also interested in feedback about how you might use this API.

Current TODOs:

• Need to enable agents to identify sessions uniquely and store state, including across a window reload
• Should be able to use shouldRepopulate for agents
• Maybe this should be the default behavior for agents
• ChatAgentReplyFollowup should parameterize the agent and slash command names, don't require the agent to include them in the prompt as text
• Maybe this goes for variables too, but variables like #file are complex
• Enable references and progress messages from chat variable resolvers #204539
• ChatAgentContentReference needs to support https URIs #203822
• Change the history context shape to be a flat list of messages, instead of request/response pairs
• Understand how duplicate agent IDs should be handled
• Enable agents to resolve a variable value on-demand, instead of resolving all variables before invoking the agent
• Add a persist flag to ChatAgentProgressMessage for it to stay visible with a ✔️ even when more content has been added.
• Better support for /help with chat agents #197081
• ChatAgentCommandFollowup should become a progress message instead of a followup
• In history, agents have to parse agent names and slash commands from the raw ChatMessage. They shouldn't have to understand the input syntax- the type in history should be related to the request and progress/result types that we already have.
• Add generic type parameter for the type of the result, which can include custom properties
• 'variables' proposal is referenced from chatAgents2, need to merge it in
• Merge ChatAgentTask and ChatAgentProgressMessage #200598

URL: microsoft/vscode

Suggested labels

{'label-name': 'Chat-API', 'label-description': 'Discussion and implementation of a chat participant API', 'confidence': 70.76}

[irthomasthomas]

Related issues

#676: language-model.md - vscode-docs [GitHub] - Visual Studio Code - GitHub

### Details

Similarity score: 0.88 - [ ] [language-model.md - vscode-docs [GitHub] - Visual Studio Code - GitHub](https://vscode.dev/github/microsoft/vscode-docs/blob/main/api/extension-guides/language-model.md)

language-model.md - vscode-docs [GitHub] - Visual Studio Code - GitHub

DESCRIPTION:

DO NOT TOUCH — Managed by doc writer

ContentId: 9bdc3d4e-e6ba-43d3-bd09-2e127cb63ce7
DateApproved: 02/28/2024

Summarize the whole topic in less than 300 characters for SEO purpose

MetaDescription: A guide to adding AI-powered features to a VS Code extension by using language models and natural language understanding.

Language Model API

• By publishing to the VS Marketplace your extension is adhering to the GitHub Copilot extensibility acceptable development and use policy
• Update the attributes in the package.json to make it easy for users to find your extension. Add "AI" to the categories field in your package.json. If your extension contributes a Chat Participant, add "Chat" as well.
• Upload to the Marketplace as described in Publishing Extension.

Related content

• Build a chat extension

URL: language-model.md

Suggested labels

{'label-name': 'VSCode-Extension-Guide', 'label-description': 'Content related to creating VS Code extensions with language models.', 'gh-repo': 'microsoft/vscode-docs', 'confidence': 62.54}

#418: openchat/openchat-3.5-1210 · Hugging Face

### Details

Similarity score: 0.86 - [ ] [openchat/openchat-3.5-1210 · Hugging Face](https://huggingface.co/openchat/openchat-3.5-1210#conversation-templates)

Using the OpenChat Model

We highly recommend installing the OpenChat package and using the OpenChat OpenAI-compatible API server for an optimal experience. The server is optimized for high-throughput deployment using vLLM and can run on a consumer GPU with 24GB RAM.

• Installation Guide: Follow the installation guide in our repository.

• Serving: Use the OpenChat OpenAI-compatible API server by running the serving command from the table below. To enable tensor parallelism, append --tensor-parallel-size N to the serving command.

  Model	Size	Context	Weights	Serving
  OpenChat 3.5	1210	7B	8192	python -m ochat.serving.openai_api_server --model openchat/openchat-3.5-1210 --engine-use-ray --worker-use-ray

• API Usage: Once started, the server listens at localhost:18888 for requests and is compatible with the OpenAI ChatCompletion API specifications. Here's an example request:

  curl http://localhost:18888/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
          "model": "openchat_3.5",
          "messages": [{"role": "user", "content": "You are a large language model named OpenChat. Write a poem to describe yourself"}]
        }'

• Web UI: Use the OpenChat Web UI for a user-friendly experience.

Online Deployment

If you want to deploy the server as an online service, use the following options:

• --api-keys sk-KEY1 sk-KEY2 ... to specify allowed API keys
• --disable-log-requests --disable-log-stats --log-file openchat.log for logging only to a file.

For security purposes, we recommend using an HTTPS gateway in front of the server.

Mathematical Reasoning Mode

The OpenChat model also supports mathematical reasoning mode. To use this mode, include condition: "Math Correct" in your request.

```bash
curl http://localhost:18888/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
        "model": "openchat_3.5",
        "condition": "Math Correct",
        "messages": [{"role": "user", "content": "10.3 − 7988.8133 = "}]
      }'
```

Conversation Templates

We provide several pre-built conversation templates to help you get started.

• Default Mode (GPT4 Correct):

  GPT4 Correct User: Hello<|end_of_turn|>
  GPT4 Correct Assistant: Hi<|end_of_turn|>
  GPT4 Correct User: How are you today?<|end_of_turn|>
  GPT4 Correct Assistant:

• Mathematical Reasoning Mode:

  Math Correct User: 10.3 − 7988.8133=<|end_of_turn|>
  Math Correct Assistant:

  NOTE: Remember to set <|end_of_turn|> as end of generation token.

• Integrated Tokenizer: The default (GPT4 Correct) template is also available as the integrated tokenizer.chat_template, which can be used instead of manually specifying the template.

Suggested labels

{ "label": "chat-templates", "description": "Pre-defined conversation structures for specific modes of interaction." }

#632: OpenRouter: Assistant Prefill supports asking models to complete a partial response.

### Details

Similarity score: 0.84 - [ ] [Docs | OpenRouter](https://openrouter.ai/docs#responses)

Docs | OpenRouter

DESCRIPTION: "Assistant Prefill: OpenRouter supports asking models to complete a partial response. This can be useful for guiding models to respond in a certain way.
To use this features, simply include a message with role: "assistant" at the end of your messages array. Example:

fetch("https://openrouter.ai/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${OPENROUTER_API_KEY}`,
    "HTTP-Referer": `${YOUR_SITE_URL}`, // Optional, for including your app on openrouter.ai rankings.
    "X-Title": `${YOUR_SITE_NAME}`, // Optional. Shows in rankings on openrouter.ai.
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "messages": [
      {"role": "user", "content": "Who are you?"},
      {"role": "assistant", "content": "I'm not sure, but my best guess is"},
    ],
  })
});

URL: OpenRouter Documentation

Suggested labels

{'label-name': 'Chatbot-API', 'label-description': 'API documentation for interacting with chatbots on OpenRouter.', 'gh-repo': 'AI-Chatbots', 'confidence': 65.43}

#656: ChatCraft - open-source web companion for coding with LLMs.

### Details

Similarity score: 0.84 - [ ] [New Chat - ChatCraft](https://chatcraft.org/c/zIpIpIKuJ8E3S8jlMsJTO)

New Chat - ChatCraft

DESCRIPTION:
Welcome to ChatCraft, your open-source web companion for coding with Large Language Models (LLMs). Designed with developers in mind, ChatCraft transforms the way you interact with GPT models, making it effortless to read, write, debug, and enhance your code.

We think ChatCraft is the best platform for learning, experimenting, and getting creative with code. Here's a few of the reasons why we think you'll agree:

Feature	ChatCraft	ChatGPT	Copilot
Optimized for conversations about code	✅	❌	❌
Work with models from multiple AI vendors	✅	❌	❌
Previews for Mermaid Diagrams, HTML	✅	❌	❌
Edit Generated AI Replies	✅	❌	✅
Use Custom System Prompts	✅	✅	❌
Easy to retry with different AI models	✅	❌	❌
Edit/Run Generated Code and Custom Functions	✅	❌	✅
Open Source	✅	❌	❌

Learn more about ChatCraft on GitHub

Quick Start Instructions
You can begin using ChatCraft today by following these steps:

1. Choose an AI provider below: we support both OpenAI and OpenRouter. OpenAI supports various versions of ChatGPT (gpt-3.5-turbo) and GPT-4 models, while OpenRouter adds support for even more models from vendors like Anthropic, Google, and Meta. It's easy to switch providers later, or go back-and-forth.
2. Enter an API Key. ChatCraft is a "bring your own API Key" web app. No matter which provider you choose, ChatCraft needs an API Key to start making API calls on your behalf. API Keys are never shared, and get stored in your browser's local storage.
3. Start chatting with AI! Type your question in the textbox at the bottom of the screen and click the Ask button to prompt a particular model (switch to a different model whenever you like).
4. Copy, edit, delete, or retry any AI response with a different model until you get the results you need.
5. Every chat is saved to a local, offline database in your browser, which you can search (top of UI) or navigate by opening the sidebar with the hamburger menu in the top-left.

Suggested labels

#122: Timeout issue with AutoExpertV6 · Issue #88 · spdustin/ChatGPT-AutoExpert

### Details

Similarity score: 0.83 - [ ] [Timeout issue with AutoExpertV6 · Issue #88 · spdustin/ChatGPT-AutoExpert](https://github.com/spdustin/ChatGPT-AutoExpert/issues/88#user-content-fnref-1-ef146207cdb1f9c774ad0c13dbdc8219)

Quote

I am having the same issue as well. Here is the workaround that has been working for me (so far):

When the response is interrupted due to an error (e.g., {network error}), refresh the page in your browser1 – do not regenerate the response.
The "Regenerate" button will have disappeared, and you will be able to enter a new message prompt.
Give AutoExpert the prompt, "Please continue."
It will then continue from where it left off.

I'm paranoid about losing chat context, so before I refresh the page, I copy the response that was interrupted and paste it into an editor (I use MarkText and Obsidian). That way, if the response isn't there after refreshing the page, I can provide the response to AutoExpert and ask it to continue where it left off. ↩

</details></details>

#305: Home - LibreChat

### Details

Similarity score: 0.82 - [ ] [Home - LibreChat](https://docs.librechat.ai/index.html)

Table of contents
🪶 Features
📃 All-In-One AI Conversations with LibreChat
⭐ Star History
✨ Contributors
💖 This project exists in its current state thanks to all the people who contribute

LibreChat

🪶 Features

🖥️ UI matching ChatGPT, including Dark mode, Streaming, and 11-2023 updates
💬 Multimodal Chat:
Upload and analyze images with GPT-4 and Gemini Vision 📸
More filetypes and Assistants API integration in Active Development 🚧
🌎 Multilingual UI:
English, 中文, Deutsch, Español, Français, Italiano, Polski, Português Brasileiro, Русский
日本語, Svenska, 한국어, Tiếng Việt, 繁體中文, العربية, Türkçe, Nederlands
🤖 AI model selection: OpenAI API, Azure, BingAI, ChatGPT, Google Vertex AI, Anthropic (Claude), Plugins
💾 Create, Save, & Share Custom Presets
🔄 Edit, Resubmit, and Continue messages with conversation branching
📤 Export conversations as screenshots, markdown, text, json.
🔍 Search all messages/conversations
🔌 Plugins, including web access, image generation with DALL-E-3 and more
👥 Multi-User, Secure Authentication with Moderation and Token spend tools
⚙️ Configure Proxy, Reverse Proxy, Docker, many Deployment options, and completely Open-Source
📃 All-In-One AI Conversations with LibreChat

LibreChat brings together the future of assistant AIs with the revolutionary technology of OpenAI's ChatGPT. Celebrating the original styling, LibreChat gives you the ability to integrate multiple AI models. It also integrates and enhances original client features such as conversation and message search, prompt templates and plugins.

With LibreChat, you no longer need to opt for ChatGPT Plus and can instead use free or pay-per-call APIs. We welcome contributions, cloning, and forking to enhance the capabilities of this advanced chatbot platform.

Suggested labels

"ai-platform"