diff --git a/README.md b/README.md index 55d60a7f..08ad8ebc 100644 --- a/README.md +++ b/README.md @@ -18,14 +18,14 @@ https://github.com/user-attachments/assets/8cad5643-63b2-4641-a5c4-68bc313f20e6 CopilotChat.nvim is a Neovim plugin that brings GitHub Copilot Chat capabilities directly into your editor. It provides: -- 🤖 GitHub Copilot Chat integration with official model and agent support (GPT-4o, Claude 3.7 Sonnet, Gemini 2.0 Flash, and more) +- 🤖 GitHub Copilot Chat integration with official model support (GPT-4o, Claude 3.7 Sonnet, Gemini 2.0 Flash, and more) - 💻 Rich workspace context powered by smart embeddings system -- 🔒 Explicit context sharing - only sends what you specifically request, either as context or selection (by default visual selection) -- 🔌 Modular provider architecture supporting both official and custom LLM backends (Ollama, LM Studio, Mistral.ai and more) +- 🔒 Explicit data sharing - only sends what you specifically request, either as resource or selection (by default visual selection) +- 🔌 Modular provider architecture supporting both official and custom LLM backends (Ollama, Gemini, Mistral.ai and more) - 📝 Interactive chat UI with completion, diffs and quickfix integration - 🎯 Powerful prompt system with composable templates and sticky prompts -- 🔄 Extensible context providers for granular workspace understanding (buffers, files, git diffs, URLs, and more) -- ⚡ Efficient token usage with tiktoken token counting and memory management +- 🔄 Extensible function calling system for granular workspace understanding (buffers, files, git diffs, URLs, and more) +- ⚡ Efficient token usage with tiktoken token counting and history management # Requirements @@ -62,8 +62,7 @@ Plugin features that use picker: - `:CopilotChatPrompts` - for selecting prompts - `:CopilotChatModels` - for selecting models -- `:CopilotChatAgents` - for selecting agents -- `#:` - for selecting context input +- `#:` - for selecting function input # Installation @@ -148,7 +147,6 @@ Commands are used to control the chat interface: | `:CopilotChatLoad ?` | Load chat history | | `:CopilotChatPrompts` | View/select prompt templates | | `:CopilotChatModels` | View/select available models | -| `:CopilotChatAgents` | View/select available agents | | `:CopilotChat` | Use specific prompt template | ## Key Mappings @@ -253,7 +251,7 @@ Define your own system prompts in the configuration (similar to `prompts`): ### Sticky Prompts -Sticky prompts persist across chat sessions. They're useful for maintaining context or agent selection. They work as follows: +Sticky prompts persist across chat sessions. They're useful for maintaining model or resource selection. They work as follows: 1. Prefix text with `> ` using markdown blockquote syntax 2. The prompt will be copied at the start of every new chat prompt @@ -262,7 +260,7 @@ Sticky prompts persist across chat sessions. They're useful for maintaining cont Examples: ```markdown -> #files +> #glob:`*.lua` > List all files in the workspace > @models Using Mistral-small @@ -274,15 +272,12 @@ You can also set default sticky prompts in the configuration: ```lua { sticky = { - '@models Using Mistral-small', - '#files', + '#glob:*.lua', } } ``` -## Models and Agents - -### Models +## Models You can control which AI model to use in three ways: @@ -295,69 +290,65 @@ For supported models, see: - [Copilot Chat Models](https://docs.github.com/en/copilot/using-github-copilot/ai-models/changing-the-ai-model-for-copilot-chat#ai-models-for-copilot-chat) - [GitHub Marketplace Models](https://github.com/marketplace/models) (experimental, limited usage) -### Agents - -Agents determine the AI assistant's capabilities. Control agents in three ways: - -1. List available agents with `:CopilotChatAgents` -2. Set agent in prompt with `@agent_name` -3. Configure default agent via `agent` config key - -The default "noop" agent is `none`. For more information: - -- [Extension Agents Documentation](https://docs.github.com/en/copilot/using-github-copilot/using-extensions-to-integrate-external-tools-with-copilot-chat) -- [Available Agents](https://github.com/marketplace?type=apps&copilot_app=true) - -## Contexts - -Contexts provide additional information to the chat. Add context using `#context_name[:input]` syntax: - -| Context | Input Support | Description | -| ----------- | ------------- | ----------------------------------- | -| `buffer` | ✓ (number) | Current or specified buffer content | -| `buffers` | ✓ (type) | All buffers content (listed/all) | -| `file` | ✓ (path) | Content of specified file | -| `files` | ✓ (glob) | Workspace files | -| `filenames` | ✓ (glob) | Workspace file names | -| `git` | ✓ (ref) | Git diff (unstaged/staged/commit) | -| `url` | ✓ (url) | Content from URL | -| `register` | ✓ (name) | Content of vim register | -| `quickfix` | - | Quickfix list file contents | -| `system` | ✓ (command) | Output of shell command | - -> [!TIP] -> The AI is aware of these context providers and may request additional context -> if needed by asking you to input a specific context command like `#file:path/to/file.js`. +## Functions + +Functions provide additional information and behaviour to the chat. +Tools can be organized into groups by setting the `group` property. Tools assigned to a group are not automatically made available to the LLM - they must be explicitly activated. +To use grouped tools in your prompt, include `@group_name` in your message. This allows the LLM to access and use all tools in that group during the current interaction. +Add tools using `#tool_name[:input]` syntax: + +| Function | Input Support | Description | +| ------------- | ------------- | ------------------------------------------------------ | +| `buffer` | ✓ (name) | Retrieves content from a specific buffer | +| `buffers` | ✓ (scope) | Fetches content from multiple buffers (listed/visible) | +| `diagnostics` | ✓ (scope) | Collects code diagnostics (errors, warnings) | +| `file` | ✓ (path) | Reads content from a specified file path | +| `gitdiff` | ✓ (sha) | Retrieves git diff information (unstaged/staged/sha) | +| `gitstatus` | - | Retrieves git status information | +| `glob` | ✓ (pattern) | Lists filenames matching a pattern in workspace | +| `grep` | ✓ (pattern) | Searches for a pattern across files in workspace | +| `quickfix` | - | Includes content of files in quickfix list | +| `register` | ✓ (register) | Provides access to specified Vim register | +| `url` | ✓ (url) | Fetches content from a specified URL | Examples: ```markdown -> #buffer -> #buffer:2 -> #files:\*.lua -> #filenames +> #buffer:init.lua +> #buffers:visible +> #diagnostics:current +> #file:path/to/file.js > #git:staged +> #glob:`**/*.lua` +> #grep:`function setup` +> #quickfix +> #register:+ > #url:https://example.com -> #system:`ls -la | grep lua` ``` -Define your own contexts in the configuration with input handling and resolution: +Define your own functions in the configuration with input handling and schema: ```lua { - contexts = { + functions = { birthday = { - input = function(callback) - vim.ui.select({ 'user', 'napoleon' }, { - prompt = 'Select birthday> ', - }, callback) - end, + description = "Retrieves birthday information for a person", + schema = { + type = 'object', + required = { 'name' }, + properties = { + name = { + type = 'string', + enum = { 'Alice', 'Bob', 'Charlie' }, + description = "Person's name", + }, + }, + }, resolve = function(input) return { { - content = input .. ' birthday info', - filename = input .. '_birthday', - filetype = 'text', + type = 'text', + data = input.name .. ' birthday info', } } end @@ -366,9 +357,9 @@ Define your own contexts in the configuration with input handling and resolution } ``` -### External Contexts +### External Functions -For external contexts, see the [contexts discussion page](https://github.com/CopilotC-Nvim/CopilotChat.nvim/discussions/categories/contexts). +For external functions implementations, see the [discussion page](https://github.com/CopilotC-Nvim/CopilotChat.nvim/discussions/categories/functions). ## Selections @@ -430,9 +421,6 @@ Custom providers can implement these methods: -- Optional: Get available models get_models?(headers: table): table, - - -- Optional: Get available agents - get_agents?(headers: table): table, } ``` @@ -454,15 +442,14 @@ Below are all available configuration options with their default values: system_prompt = 'COPILOT_INSTRUCTIONS', -- System prompt to use (can be specified manually in prompt via /). model = 'gpt-4o-2024-11-20', -- Default model to use, see ':CopilotChatModels' for available models (can be specified manually in prompt via $). - agent = 'copilot', -- Default agent to use, see ':CopilotChatAgents' for available agents (can be specified manually in prompt via @). - context = nil, -- Default context or array of contexts to use (can be specified manually in prompt via #). - sticky = nil, -- Default sticky prompt or array of sticky prompts to use at start of every new chat. + group = nil, -- Default group of tools or array of groups to use (can be specified manually in prompt via @). + sticky = nil, -- Default sticky prompt or array of sticky prompts to use at start of every new chat (can be specified manually in prompt via >). temperature = 0.1, -- GPT result temperature headless = false, -- Do not write to chat buffer and use history (useful for using custom processing) stream = nil, -- Function called when receiving stream updates (returned string is appended to the chat buffer) callback = nil, -- Function called when full response is received (retuned string is stored to history) - remember_as_sticky = true, -- Remember model/agent/context as sticky prompts when asking questions + remember_as_sticky = true, -- Remember model as sticky prompts when asking questions -- default selection -- see select.lua for implementation @@ -520,27 +507,31 @@ Below are all available configuration options with their default values: }, }, - -- default contexts - -- see config/contexts.lua for implementation - contexts = { + -- default tools + -- see config/tools.lua for implementation + tools = { buffer = { }, buffers = { }, file = { }, - files = { + glob = { }, - git = { + grep = { + }, + quickfix = { + }, + diagnostics = { + }, + gitdiff = { + }, + gitstatus = { }, url = { }, register = { }, - quickfix = { - }, - system = { - } }, -- default prompts @@ -568,7 +559,7 @@ Below are all available configuration options with their default values: }, Commit = { prompt = 'Write commit message for the change with commitizen convention. Keep the title under 50 characters and wrap message at 72 characters. Format as a gitcommit code block.', - context = 'git:staged', + sticky = '#git:staged', }, }, @@ -618,9 +609,6 @@ Below are all available configuration options with their default values: full_diff = false, -- Show full diff instead of unified diff when showing diff window }, show_info = { - normal = 'gi', - }, - show_context = { normal = 'gc', }, show_help = { @@ -660,8 +648,8 @@ Types of copilot highlights: - `CopilotChatStatus` - Status and spinner in chat buffer - `CopilotChatHelp` - Help messages in chat buffer (help, references) - `CopilotChatSelection` - Selection highlight in source buffer -- `CopilotChatKeyword` - Keyword highlight in chat buffer (e.g. prompts, contexts) -- `CopilotChatInput` - Input highlight in chat buffer (for contexts) +- `CopilotChatKeyword` - Keyword highlight in chat buffer (e.g. prompts, tools) +- `CopilotChatAnnotation` - Annotation highlight in chat buffer (file headers, tool call headers, tool call body) # API Reference @@ -674,8 +662,7 @@ local chat = require("CopilotChat") chat.ask(prompt, config) -- Ask a question with optional config chat.response() -- Get the last response text chat.resolve_prompt() -- Resolve prompt references -chat.resolve_context() -- Resolve context embeddings (WARN: async, requires plenary.async.run) -chat.resolve_agent() -- Resolve agent from prompt (WARN: async, requires plenary.async.run) +chat.resolve_tools() -- Resolve tools that are available for automatic use by LLM chat.resolve_model() -- Resolve model from prompt (WARN: async, requires plenary.async.run) -- Window Management @@ -693,10 +680,9 @@ chat.set_source(winnr) -- Set the source window chat.get_selection() -- Get the current selection chat.set_selection(bufnr, start_line, end_line, clear) -- Set or clear selection --- Prompt & Context Management +-- Prompt & Model Management chat.select_prompt(config) -- Open prompt selector with optional config chat.select_model() -- Open model selector -chat.select_agent() -- Open agent selector chat.prompts() -- Get all available prompts -- Completion @@ -747,22 +733,21 @@ window:overlay(opts) -- Show overlay with specified options ```lua -- Open chat, ask a question and handle response require("CopilotChat").open() -require("CopilotChat").ask("Explain this code", { +require("CopilotChat").ask("#buffer Explain this code", { callback = function(response) vim.notify("Got response: " .. response:sub(1, 50) .. "...") return response end, - context = "buffer" }) -- Save and load chat history require("CopilotChat").save("my_debugging_session") require("CopilotChat").load("my_debugging_session") --- Use custom context and model +-- Use custom sticky and model require("CopilotChat").ask("How can I optimize this?", { model = "gpt-4o", - context = {"buffer", "git:staged"} + sticky = {"#buffer", "#git:staged"} }) ``` diff --git a/lua/CopilotChat/actions.lua b/lua/CopilotChat/actions.lua deleted file mode 100644 index 2ab795b7..00000000 --- a/lua/CopilotChat/actions.lua +++ /dev/null @@ -1,49 +0,0 @@ ----@class CopilotChat.integrations.actions ----@field prompt string: The prompt to display ----@field actions table: A table with the actions to pick from - -local chat = require('CopilotChat') - -local M = {} - ---- User prompt actions ----@param config CopilotChat.config.shared?: The chat configuration ----@return CopilotChat.integrations.actions?: The prompt actions ----@deprecated Use |CopilotChat.select_prompt| instead -function M.prompt_actions(config) - local actions = {} - for name, prompt in pairs(chat.prompts()) do - if prompt.prompt then - actions[name] = vim.tbl_extend('keep', prompt, config or {}) - end - end - return { - prompt = 'Copilot Chat Prompt Actions', - actions = actions, - } -end - ---- Pick an action from a list of actions ----@param pick_actions CopilotChat.integrations.actions?: A table with the actions to pick from ----@param opts table?: vim.ui.select options ----@deprecated Use |CopilotChat.select_prompt| instead -function M.pick(pick_actions, opts) - if not pick_actions or not pick_actions.actions or vim.tbl_isempty(pick_actions.actions) then - return - end - - opts = vim.tbl_extend('force', { - prompt = pick_actions.prompt .. '> ', - }, opts or {}) - - vim.ui.select(vim.tbl_keys(pick_actions.actions), opts, function(selected) - if not selected then - return - end - vim.defer_fn(function() - chat.ask(pick_actions.actions[selected].prompt, pick_actions.actions[selected]) - end, 100) - end) -end - -return M diff --git a/lua/CopilotChat/client.lua b/lua/CopilotChat/client.lua index dddd1b89..4259af68 100644 --- a/lua/CopilotChat/client.lua +++ b/lua/CopilotChat/client.lua @@ -1,19 +1,59 @@ ----@class CopilotChat.Client.ask +---@class CopilotChat.client.AskOptions ---@field headless boolean ----@field contexts table? ----@field selection CopilotChat.select.selection? ----@field embeddings table? +---@field selection CopilotChat.select.Selection? +---@field tools table? +---@field resources table? ---@field system_prompt string ---@field model string ----@field agent string? ---@field temperature number ---@field on_progress? fun(response: string):nil ----@class CopilotChat.Client.model : CopilotChat.Provider.model ----@field provider string - ----@class CopilotChat.Client.agent : CopilotChat.Provider.agent ----@field provider string +---@class CopilotChat.client.AskResponse +---@field content string +---@field token_count number +---@field token_max_count number +---@field tool_calls table? + +---@class CopilotChat.client.Message +---@field role string +---@field content string +---@field tool_calls table? + +---@class CopilotChat.client.Reference +---@field name string +---@field url string + +---@class CopilotChat.client.ToolCall +---@field id number +---@field index number +---@field name string +---@field arguments string + +---@class CopilotChat.client.Tool +---@field name string name of the tool +---@field description string description of the tool +---@field schema table? schema of the tool + +---@class CopilotChat.client.Embed +---@field index number +---@field embedding table + +---@class CopilotChat.client.Resource +---@field name string +---@field type string +---@field data string + +---@class CopilotChat.client.EmbeddedResource : CopilotChat.client.Resource, CopilotChat.client.Embed + +---@class CopilotChat.client.Model +---@field provider string? +---@field id string +---@field name string +---@field tokenizer string? +---@field max_input_tokens number? +---@field max_output_tokens number? +---@field streaming boolean? +---@field tools boolean? local log = require('plenary.log') local tiktoken = require('CopilotChat.tiktoken') @@ -22,7 +62,7 @@ local utils = require('CopilotChat.utils') local class = utils.class --- Constants -local CONTEXT_FORMAT = '[#file:%s](#file:%s-context)' +local RESOURCE_FORMAT = '# %s\n```%s\n%s\n```' local LINE_CHARACTERS = 100 local BIG_FILE_THRESHOLD = 1000 * LINE_CHARACTERS local BIG_EMBED_THRESHOLD = 200 * LINE_CHARACTERS @@ -30,8 +70,8 @@ local TRUNCATED = '... (truncated)' --- Resolve provider function ---@param model string ----@param models table ----@param providers table +---@param models table +---@param providers table ---@return string, function local function resolve_provider_function(name, model, models, providers) local model_config = models[model] @@ -65,23 +105,18 @@ local function resolve_provider_function(name, model, models, providers) end --- Generate content block with line numbers, truncating if necessary ----@param content string: The content ----@param outline string?: The outline +---@param content string ---@param threshold number: The threshold for truncation ----@param start_line number|nil: The starting line number +---@param start_line number?: The starting line number ---@return string -local function generate_content_block(content, outline, threshold, start_line) +local function generate_content_block(content, threshold, start_line) local total_chars = #content - if total_chars > threshold and outline then - content = outline - total_chars = #content - end if total_chars > threshold then content = content:sub(1, threshold) content = content .. '\n' .. TRUNCATED end - if start_line ~= -1 then + if start_line ~= nil then local lines = vim.split(content, '\n') local total_lines = #lines local max_length = #tostring(total_lines) @@ -96,44 +131,19 @@ local function generate_content_block(content, outline, threshold, start_line) return content end ---- Generate diagnostics message ----@param diagnostics table ----@return string -local function generate_diagnostics(diagnostics) - local out = {} - for _, diagnostic in ipairs(diagnostics) do - table.insert( - out, - string.format( - '%s line=%d-%d: %s', - diagnostic.severity, - diagnostic.start_line, - diagnostic.end_line, - diagnostic.content - ) - ) - end - return table.concat(out, '\n') -end - --- Generate messages for the given selection ---- @param selection CopilotChat.select.selection? ---- @return table -local function generate_selection_messages(selection) - if not selection then - return {} - end - +--- @param selection CopilotChat.select.Selection +--- @return CopilotChat.client.Message? +local function generate_selection_message(selection) local filename = selection.filename or 'unknown' local filetype = selection.filetype or 'text' local content = selection.content if not content or content == '' then - return {} + return nil end - local out = string.format('# FILE:%s CONTEXT\n', filename:upper()) - out = out .. "User's active selection:\n" + local out = "User's active selection:\n" if selection.start_line and selection.end_line then out = out .. string.format('Excerpt from %s, lines %s to %s:\n', filename, selection.start_line, selection.end_line) end @@ -141,103 +151,45 @@ local function generate_selection_messages(selection) .. string.format( '```%s\n%s\n```', filetype, - generate_content_block(content, nil, BIG_FILE_THRESHOLD, selection.start_line) + generate_content_block(content, BIG_FILE_THRESHOLD, selection.start_line) ) - if selection.diagnostics then - out = out - .. string.format("\nDiagnostics in user's active selection:\n%s", generate_diagnostics(selection.diagnostics)) - end - return { - { - name = filename, - context = string.format(CONTEXT_FORMAT, filename, filename), - content = out, - role = 'user', - }, + content = out, + role = 'user', } end ---- Generate messages for the given embeddings ---- @param embeddings table? ---- @return table -local function generate_embeddings_messages(embeddings) - if not embeddings then - return {} - end - - return vim.tbl_map(function(embedding) - local out = string.format( - '# FILE:%s CONTEXT\n```%s\n%s\n```', - embedding.filename:upper(), - embedding.filetype or 'text', - generate_content_block(embedding.content, embedding.outline, BIG_FILE_THRESHOLD) - ) - - if embedding.diagnostics then - out = out - .. string.format( - '\nFILE:%s DIAGNOSTICS:\n%s', - embedding.filename:upper(), - generate_diagnostics(embedding.diagnostics) - ) - end - - return { - name = embedding.filename, - context = string.format(CONTEXT_FORMAT, embedding.filename, embedding.filename), - content = out, - role = 'user', - } - end, embeddings) +--- Generate messages for the given resources +--- @param resources CopilotChat.client.Resource[] +--- @return table +local function generate_resource_messages(resources) + return vim + .iter(resources or {}) + :filter(function(resource) + return resource.data and resource.data ~= '' + end) + :map(function(resource) + local content = generate_content_block(resource.data, BIG_FILE_THRESHOLD, 1) + + return { + content = string.format(RESOURCE_FORMAT, resource.name, resource.type, content), + role = 'user', + } + end) + :totable() end --- Generate ask request ---- @param history table ---- @param contexts table? --- @param prompt string --- @param system_prompt string ---- @param generated_messages table -local function generate_ask_request(history, contexts, prompt, system_prompt, generated_messages) +--- @param history table +--- @param generated_messages table +local function generate_ask_request(prompt, system_prompt, history, generated_messages) local messages = {} system_prompt = vim.trim(system_prompt) - -- Include context help - if contexts and not vim.tbl_isempty(contexts) then - local help_text = [[When you need additional context, request it using this format: - -> #:`` - -Examples: -> #file:`path/to/file.js` (loads specific file) -> #buffers:`visible` (loads all visible buffers) -> #git:`staged` (loads git staged changes) -> #system:`uname -a` (loads system information) - -Guidelines: -- Always request context when needed rather than guessing about files or code -- Use the > format on a new line when requesting context -- Output context commands directly - never ask if the user wants to provide information -- Assume the user will provide requested context in their next response - -Available context providers and their usage:]] - - local context_names = vim.tbl_keys(contexts) - table.sort(context_names) - for _, name in ipairs(context_names) do - local description = contexts[name] - description = description:gsub('\n', '\n ') - help_text = help_text .. '\n\n - #' .. name .. ': ' .. description - end - - if system_prompt ~= '' then - system_prompt = system_prompt .. '\n\n' - end - system_prompt = system_prompt .. help_text - end - -- Include system prompt if not utils.empty(system_prompt) then table.insert(messages, { @@ -246,30 +198,15 @@ Available context providers and their usage:]] }) end - local context_references = {} - - -- Include embeddings and history + -- Include generated messages and history + for _, message in ipairs(history) do + table.insert(messages, message) + end for _, message in ipairs(generated_messages) do table.insert(messages, { content = message.content, role = message.role, }) - - if message.context then - context_references[message.context] = true - end - end - for _, message in ipairs(history) do - table.insert(messages, message) - end - - -- Include context references - prompt = vim.trim(prompt) - if not vim.tbl_isempty(context_references) then - if prompt ~= '' then - prompt = '\n\n' .. prompt - end - prompt = table.concat(vim.tbl_keys(context_references), '\n') .. prompt end -- Include user prompt @@ -286,34 +223,28 @@ Available context providers and their usage:]] end --- Generate embedding request ---- @param inputs table +--- @param inputs table --- @param threshold number --- @return table local function generate_embedding_request(inputs, threshold) return vim.tbl_map(function(embedding) - local content = generate_content_block(embedding.outline or embedding.content, nil, threshold, -1) - if embedding.filetype == 'raw' then - return content - else - return string.format('File: `%s`\n```%s\n%s\n```', embedding.filename, embedding.filetype, content) - end + local content = generate_content_block(embedding.data, threshold) + return string.format(RESOURCE_FORMAT, embedding.name, embedding.type, content) end, inputs) end ----@class CopilotChat.Client : Class ----@field history table ----@field providers table ----@field provider_cache table ----@field models table? ----@field agents table? ----@field current_job string? ----@field headers table? +---@class CopilotChat.client.Client : Class +---@field history table +---@field private providers table +---@field private provider_cache table +---@field private models table? +---@field private current_job string? +---@field private headers table? local Client = class(function(self) self.history = {} self.providers = {} self.provider_cache = {} self.models = nil - self.agents = nil self.current_job = nil self.headers = nil end) @@ -336,7 +267,7 @@ function Client:authenticate(provider_name) end --- Fetch models from the Copilot API ----@return table +---@return table function Client:fetch_models() if self.models then return self.models @@ -377,62 +308,17 @@ function Client:fetch_models() return self.models end ---- Fetch agents from the Copilot API ----@return table -function Client:fetch_agents() - if self.agents then - return self.agents - end - - local agents = {} - local provider_order = vim.tbl_keys(self.providers) - table.sort(provider_order) - for _, provider_name in ipairs(provider_order) do - local provider = self.providers[provider_name] - if not provider.disabled and provider.get_agents then - notify.publish(notify.STATUS, 'Fetching agents from ' .. provider_name) - local ok, headers = pcall(self.authenticate, self, provider_name) - if not ok then - log.warn('Failed to authenticate with ' .. provider_name .. ': ' .. headers) - goto continue - end - local ok, provider_agents = pcall(provider.get_agents, headers) - if not ok then - log.warn('Failed to fetch agents from ' .. provider_name .. ': ' .. provider_agents) - goto continue - end - - for _, agent in ipairs(provider_agents) do - agent.provider = provider_name - if agents[agent.id] then - agent.id = agent.id .. ':' .. provider_name - end - agents[agent.id] = agent - end - - ::continue:: - end - end - - self.agents = agents - return self.agents -end - --- Ask a question to Copilot ---@param prompt string: The prompt to send to Copilot ----@param opts CopilotChat.Client.ask: Options for the request ----@return string?, table?, number?, number? +---@param opts CopilotChat.client.AskOptions: Options for the request +---@return CopilotChat.client.AskResponse? function Client:ask(prompt, opts) opts = opts or {} - - if opts.agent == 'none' or opts.agent == 'copilot' then - opts.agent = nil - end - local job_id = utils.uuid() log.debug('Model:', opts.model) - log.debug('Agent:', opts.agent) + log.debug('Tools:', #opts.tools) + log.debug('Resources:', #opts.resources) local models = self:fetch_models() local model_config = models[opts.model] @@ -440,12 +326,6 @@ function Client:ask(prompt, opts) error('Model not found: ' .. opts.model) end - local agents = self:fetch_agents() - local agent_config = opts.agent and agents[opts.agent] - if opts.agent and not agent_config then - error('Agent not found: ' .. opts.agent) - end - local provider_name = model_config.provider if not provider_name then error('Provider not found for model: ' .. opts.model) @@ -459,10 +339,8 @@ function Client:ask(prompt, opts) model = vim.tbl_extend('force', model_config, { id = opts.model:gsub(':' .. provider_name .. '$', ''), }), - agent = agent_config and vim.tbl_extend('force', agent_config, { - id = opts.agent and opts.agent:gsub(':' .. provider_name .. '$', ''), - }), temperature = opts.temperature, + tools = opts.tools, } local max_tokens = model_config.max_input_tokens @@ -478,36 +356,25 @@ function Client:ask(prompt, opts) end local history = not opts.headless and vim.list_slice(self.history) or {} - local references = utils.ordered_map() + local tool_calls = utils.ordered_map() local generated_messages = {} - local selection_messages = generate_selection_messages(opts.selection) - local embeddings_messages = generate_embeddings_messages(opts.embeddings) - - for _, message in ipairs(selection_messages) do - table.insert(generated_messages, message) - references:set(message.name, { - name = utils.filename(message.name), - url = message.name, - }) + local selection_message = opts.selection and generate_selection_message(opts.selection) + local resource_messages = generate_resource_messages(opts.resources) + + if selection_message then + table.insert(generated_messages, selection_message) end if max_tokens then - -- Count tokens from selection messages - local selection_tokens = 0 - for _, message in ipairs(selection_messages) do - selection_tokens = selection_tokens + tiktoken.count(message.content) - end - -- Count required tokens that we cannot reduce + local selection_tokens = selection_message and tiktoken.count(selection_message.content) or 0 local prompt_tokens = tiktoken.count(prompt) local system_tokens = tiktoken.count(opts.system_prompt) - local required_tokens = prompt_tokens + system_tokens + selection_tokens - - -- Reserve space for first embedding - local reserved_tokens = #embeddings_messages > 0 and tiktoken.count(embeddings_messages[1].content) or 0 + local resource_tokens = #resource_messages > 0 and tiktoken.count(resource_messages[1].content) or 0 + local required_tokens = prompt_tokens + system_tokens + selection_tokens + resource_tokens -- Calculate how many tokens we can use for history - local history_limit = max_tokens - required_tokens - reserved_tokens + local history_limit = max_tokens - required_tokens local history_tokens = 0 for _, msg in ipairs(history) do history_tokens = history_tokens + tiktoken.count(msg.content) @@ -521,35 +388,25 @@ function Client:ask(prompt, opts) -- Now add as many files as possible with remaining token budget local remaining_tokens = max_tokens - required_tokens - history_tokens - for _, message in ipairs(embeddings_messages) do + for _, message in ipairs(resource_messages) do local tokens = tiktoken.count(message.content) if remaining_tokens - tokens >= 0 then remaining_tokens = remaining_tokens - tokens table.insert(generated_messages, message) - references:set(message.name, { - name = utils.filename(message.name), - url = message.name, - }) else break end end else -- Add all embedding messages as we cant limit them - for _, message in ipairs(embeddings_messages) do + for _, message in ipairs(resource_messages) do table.insert(generated_messages, message) - references:set(message.name, { - name = utils.filename(message.name), - url = message.name, - }) end end - log.debug('References:', #generated_messages) - - local last_message = nil local errored = false local finished = false + local token_count = 0 local response_buffer = utils.string_buffer() local function finish_stream(err, job) @@ -589,11 +446,19 @@ function Client:ask(prompt, opts) end local out = provider.prepare_output(content, options) - last_message = out - if out.references then - for _, reference in ipairs(out.references) do - references:set(reference.name, reference) + if out.total_tokens then + token_count = out.total_tokens + end + + if out.tool_calls then + for _, tool_call in ipairs(out.tool_calls) do + local val = tool_calls:get(tool_call.index) + if not val then + tool_calls:set(tool_call.index, tool_call) + else + val.arguments = val.arguments .. tool_call.arguments + end end end @@ -606,7 +471,7 @@ function Client:ask(prompt, opts) if out.finish_reason then local reason = out.finish_reason - if reason == 'stop' then + if reason == 'stop' or reason == 'tool_calls' then reason = nil else reason = 'Early stop: ' .. reason @@ -656,10 +521,8 @@ function Client:ask(prompt, opts) end local headers = self:authenticate(provider_name) - local request = provider.prepare_input( - generate_ask_request(history, opts.contexts, prompt, opts.system_prompt, generated_messages), - options - ) + local request = + provider.prepare_input(generate_ask_request(prompt, opts.system_prompt, history, generated_messages), options) local is_stream = request.stream local args = { @@ -716,7 +579,7 @@ function Client:ask(prompt, opts) if response then if is_stream then - if utils.empty(response_text) then + if utils.empty(response_text) and not finished then for _, line in ipairs(vim.split(response.body, '\n')) do parse_stream_line(line) end @@ -727,12 +590,12 @@ function Client:ask(prompt, opts) response_text = response_buffer:tostring() end - if utils.empty(response_text) then - error('Failed to get response: empty response') - return - end - - return response_text, references:values(), last_message and last_message.total_tokens or 0, max_tokens + return { + content = response_text, + token_count = token_count, + token_max_count = max_tokens, + tool_calls = tool_calls:values(), + } end --- List available models @@ -755,40 +618,20 @@ function Client:list_models() end, result) end ---- List available agents ----@return table -function Client:list_agents() - local agents = self:fetch_agents() - local result = vim.tbl_keys(agents) - - table.sort(result, function(a, b) - a = agents[a] - b = agents[b] - if a.provider ~= b.provider then - return a.provider < b.provider - end - return a.id < b.id - end) - - local out = vim.tbl_map(function(id) - return agents[id] - end, result) - table.insert(out, 1, { id = 'none', name = 'None', description = 'No agent', provider = 'none' }) - return out -end - --- Generate embeddings for the given inputs ----@param inputs table: The inputs to embed +---@param inputs table: The inputs to embed ---@param model string ----@return table +---@return table function Client:embed(inputs, model) if not inputs or #inputs == 0 then + ---@diagnostic disable-next-line: return-type-mismatch return inputs end local models = self:fetch_models() local ok, provider_name, embed = pcall(resolve_provider_function, 'embed', model, models, self.providers) if not ok then + ---@diagnostic disable-next-line: return-type-mismatch return inputs end @@ -889,5 +732,5 @@ function Client:load_providers(providers) end end ---- @type CopilotChat.Client +--- @type CopilotChat.client.Client return Client() diff --git a/lua/CopilotChat/config.lua b/lua/CopilotChat/config.lua index dc1af205..28b87e66 100644 --- a/lua/CopilotChat/config.lua +++ b/lua/CopilotChat/config.lua @@ -2,7 +2,7 @@ local select = require('CopilotChat.select') ---@alias CopilotChat.config.Layout 'vertical'|'horizontal'|'float'|'replace' ----@class CopilotChat.config.window +---@class CopilotChat.config.Window ---@field layout? CopilotChat.config.Layout|fun():CopilotChat.config.Layout ---@field relative 'editor'|'win'|'cursor'|'mouse'? ---@field border 'none'|'single'|'double'|'rounded'|'solid'|'shadow'? @@ -14,32 +14,29 @@ local select = require('CopilotChat.select') ---@field footer string? ---@field zindex number? ----@class CopilotChat.config.shared +---@class CopilotChat.config.Shared ---@field system_prompt string? ---@field model string? ----@field agent string? ----@field context string|table|nil +---@field tools string|table|nil ---@field sticky string|table|nil ---@field temperature number? ---@field headless boolean? ---@field stream nil|fun(chunk: string, source: CopilotChat.source):string ---@field callback nil|fun(response: string, source: CopilotChat.source):string ---@field remember_as_sticky boolean? ----@field include_contexts_in_prompt boolean? ----@field selection false|nil|fun(source: CopilotChat.source):CopilotChat.select.selection? ----@field window CopilotChat.config.window? +---@field selection false|nil|fun(source: CopilotChat.source):CopilotChat.select.Selection? +---@field window CopilotChat.config.Window? ---@field show_help boolean? ---@field show_folds boolean? ---@field highlight_selection boolean? ---@field highlight_headers boolean? ----@field references_display 'virtual'|'write'? ---@field auto_follow_cursor boolean? ---@field auto_insert_mode boolean? ---@field insert_at_end boolean? ---@field clear_chat_on_new_prompt boolean? --- CopilotChat default configuration ----@class CopilotChat.config : CopilotChat.config.shared +---@class CopilotChat.config.Config : CopilotChat.config.Shared ---@field debug boolean? ---@field log_level 'trace'|'debug'|'info'|'warn'|'error'|'fatal'? ---@field proxy string? @@ -51,9 +48,9 @@ local select = require('CopilotChat.select') ---@field answer_header string? ---@field error_header string? ---@field separator string? ----@field providers table? ----@field contexts table? ----@field prompts table? +---@field providers table? +---@field functions table? +---@field prompts table? ---@field mappings CopilotChat.config.mappings? return { @@ -62,17 +59,14 @@ return { system_prompt = 'COPILOT_INSTRUCTIONS', -- System prompt to use (can be specified manually in prompt via /). model = 'gpt-4o', -- Default model to use, see ':CopilotChatModels' for available models (can be specified manually in prompt via $). - agent = 'none', -- Default agent to use, see ':CopilotChatAgents' for available agents (can be specified manually in prompt via @). - context = nil, -- Default context or array of contexts to use (can be specified manually in prompt via #). - sticky = nil, -- Default sticky prompt or array of sticky prompts to use at start of every new chat. + tools = nil, -- Default tool or array of tools (or groups) to share with LLM (can be specified manually in prompt via @). + sticky = nil, -- Default sticky prompt or array of sticky prompts to use at start of every new chat (can be specified manually in prompt via >). - temperature = 0.1, -- GPT result temperature + temperature = 0.1, -- Result temperature headless = false, -- Do not write to chat buffer and use history (useful for using custom processing) stream = nil, -- Function called when receiving stream updates (returned string is appended to the chat buffer) callback = nil, -- Function called when full response is received (retuned string is stored to history) - remember_as_sticky = true, -- Remember model/agent/context as sticky prompts when asking questions - - include_contexts_in_prompt = true, -- Include contexts in prompt + remember_as_sticky = true, -- Remember model as sticky prompts when asking questions -- default selection selection = select.visual, @@ -96,7 +90,6 @@ return { show_folds = true, -- Shows folds for sections in chat highlight_selection = true, -- Highlight selection highlight_headers = true, -- Highlight headers in chat, disable if using markdown renderers (like render-markdown.nvim) - references_display = 'virtual', -- 'virtual', 'write', Display references in chat as virtual text or write to buffer auto_follow_cursor = true, -- Auto-follow cursor in chat auto_insert_mode = false, -- Automatically enter insert mode when opening window and on new prompt insert_at_end = false, -- Move cursor to end of buffer when inserting text @@ -122,8 +115,8 @@ return { -- default providers providers = require('CopilotChat.config.providers'), - -- default contexts - contexts = require('CopilotChat.config.contexts'), + -- default functions + functions = require('CopilotChat.config.functions'), -- default prompts prompts = require('CopilotChat.config.prompts'), diff --git a/lua/CopilotChat/config/contexts.lua b/lua/CopilotChat/config/contexts.lua deleted file mode 100644 index 64758f76..00000000 --- a/lua/CopilotChat/config/contexts.lua +++ /dev/null @@ -1,352 +0,0 @@ -local context = require('CopilotChat.context') -local utils = require('CopilotChat.utils') - ----@class CopilotChat.config.context ----@field description string? ----@field input fun(callback: fun(input: string?), source: CopilotChat.source)? ----@field resolve fun(input: string?, source: CopilotChat.source, prompt: string):table - ----@type table -return { - buffer = { - description = 'Includes specified buffer in chat context. Supports input (default current).', - input = function(callback) - vim.ui.select( - vim.tbl_map( - function(buf) - return { id = buf, name = vim.fn.fnamemodify(vim.api.nvim_buf_get_name(buf), ':p:.') } - end, - vim.tbl_filter(function(buf) - return utils.buf_valid(buf) and vim.fn.buflisted(buf) == 1 - end, vim.api.nvim_list_bufs()) - ), - { - prompt = 'Select a buffer> ', - format_item = function(item) - return item.name - end, - }, - function(choice) - callback(choice and choice.id) - end - ) - end, - resolve = function(input, source) - input = input and tonumber(input) or source.bufnr - - utils.schedule_main() - return { - context.get_buffer(input), - } - end, - }, - - buffers = { - description = 'Includes all buffers in chat context. Supports input (default listed).', - input = function(callback) - vim.ui.select({ 'listed', 'visible' }, { - prompt = 'Select buffer scope> ', - }, callback) - end, - resolve = function(input) - input = input or 'listed' - - utils.schedule_main() - return vim.tbl_map( - context.get_buffer, - vim.tbl_filter(function(b) - return utils.buf_valid(b) and vim.fn.buflisted(b) == 1 and (input == 'listed' or #vim.fn.win_findbuf(b) > 0) - end, vim.api.nvim_list_bufs()) - ) - end, - }, - - file = { - description = 'Includes content of provided file in chat context. Supports input.', - input = function(callback, source) - local files = utils.scan_dir(source.cwd(), { - max_count = 0, - }) - - utils.schedule_main() - vim.ui.select(files, { - prompt = 'Select a file> ', - }, callback) - end, - resolve = function(input) - if not input or input == '' then - return {} - end - - utils.schedule_main() - return { - context.get_file(utils.filepath(input), utils.filetype(input)), - } - end, - }, - - files = { - description = 'Includes all non-hidden files in the current workspace in chat context. Supports input (glob pattern).', - input = function(callback) - vim.ui.input({ - prompt = 'Enter glob> ', - }, callback) - end, - resolve = function(input, source) - local files = utils.scan_dir(source.cwd(), { - glob = input, - }) - - utils.schedule_main() - files = vim.tbl_filter( - function(file) - return file.ft ~= nil - end, - vim.tbl_map(function(file) - return { - name = utils.filepath(file), - ft = utils.filetype(file), - } - end, files) - ) - - return vim - .iter(files) - :map(function(file) - return context.get_file(file.name, file.ft) - end) - :filter(function(file_data) - return file_data ~= nil - end) - :totable() - end, - }, - - filenames = { - description = 'Includes names of all non-hidden files in the current workspace in chat context. Supports input (glob pattern).', - input = function(callback) - vim.ui.input({ - prompt = 'Enter glob> ', - }, callback) - end, - resolve = function(input, source) - local out = {} - local files = utils.scan_dir(source.cwd(), { - glob = input, - }) - - local chunk_size = 100 - for i = 1, #files, chunk_size do - local chunk = {} - for j = i, math.min(i + chunk_size - 1, #files) do - table.insert(chunk, files[j]) - end - - local chunk_number = math.floor(i / chunk_size) - local chunk_name = chunk_number == 0 and 'file_map' or 'file_map' .. tostring(chunk_number) - - table.insert(out, { - content = table.concat(chunk, '\n'), - filename = chunk_name, - filetype = 'text', - score = 0.1, - }) - end - - return out - end, - }, - - git = { - description = 'Requires `git`. Includes current git diff in chat context. Supports input (default unstaged, also accepts commit number).', - input = function(callback) - vim.ui.select({ 'unstaged', 'staged' }, { - prompt = 'Select diff type> ', - }, callback) - end, - resolve = function(input, source) - input = input or 'unstaged' - local cmd = { - 'git', - '-C', - source.cwd(), - 'diff', - '--no-color', - '--no-ext-diff', - } - - if input == 'staged' then - table.insert(cmd, '--staged') - elseif input == 'unstaged' then - table.insert(cmd, '--') - else - table.insert(cmd, input) - end - - local out = utils.system(cmd) - - return { - { - content = out.stdout, - filename = 'git_diff_' .. input, - filetype = 'diff', - }, - } - end, - }, - - url = { - description = 'Includes content of provided URL in chat context. Supports input.', - input = function(callback) - vim.ui.input({ - prompt = 'Enter URL> ', - default = 'https://', - }, callback) - end, - resolve = function(input) - return { - context.get_url(input), - } - end, - }, - - register = { - description = 'Includes contents of register in chat context. Supports input (default +, e.g clipboard).', - input = function(callback) - local choices = utils.kv_list({ - ['+'] = 'synchronized with the system clipboard', - ['*'] = 'synchronized with the selection clipboard', - ['"'] = 'last deleted, changed, or yanked content', - ['0'] = 'last yank', - ['-'] = 'deleted or changed content smaller than one line', - ['.'] = 'last inserted text', - ['%'] = 'name of the current file', - [':'] = 'most recent executed command', - ['#'] = 'alternate buffer', - ['='] = 'result of an expression', - ['/'] = 'last search pattern', - }) - - vim.ui.select(choices, { - prompt = 'Select a register> ', - format_item = function(choice) - return choice.key .. ' - ' .. choice.value - end, - }, function(choice) - callback(choice and choice.key) - end) - end, - resolve = function(input) - input = input or '+' - - utils.schedule_main() - local lines = vim.fn.getreg(input) - if not lines or lines == '' then - return {} - end - - return { - { - content = lines, - filename = 'vim_register_' .. input, - filetype = '', - }, - } - end, - }, - - quickfix = { - description = 'Includes quickfix list file contents in chat context.', - resolve = function() - utils.schedule_main() - - local items = vim.fn.getqflist() - if not items or #items == 0 then - return {} - end - - local unique_files = {} - for _, item in ipairs(items) do - local filename = item.filename or vim.api.nvim_buf_get_name(item.bufnr) - if filename then - unique_files[filename] = true - end - end - - local files = vim.tbl_filter( - function(file) - return file.ft ~= nil - end, - vim.tbl_map(function(file) - return { - name = utils.filepath(file), - ft = utils.filetype(file), - } - end, vim.tbl_keys(unique_files)) - ) - - return vim - .iter(files) - :map(function(file) - return context.get_file(file.name, file.ft) - end) - :filter(function(file_data) - return file_data ~= nil - end) - :totable() - end, - }, - - system = { - description = [[Includes output of provided system shell command in chat context. Supports input. - -Important: -- Only use system commands as last resort, they are run every time the context is requested. -- For example instead of curl use the url context, instead of finding and grepping try to check if there is any context that can query the data you need instead. -- If you absolutely need to run a system command, try to use read-only commands and avoid commands that modify the system state. -]], - input = function(callback) - vim.ui.input({ - prompt = 'Enter command> ', - }, callback) - end, - resolve = function(input) - if not input or input == '' then - return {} - end - - utils.schedule_main() - - local shell, shell_flag - if vim.fn.has('win32') == 1 then - shell, shell_flag = 'cmd.exe', '/c' - else - shell, shell_flag = 'sh', '-c' - end - - local out = utils.system({ shell, shell_flag, input }) - if not out then - return {} - end - - local out_type = 'command_output' - local out_text = out.stdout - if out.code ~= 0 then - out_type = 'command_error' - if out.stderr and out.stderr ~= '' then - out_text = out.stderr - elseif not out_text or out_text == '' then - out_text = 'Command failed with exit code ' .. out.code - end - end - - return { - { - content = out_text, - filename = out_type .. '_' .. input:gsub('[^%w]', '_'):sub(1, 20), - filetype = 'text', - }, - } - end, - }, -} diff --git a/lua/CopilotChat/config/functions.lua b/lua/CopilotChat/config/functions.lua new file mode 100644 index 00000000..df4829f3 --- /dev/null +++ b/lua/CopilotChat/config/functions.lua @@ -0,0 +1,503 @@ +local resources = require('CopilotChat.resources') +local utils = require('CopilotChat.utils') + +---@class CopilotChat.config.functions.Result +---@field data string +---@field mimetype string? +---@field uri string? + +---@class CopilotChat.config.functions.Function +---@field description string? +---@field schema table? +---@field group string? +---@field uri string? +---@field resolve fun(input: table, source: CopilotChat.source, prompt: string):table + +---@type table +return { + file = { + group = 'copilot', + uri = 'file://{path}', + description = 'Reads content from a specified file path, even if the file is not currently loaded as a buffer.', + + schema = { + type = 'object', + required = { 'path' }, + properties = { + path = { + type = 'string', + description = 'Path to file to include in chat context.', + enum = function(source) + return utils.glob(source.cwd(), { + max_count = 0, + }) + end, + }, + }, + }, + + resolve = function(input) + local data, mimetype = resources.get_file(input.path) + if not data then + error('File not found: ' .. input.path) + end + + return { + { + uri = 'file://' .. input.path, + mimetype = mimetype, + data = data, + }, + } + end, + }, + + glob = { + group = 'copilot', + uri = 'files://glob/{pattern}', + description = 'Lists filenames matching a pattern in your workspace. Useful for discovering relevant files or understanding the project structure.', + + schema = { + type = 'object', + required = { 'pattern' }, + properties = { + pattern = { + type = 'string', + description = 'Glob pattern to match files.', + default = '**/*', + }, + }, + }, + + resolve = function(input, source) + local files = utils.glob(source.cwd(), { + pattern = input.pattern, + }) + + return { + { + uri = 'files://glob/' .. input.pattern, + mimetype = 'text/plain', + data = table.concat(files, '\n'), + }, + } + end, + }, + + grep = { + group = 'copilot', + uri = 'files://grep/{pattern}', + description = 'Searches for a pattern across files in your workspace. Helpful for finding specific code elements or patterns.', + + schema = { + type = 'object', + required = { 'pattern' }, + properties = { + pattern = { + type = 'string', + description = 'Pattern to search for.', + }, + }, + }, + + resolve = function(input, source) + local files = utils.grep(source.cwd(), { + pattern = input.pattern, + }) + + return { + { + uri = 'files://grep/' .. input.pattern, + mimetype = 'text/plain', + data = table.concat(files, '\n'), + }, + } + end, + }, + + buffer = { + group = 'copilot', + uri = 'neovim://buffer/{name}', + description = 'Retrieves content from a specific buffer. Useful for discussing or analyzing code from a particular file that is currently loaded.', + + schema = { + type = 'object', + required = { 'name' }, + properties = { + name = { + type = 'string', + description = 'Buffer filename to include in chat context.', + enum = function() + return vim + .iter(vim.api.nvim_list_bufs()) + :filter(function(buf) + return buf and utils.buf_valid(buf) and vim.fn.buflisted(buf) == 1 + end) + :map(function(buf) + return vim.api.nvim_buf_get_name(buf) + end) + :totable() + end, + }, + }, + }, + + resolve = function(input, source) + utils.schedule_main() + local name = input.name or vim.api.nvim_buf_get_name(source.bufnr) + local found_buf = nil + for _, buf in ipairs(vim.api.nvim_list_bufs()) do + if vim.api.nvim_buf_get_name(buf) == name then + found_buf = buf + break + end + end + if not found_buf then + error('Buffer not found: ' .. name) + end + local data, mimetype = resources.get_buffer(found_buf) + if not data then + error('Buffer not found: ' .. name) + end + return { + { + uri = 'file://' .. name, + mimetype = mimetype, + data = data, + }, + } + end, + }, + + buffers = { + group = 'copilot', + uri = 'neovim://buffers/{scope}', + description = 'Fetches content from multiple buffers. Helps with discussing or analyzing code across multiple files simultaneously.', + + schema = { + type = 'object', + required = { 'scope' }, + properties = { + scope = { + type = 'string', + description = 'Scope of buffers to include in chat context.', + enum = { 'listed', 'visible' }, + default = 'listed', + }, + }, + }, + + resolve = function(input) + utils.schedule_main() + return vim + .iter(vim.api.nvim_list_bufs()) + :filter(function(bufnr) + return utils.buf_valid(bufnr) + and vim.fn.buflisted(bufnr) == 1 + and (input.scope == 'listed' or #vim.fn.win_findbuf(bufnr) > 0) + end) + :map(function(bufnr) + local name = vim.api.nvim_buf_get_name(bufnr) + local data, mimetype = resources.get_buffer(bufnr) + if not data then + return nil + end + return { + uri = 'file://' .. name, + mimetype = mimetype, + data = data, + } + end) + :filter(function(file_data) + return file_data ~= nil + end) + :totable() + end, + }, + + quickfix = { + group = 'copilot', + uri = 'neovim://quickfix', + description = 'Includes the content of all files referenced in the current quickfix list. Useful for discussing compilation errors, search results, or other collected locations.', + + resolve = function() + utils.schedule_main() + + local items = vim.fn.getqflist() + if not items or #items == 0 then + return {} + end + + local unique_files = {} + for _, item in ipairs(items) do + local filename = item.filename or vim.api.nvim_buf_get_name(item.bufnr) + if filename then + unique_files[filename] = true + end + end + + return vim + .iter(vim.tbl_keys(unique_files)) + :map(function(file) + local data, mimetype = resources.get_file(file) + if not data then + return nil + end + return { + uri = 'file://' .. file, + mimetype = mimetype, + data = data, + } + end) + :filter(function(file_data) + return file_data ~= nil + end) + :totable() + end, + }, + + diagnostics = { + group = 'copilot', + uri = 'neovim://diagnostics/{scope}', + description = 'Collects code diagnostics (errors, warnings, etc.) from specified buffers. Helpful for troubleshooting and fixing code issues.', + + schema = { + type = 'object', + required = { 'scope' }, + properties = { + scope = { + type = 'string', + description = 'Scope of buffers to use for retrieving diagnostics.', + enum = { 'current', 'listed', 'visible' }, + default = 'current', + }, + severity = { + type = 'string', + description = 'Minimum severity level of diagnostics to include.', + enum = { 'error', 'warn', 'info', 'hint' }, + default = 'warn', + }, + }, + }, + + resolve = function(input, source) + utils.schedule_main() + local out = {} + local scope = input.scope or 'current' + local buffers = {} + + -- Get buffers based on scope + if scope == 'current' then + if source and source.bufnr and utils.buf_valid(source.bufnr) then + buffers = { source.bufnr } + end + elseif scope == 'listed' then + buffers = vim.tbl_filter(function(b) + return utils.buf_valid(b) and vim.fn.buflisted(b) == 1 + end, vim.api.nvim_list_bufs()) + elseif scope == 'visible' then + buffers = vim.tbl_filter(function(b) + return utils.buf_valid(b) and vim.fn.buflisted(b) == 1 and #vim.fn.win_findbuf(b) > 0 + end, vim.api.nvim_list_bufs()) + else + buffers = vim.tbl_filter(function(b) + return utils.buf_valid(b) and vim.api.nvim_buf_get_name(b) == input.scope + end, vim.api.nvim_list_bufs()) + end + + -- Collect diagnostics for each buffer + for _, bufnr in ipairs(buffers) do + local name = vim.api.nvim_buf_get_name(bufnr) + local diagnostics = vim.diagnostic.get(bufnr, { + severity = { + min = vim.diagnostic.severity[input.severity:upper()], + }, + }) + + if #diagnostics > 0 then + local diag_lines = {} + for _, diag in ipairs(diagnostics) do + local severity = vim.diagnostic.severity[diag.severity] or 'UNKNOWN' + local line_text = vim.api.nvim_buf_get_lines(bufnr, diag.lnum, diag.lnum + 1, false)[1] or '' + + table.insert( + diag_lines, + string.format( + '%s line=%d-%d: %s\n > %s', + severity, + diag.lnum + 1, + diag.end_lnum and (diag.end_lnum + 1) or (diag.lnum + 1), + diag.message, + line_text + ) + ) + end + + table.insert(out, { + uri = 'neovim://diagnostics/' .. name, + mimetype = 'text/plain', + data = table.concat(diag_lines, '\n'), + }) + end + end + + return out + end, + }, + + register = { + group = 'copilot', + uri = 'neovim://register/{register}', + description = 'Provides access to the content of a specified Vim register. Useful for discussing yanked text, clipboard content, or previously executed commands.', + + schema = { + type = 'object', + required = { 'register' }, + properties = { + register = { + type = 'string', + description = 'Register to include in chat context.', + enum = { + '+', + '*', + '"', + '0', + '-', + '.', + '%', + ':', + '#', + '=', + '/', + }, + default = '+', + }, + }, + }, + + resolve = function(input) + utils.schedule_main() + local lines = vim.fn.getreg(input.register) + if not lines or lines == '' then + return {} + end + + return { + { + uri = 'neovim://register/' .. input.register, + mimetype = 'text/plain', + data = lines, + }, + } + end, + }, + + gitdiff = { + group = 'copilot', + uri = 'git://diff/{target}', + description = 'Retrieves git diff information. Requires git to be installed. Useful for discussing code changes or explaining the purpose of modifications.', + + schema = { + type = 'object', + required = { 'target' }, + properties = { + target = { + type = 'string', + description = 'Target to diff against.', + enum = { 'unstaged', 'staged', '' }, + default = 'unstaged', + }, + }, + }, + + resolve = function(input, source) + local cmd = { + 'git', + '-C', + source.cwd(), + 'diff', + '--no-color', + '--no-ext-diff', + } + + if input.target == 'staged' then + table.insert(cmd, '--staged') + elseif input.target == 'unstaged' then + table.insert(cmd, '--') + else + table.insert(cmd, input.target) + end + + local out = utils.system(cmd) + + return { + { + uri = 'git://diff/' .. input.target, + mimetype = 'text/plain', + data = out.stdout, + }, + } + end, + }, + + gitstatus = { + group = 'copilot', + uri = 'git://status', + description = 'Retrieves the status of the current git repository. Useful for discussing changes, commits, and other git-related tasks.', + + resolve = function(_, source) + local cmd = { + 'git', + '-C', + source.cwd(), + 'status', + } + + local out = utils.system(cmd) + + return { + { + uri = 'git://status', + mimetype = 'text/plain', + data = out.stdout, + }, + } + end, + }, + + url = { + group = 'copilot', + uri = 'https://{url}', + description = 'Fetches content from a specified URL. Useful for referencing documentation, examples, or other online resources.', + + schema = { + type = 'object', + required = { 'url' }, + properties = { + url = { + type = 'string', + description = 'URL to include in chat context.', + }, + }, + }, + + resolve = function(input) + if not input.url:match('^https?://') then + input.url = 'https://' .. input.url + end + + local data, mimetype = resources.get_url(input.url) + if not data then + error('URL not found: ' .. input.url) + end + + return { + { + uri = input.url, + mimetype = mimetype, + data = data, + }, + } + end, + }, +} diff --git a/lua/CopilotChat/config/mappings.lua b/lua/CopilotChat/config/mappings.lua index c7a257b5..1927c267 100644 --- a/lua/CopilotChat/config/mappings.lua +++ b/lua/CopilotChat/config/mappings.lua @@ -1,9 +1,8 @@ local async = require('plenary.async') local copilot = require('CopilotChat') -local client = require('CopilotChat.client') local utils = require('CopilotChat.utils') ----@class CopilotChat.config.mappings.diff +---@class CopilotChat.config.mappings.Diff ---@field change string ---@field reference string ---@field filename string @@ -13,8 +12,8 @@ local utils = require('CopilotChat.utils') ---@field bufnr number? --- Get diff data from a block ----@param block CopilotChat.ui.Chat.Section.Block? ----@return CopilotChat.config.mappings.diff? +---@param block CopilotChat.ui.chat.Block? +---@return CopilotChat.config.mappings.Diff? local function get_diff(block) -- If no block found, return nil if not block then @@ -44,7 +43,7 @@ local function get_diff(block) end filename = header.filename - filetype = header.filetype or vim.filetype.match({ filename = filename }) + filetype = header.filetype or utils.filetype(filename) start_line = header.start_line end_line = header.end_line @@ -64,7 +63,7 @@ local function get_diff(block) change = block.content, reference = reference or '', filetype = filetype or '', - filename = utils.filename(filename), + filename = filename, start_line = start_line, end_line = end_line, bufnr = bufnr, @@ -72,9 +71,9 @@ local function get_diff(block) end --- Prepare a buffer for applying a diff ----@param diff CopilotChat.config.mappings.diff? +---@param diff CopilotChat.config.mappings.Diff? ---@param source CopilotChat.source? ----@return CopilotChat.config.mappings.diff? +---@return CopilotChat.config.mappings.Diff? local function prepare_diff_buffer(diff, source) if not diff then return diff @@ -422,7 +421,7 @@ return { }, show_info = { - normal = 'gi', + normal = 'gc', callback = function(source) local section = copilot.chat:get_closest_section('question') if not section or section.answer then @@ -434,8 +433,11 @@ return { local system_prompt = config.system_prompt async.run(function() - local selected_agent = copilot.resolve_agent(prompt, config) local selected_model = copilot.resolve_model(prompt, config) + local selected_tools, selected_resources = copilot.resolve_tools(prompt, config) + selected_tools = vim.tbl_map(function(tool) + return tool.name + end, selected_tools) utils.schedule_main() table.insert(lines, '**Logs**: `' .. copilot.config.log_path .. '`') @@ -454,82 +456,55 @@ return { table.insert(lines, '') end - if selected_agent then - table.insert(lines, '**Agent**: `' .. selected_agent .. '`') + if not utils.empty(selected_tools) then + table.insert(lines, '**Tools**') + table.insert(lines, '```') + table.insert(lines, table.concat(selected_tools, ', ')) + table.insert(lines, '```') table.insert(lines, '') end if system_prompt then table.insert(lines, '**System Prompt**') - table.insert(lines, '```') + table.insert(lines, '````') for _, line in ipairs(vim.split(vim.trim(system_prompt), '\n')) do table.insert(lines, line) end - table.insert(lines, '```') + table.insert(lines, '````') table.insert(lines, '') end - if client.memory then - table.insert(lines, '**Memory**') - table.insert(lines, '```markdown') - for _, line in ipairs(vim.split(client.memory.content, '\n')) do + local selection = copilot.get_selection() + if selection then + table.insert(lines, '**Selection**') + table.insert(lines, '') + table.insert( + lines, + string.format('**%s** (%s-%s)', selection.filename, selection.start_line, selection.end_line) + ) + table.insert(lines, string.format('````%s', selection.filetype)) + for _, line in ipairs(vim.split(selection.content, '\n')) do table.insert(lines, line) end - table.insert(lines, '```') + table.insert(lines, '````') table.insert(lines, '') end - if not utils.empty(client.history) then - table.insert(lines, ('**History** (#%s, truncated)'):format(#client.history)) + if not utils.empty(selected_resources) then + table.insert(lines, '**Resources**') table.insert(lines, '') - - for _, message in ipairs(client.history) do - table.insert(lines, '**' .. message.role .. '**') - table.insert(lines, '`' .. vim.split(message.content, '\n')[1] .. '`') - end end - copilot.chat:overlay({ - text = vim.trim(table.concat(lines, '\n')) .. '\n', - }) - end) - end, - }, - - show_context = { - normal = 'gc', - callback = function() - local section = copilot.chat:get_closest_section('question') - if not section or section.answer then - return - end - - local lines = {} - - local selection = copilot.get_selection() - if selection then - table.insert(lines, '**Selection**') - table.insert(lines, '```' .. selection.filetype) - for _, line in ipairs(vim.split(selection.content, '\n')) do - table.insert(lines, line) - end - table.insert(lines, '```') - table.insert(lines, '') - end - - async.run(function() - local embeddings = copilot.resolve_context(section.content) - - for _, embedding in ipairs(embeddings) do - local embed_lines = vim.split(embedding.content, '\n') - local preview = vim.list_slice(embed_lines, 1, math.min(10, #embed_lines)) - local header = string.format('**%s** (%s lines)', embedding.filename, #embed_lines) - if #embed_lines > 10 then + for _, resource in ipairs(selected_resources) do + local resource_lines = vim.split(resource.data, '\n') + local preview = vim.list_slice(resource_lines, 1, math.min(10, #resource_lines)) + local header = string.format('**%s** (%s lines)', resource.name, #resource_lines) + if #resource_lines > 10 then header = header .. ' (truncated)' end table.insert(lines, header) - table.insert(lines, '```' .. embedding.filetype) + table.insert(lines, '```' .. resource.type) for _, line in ipairs(preview) do table.insert(lines, line) end @@ -537,7 +512,6 @@ return { table.insert(lines, '') end - utils.schedule_main() copilot.chat:overlay({ text = vim.trim(table.concat(lines, '\n')) .. '\n', }) @@ -549,9 +523,9 @@ return { normal = 'gh', callback = function() local chat_help = '**`Special tokens`**\n' - chat_help = chat_help .. '`@` to select an agent\n' - chat_help = chat_help .. '`#` to select a context\n' - chat_help = chat_help .. '`#:` to select input for context\n' + chat_help = chat_help .. '`@` to share function\n' + chat_help = chat_help .. '`#` to add resource\n' + chat_help = chat_help .. '`#:` to add resource with input\n' chat_help = chat_help .. '`/` to select a prompt\n' chat_help = chat_help .. '`$` to select a model\n' chat_help = chat_help .. '`> ` to make a sticky prompt (copied to next prompt)\n' diff --git a/lua/CopilotChat/config/prompts.lua b/lua/CopilotChat/config/prompts.lua index 85552adf..7feeb17d 100644 --- a/lua/CopilotChat/config/prompts.lua +++ b/lua/CopilotChat/config/prompts.lua @@ -1,35 +1,77 @@ -local COPILOT_BASE = string.format( - [[ +local COPILOT_BASE = [[ When asked for your name, you must respond with "GitHub Copilot". Follow the user's requirements carefully & to the letter. -Follow Microsoft content policies. -Avoid content that violates copyrights. -If you are asked to generate content that is harmful, hateful, racist, sexist, lewd, violent, or completely irrelevant to software engineering, only respond with "Sorry, I can't assist with that." Keep your answers short and impersonal. -The user works in an IDE called Neovim which has a concept for editors with open files, integrated unit test support, an output pane that shows the output of running the code as well as an integrated terminal. -The user is working on a %s machine. Please respond with system specific commands if applicable. + +The user works in editor called Neovim which has these core concepts: +- Buffer: An in-memory text content that may be associated with a file +- Window: A viewport that displays a buffer +- Tab: A collection of windows +- Quickfix/Location lists: Lists of positions in files, often used for errors or search results +- Registers: Named storage for text and commands (like clipboard) +- Normal/Insert/Visual/Command modes: Different interaction states +- LSP (Language Server Protocol): Provides code intelligence features like completion, diagnostics, and code actions +- Treesitter: Provides syntax highlighting, code folding, and structural text editing based on syntax tree parsing +The user is working on a {OS_NAME} machine. Please respond with system specific commands if applicable. +The user is currently in workspace directory {DIR} (typically the project root). Current file paths will be relative to this directory. + + +The user will ask a question or request a task that may require analysis to answer correctly. +If you can infer the project type (languages, frameworks, libraries) from context, consider them when making changes. +For implementing features, break down the request into concepts and provide a clear solution. +Think creatively to provide complete solutions based on the information available. +Never fabricate or hallucinate file contents you haven't actually seen. + + +If tools are explicitly defined in your system context: +- Follow JSON schema precisely when using tools, including all required properties and outputting valid JSON. +- Use appropriate tools for tasks rather than asking for manual actions. +- Execute actions directly when you indicate you'll do so, without asking for permission. +- Only use tools that exist and use proper invocation procedures - no multi_tool_use.parallel. +- Before using tools to retrieve information, check if it's already available in context: + 1. Content shared via "#:" references or headers + 2. Code blocks with file path labels + 3. Other contextual sharing like selected text or conversation history +- If you don't have explicit tool definitions in your system context, assume NO tools are available and clearly state this limitation when asked. NEVER pretend to retrieve content you cannot access. + + You will receive code snippets that include line number prefixes - use these to maintain correct position references but remove them when generating output. +Always use code blocks to present code changes, even if the user doesn't ask for it. When presenting code changes: - -1. For each change, first provide a header outside code blocks with format: - [file:]() line:- - -2. Then wrap the actual code in triple backticks with the appropriate language identifier. - -3. Keep changes minimal and focused to produce short diffs. - -4. Include complete replacement code for the specified line range with: +1. For each change, use the following markdown code block format with triple backticks: + ``` path= start_line= end_line= + + ``` + + Examples: + + ```lua path=lua/CopilotChat/init.lua start_line=40 end_line=50 + local function example() + print("This is an example function.") + end + ``` + + ```python path=scripts/example.py start_line=10 end_line=15 + def example_function(): + print("This is an example function.") + ``` + + ```json path=config/settings.json start_line=5 end_line=8 + { + "setting": "value", + "enabled": true + } + ``` +2. Keep changes minimal and focused to produce short diffs. +3. Include complete replacement code for the specified line range with: - Proper indentation matching the source - All necessary lines (no eliding with comments) - No line number prefixes in the code - -5. Address any diagnostics issues when fixing code. - -6. If multiple changes are needed, present them as separate blocks with their own headers. -]], - vim.uv.os_uname().sysname -) +4. Address any diagnostics issues when fixing code. +5. If multiple changes are needed, present them as separate code blocks. + +]] local COPILOT_INSTRUCTIONS = [[ You are a code-focused AI programming assistant that specializes in practical software engineering solutions. @@ -76,12 +118,12 @@ End with: "**`To clear buffer highlights, please ask a different question.`**" If no issues found, confirm the code is well-written and explain why. ]] ----@class CopilotChat.config.prompt : CopilotChat.config.shared +---@class CopilotChat.config.prompts.Prompt : CopilotChat.config.Shared ---@field prompt string? ---@field description string? ---@field mapping string? ----@type table +---@type table return { COPILOT_BASE = { system_prompt = COPILOT_BASE, @@ -163,6 +205,6 @@ return { Commit = { prompt = 'Write commit message for the change with commitizen convention. Keep the title under 50 characters and wrap message at 72 characters. Format as a gitcommit code block.', - context = 'git:staged', + sticky = '#git:staged', }, } diff --git a/lua/CopilotChat/config/providers.lua b/lua/CopilotChat/config/providers.lua index c34a398b..a9762737 100644 --- a/lua/CopilotChat/config/providers.lua +++ b/lua/CopilotChat/config/providers.lua @@ -67,52 +67,27 @@ local function get_github_token() error('Failed to find GitHub token') end ----@class CopilotChat.Provider.model ----@field id string ----@field name string ----@field tokenizer string? ----@field max_input_tokens number? ----@field max_output_tokens number? - ----@class CopilotChat.Provider.agent ----@field id string ----@field name string ----@field description string? - ----@class CopilotChat.Provider.embed ----@field index number ----@field embedding table - ----@class CopilotChat.Provider.options ----@field model CopilotChat.Provider.model ----@field agent CopilotChat.Provider.agent? +---@class CopilotChat.config.providers.Options +---@field model CopilotChat.client.Model ---@field temperature number? +---@field tools table? ----@class CopilotChat.Provider.input ----@field role string ----@field content string - ----@class CopilotChat.Provider.reference ----@field name string ----@field url string - ----@class CopilotChat.Provider.output +---@class CopilotChat.config.providers.Output ---@field content string ---@field finish_reason string? ---@field total_tokens number? ----@field references table? +---@field tool_calls table ----@class CopilotChat.Provider +---@class CopilotChat.config.providers.Provider ---@field disabled nil|boolean ---@field get_headers nil|fun():table,number? ----@field get_agents nil|fun(headers:table):table ----@field get_models nil|fun(headers:table):table ----@field embed nil|string|fun(inputs:table, headers:table):table ----@field prepare_input nil|fun(inputs:table, opts:CopilotChat.Provider.options):table ----@field prepare_output nil|fun(output:table, opts:CopilotChat.Provider.options):CopilotChat.Provider.output ----@field get_url nil|fun(opts:CopilotChat.Provider.options):string - ----@type table +---@field get_models nil|fun(headers:table):table +---@field embed nil|string|fun(inputs:table, headers:table):table +---@field prepare_input nil|fun(inputs:table, opts:CopilotChat.config.providers.Options):table +---@field prepare_output nil|fun(output:table, opts:CopilotChat.config.providers.Options):CopilotChat.config.providers.Output +---@field get_url nil|fun(opts:CopilotChat.config.providers.Options):string + +---@type table local M = {} M.copilot = { @@ -139,25 +114,6 @@ M.copilot = { response.body.expires_at end, - get_agents = function(headers) - local response, err = utils.curl_get('https://api.githubcopilot.com/agents', { - json_response = true, - headers = headers, - }) - - if err then - error(err) - end - - return vim.tbl_map(function(agent) - return { - id = agent.slug, - name = agent.name, - description = agent.description, - } - end, response.body.agents) - end, - get_models = function(headers) local response, err = utils.curl_get('https://api.githubcopilot.com/models', { json_response = true, @@ -171,7 +127,7 @@ M.copilot = { local models = vim .iter(response.body.data) :filter(function(model) - return model.capabilities.type == 'chat' and not vim.endswith(model.id, 'paygo') + return model.capabilities.type == 'chat' and model.model_picker_enabled end) :map(function(model) return { @@ -180,6 +136,8 @@ M.copilot = { tokenizer = model.capabilities.tokenizer, max_input_tokens = model.capabilities.limits.max_prompt_tokens, max_output_tokens = model.capabilities.limits.max_output_tokens, + streaming = model.capabilities.supports.streaming, + tools = model.capabilities.supports.tool_calls, policy = not model['policy'] or model['policy']['state'] == 'enabled', version = model.version, } @@ -224,12 +182,25 @@ M.copilot = { local out = { messages = inputs, model = opts.model.id, + stream = opts.model.streaming or false, } + if opts.tools and opts.model.tools then + out.tools = vim.tbl_map(function(tool) + return { + type = 'function', + ['function'] = { + name = tool.name, + description = tool.description, + parameters = tool.schema, + }, + } + end, opts.tools) + end + if not is_o1 then out.n = 1 out.top_p = 1 - out.stream = true out.temperature = opts.temperature end @@ -241,46 +212,51 @@ M.copilot = { end, prepare_output = function(output) - local references = {} - - if output.copilot_references then - for _, reference in ipairs(output.copilot_references) do - local metadata = reference.metadata - if metadata and metadata.display_name and metadata.display_url then - table.insert(references, { - name = metadata.display_name, - url = metadata.display_url, - }) + local tool_calls = {} + + local choice + if output.choices and #output.choices > 0 then + for _, choice in ipairs(output.choices) do + local message = choice.message or choice.delta + if message and message.tool_calls then + for i, tool_call in ipairs(message.tool_calls) do + local fn = tool_call['function'] + if fn then + local index = tool_call.index or i + local id = utils.empty(tool_call.id) and ('tooluse_' .. index) or tool_call.id + table.insert(tool_calls, { + id = id, + index = index, + name = fn.name, + arguments = fn.arguments or '', + }) + end + end end end - end - local message - if output.choices and #output.choices > 0 then - message = output.choices[1] + choice = output.choices[1] else - message = output + choice = output end - local content = message.message and message.message.content or message.delta and message.delta.content - - local usage = message.usage and message.usage.total_tokens or output.usage and output.usage.total_tokens - - local finish_reason = message.finish_reason or message.done_reason or output.finish_reason or output.done_reason + local message = choice.message or choice.delta + local content = message and message.content + local usage = choice.usage and choice.usage.total_tokens + if not usage then + usage = output.usage and output.usage.total_tokens + end + local finish_reason = choice.finish_reason or choice.done_reason or output.finish_reason or output.done_reason return { content = content, finish_reason = finish_reason, total_tokens = usage, - references = references, + tool_calls = tool_calls, } end, - get_url = function(opts) - if opts.agent then - return 'https://api.githubcopilot.com/agents/' .. opts.agent.id .. '?chat' - end - + get_url = function() return 'https://api.githubcopilot.com/chat/completions' end, } @@ -336,6 +312,7 @@ M.github_models = { tokenizer = 'o200k_base', max_input_tokens = max_input_tokens, max_output_tokens = max_output_tokens, + streaming = true, } end) :totable() diff --git a/lua/CopilotChat/functions.lua b/lua/CopilotChat/functions.lua new file mode 100644 index 00000000..8a28ca81 --- /dev/null +++ b/lua/CopilotChat/functions.lua @@ -0,0 +1,198 @@ +local utils = require('CopilotChat.utils') + +local M = {} + +local INPUT_SEPARATOR = ';;' + +local function sorted_propnames(schema) + local prop_names = vim.tbl_keys(schema.properties) + local required_set = {} + if schema.required then + for _, name in ipairs(schema.required) do + required_set[name] = true + end + end + + -- Sort properties with priority: required without default > required with default > optional + table.sort(prop_names, function(a, b) + local a_required = required_set[a] or false + local b_required = required_set[b] or false + local a_has_default = schema.properties[a].default ~= nil + local b_has_default = schema.properties[b].default ~= nil + + -- First priority: required properties without default + if a_required and not a_has_default and (not b_required or b_has_default) then + return true + end + if b_required and not b_has_default and (not a_required or a_has_default) then + return false + end + + -- Second priority: required properties with default + if a_required and not b_required then + return true + end + if b_required and not a_required then + return false + end + + -- Finally sort alphabetically + return a < b + end) + + return prop_names +end + +local function filter_schema(tbl) + if type(tbl) ~= 'table' then + return tbl + end + + local result = {} + for k, v in pairs(tbl) do + if type(v) ~= 'function' and k ~= 'examples' then + result[k] = type(v) == 'table' and filter_schema(v) or v + end + end + return result +end + +---@param uri string The URI to parse +---@param pattern string The pattern to match against (e.g., 'file://{path}') +---@return table|nil inputs Extracted parameters or nil if no match +function M.match_uri(uri, pattern) + -- Convert the pattern into a Lua pattern by escaping special characters + -- and replacing {name} placeholders with capture groups + local lua_pattern = pattern:gsub('([%(%)%.%%%+%-%*%?%[%]%^%$])', '%%%1') + + -- Extract parameter names from the pattern + local param_names = {} + for param in pattern:gmatch('{([^}:*]+)[^}]*}') do + table.insert(param_names, param) + -- Replace {param} with a capture group in our Lua pattern + -- Use non-greedy capture to handle multiple params properly + lua_pattern = lua_pattern:gsub('{' .. param .. '[^}]*}', '(.-)') + end + + -- If no parameters, just do a direct comparison + if #param_names == 0 then + return uri == pattern and {} or nil + end + + -- Match the URI against our constructed pattern + local matches = { uri:match('^' .. lua_pattern .. '$') } + + -- If match failed, return nil + if #matches == 0 or matches[1] == nil then + return nil + end + + -- Build the result table mapping parameter names to their values + local result = {} + for i, param_name in ipairs(param_names) do + result[param_name] = matches[i] + end + + return result +end + +--- Prepare the schema for use +---@param tools table +---@return table +function M.parse_tools(tools) + local tool_names = vim.tbl_keys(tools) + table.sort(tool_names) + return vim.tbl_map(function(name) + local tool = tools[name] + local schema = tool.schema + + if schema then + schema = filter_schema(schema) + end + + return { + name = name, + description = tool.description, + schema = schema, + } + end, tool_names) +end + +--- Parse context input string into a table based on the schema +---@param input string|table|nil +---@param schema table? +---@return table +function M.parse_input(input, schema) + if not schema or not schema.properties then + return {} + end + + if type(input) == 'table' then + return input + end + + local parts = vim.split(input or '', INPUT_SEPARATOR) + local result = {} + local prop_names = sorted_propnames(schema) + + -- Map input parts to schema properties in sorted order + local i = 1 + for _, prop_name in ipairs(prop_names) do + local prop_schema = schema.properties[prop_name] + local value = not utils.empty(parts[i]) and parts[i] or nil + if value == nil and prop_schema.default ~= nil then + value = prop_schema.default + end + + result[prop_name] = value + i = i + 1 + if i > #parts then + break + end + end + + return result +end + +--- Get input from the user based on the schema +---@param schema table? +---@param source CopilotChat.source +---@return string? +function M.enter_input(schema, source) + if not schema or not schema.properties then + return nil + end + + local prop_names = sorted_propnames(schema) + local out = {} + + for _, prop_name in ipairs(prop_names) do + local cfg = schema.properties[prop_name] + if not schema.required or vim.tbl_contains(schema.required, prop_name) then + if cfg.enum then + local choices = type(cfg.enum) == 'table' and cfg.enum or cfg.enum(source) + local choice = utils.select(choices, { + prompt = string.format('Select %s> ', prop_name), + }) + + table.insert(out, choice or '') + elseif cfg.type == 'boolean' then + table.insert(out, utils.select({ 'true', 'false' }, { + prompt = string.format('Select %s> ', prop_name), + }) or '') + else + table.insert(out, utils.input({ + prompt = string.format('Enter %s> ', prop_name), + }) or '') + end + end + end + + local out = vim.trim(table.concat(out, INPUT_SEPARATOR)) + if out:match('%s+') then + out = string.format('`%s`', out) + end + return out +end + +return M diff --git a/lua/CopilotChat/health.lua b/lua/CopilotChat/health.lua index 5d02df19..cf1e568a 100644 --- a/lua/CopilotChat/health.lua +++ b/lua/CopilotChat/health.lua @@ -4,7 +4,6 @@ local start = vim.health.start or vim.health.report_start local error = vim.health.error or vim.health.report_error local warn = vim.health.warn or vim.health.report_warn local ok = vim.health.ok or vim.health.report_ok -local info = vim.health.info or vim.health.report_info --- Run a command and handle potential errors ---@param executable string @@ -42,7 +41,7 @@ end function M.check() start('CopilotChat.nvim [core]') - local vim_version = vim.trim(vim.api.nvim_command_output('version')) + local vim_version = vim.trim(vim.api.nvim_exec2('version', { output = true }).output) if vim.fn.has('nvim-0.10.0') == 1 then ok('nvim: ' .. vim_version) else diff --git a/lua/CopilotChat/init.lua b/lua/CopilotChat/init.lua index 1062e1a4..cf9b84ec 100644 --- a/lua/CopilotChat/init.lua +++ b/lua/CopilotChat/init.lua @@ -1,17 +1,21 @@ local async = require('plenary.async') local log = require('plenary.log') -local context = require('CopilotChat.context') +local functions = require('CopilotChat.functions') +local resources = require('CopilotChat.resources') local client = require('CopilotChat.client') local notify = require('CopilotChat.notify') local utils = require('CopilotChat.utils') local PLUGIN_NAME = 'CopilotChat' -local WORD = '([^%s]+)' -local WORD_INPUT = '([^%s:]+:`[^`]+`)' +local WORD = '([^%s:]+)' +local WORD_NO_INPUT = '([^%s]+)' +local WORD_WITH_INPUT_QUOTED = WORD .. ':`([^`]+)`' +local WORD_WITH_INPUT_UNQUOTED = WORD .. ':?([^%s`]*)' +local TOOL_OUTPUT_FORMAT = '```%s tool=%s id=%s\n%s\n```' ---@class CopilotChat ----@field config CopilotChat.config ----@field chat CopilotChat.ui.Chat +---@field config CopilotChat.config.Config +---@field chat CopilotChat.ui.chat.Chat local M = {} --- @class CopilotChat.source @@ -21,23 +25,19 @@ local M = {} --- @class CopilotChat.state --- @field source CopilotChat.source? ---- @field last_prompt string? ---- @field last_response string? ---- @field highlights_loaded boolean +--- @field sticky string[]? local state = { -- Current state tracking source = nil, -- Last state tracking - last_prompt = nil, - last_response = nil, - highlights_loaded = false, + sticky = nil, } --- Insert sticky values from config into prompt ---@param prompt string ----@param config CopilotChat.config.shared -local function insert_sticky(prompt, config, override_sticky) +---@param config CopilotChat.config.Shared +local function insert_sticky(prompt, config) local lines = vim.split(prompt or '', '\n') local stickies = utils.ordered_map() @@ -58,8 +58,10 @@ local function insert_sticky(prompt, config, override_sticky) stickies:set('$' .. config.model, true) end - if config.remember_as_sticky and config.agent and config.agent ~= M.config.agent then - stickies:set('@' .. config.agent, true) + if config.remember_as_sticky and config.tools and not vim.deep_equal(config.tools, M.config.tools) then + for _, tool in ipairs(utils.to_table(config.tools)) do + stickies:set('@' .. tool, true) + end end if @@ -71,32 +73,18 @@ local function insert_sticky(prompt, config, override_sticky) stickies:set('/' .. config.system_prompt, true) end - if config.remember_as_sticky and config.context and not vim.deep_equal(config.context, M.config.context) then - if type(config.context) == 'table' then - ---@diagnostic disable-next-line: param-type-mismatch - for _, context in ipairs(config.context) do - stickies:set('#' .. context, true) - end - else - stickies:set('#' .. config.context, true) - end - end - - if config.sticky and (override_sticky or not vim.deep_equal(config.sticky, M.config.sticky)) then - if type(config.sticky) == 'table' then - ---@diagnostic disable-next-line: param-type-mismatch - for _, sticky in ipairs(config.sticky) do - stickies:set(sticky, true) - end - else - stickies:set(config.sticky, true) + if config.sticky and not vim.deep_equal(config.sticky, M.config.sticky) then + for _, sticky in ipairs(utils.to_table(config.sticky)) do + stickies:set(sticky, true) end end -- Insert stickies at start of prompt local prompt_lines = {} for _, sticky in ipairs(stickies:keys()) do - table.insert(prompt_lines, '> ' .. sticky) + if sticky ~= '' then + table.insert(prompt_lines, '> ' .. sticky) + end end if #prompt_lines > 0 then table.insert(prompt_lines, '') @@ -130,65 +118,36 @@ local function update_highlights() strict = false, }) end - - if state.highlights_loaded then - return - end - - async.run(function() - local items = M.complete_items() - utils.schedule_main() - - for _, item in ipairs(items) do - local pattern = vim.fn.escape(item.word, '.-$^*[]') - if vim.startswith(item.word, '#') then - vim.cmd('syntax match CopilotChatKeyword "' .. pattern .. '\\(:.\\+\\)\\?" containedin=ALL') - else - vim.cmd('syntax match CopilotChatKeyword "' .. pattern .. '" containedin=ALL') - end - end - - vim.cmd('syntax match CopilotChatInput ":\\(.\\+\\)" contained containedin=CopilotChatKeyword') - state.highlights_loaded = true - end) end --- Finish writing to chat buffer. ---@param start_of_chat boolean? local function finish(start_of_chat) - if not start_of_chat then + if start_of_chat then + local sticky = {} + if M.config.sticky then + for _, sticky_line in ipairs(utils.to_table(M.config.sticky)) do + table.insert(sticky, sticky_line) + end + end + state.sticky = sticky + else M.chat:append('\n\n') end M.chat:append(M.config.question_header .. M.config.separator .. '\n\n') - -- Insert sticky values from config into prompt - if start_of_chat then - state.last_prompt = insert_sticky(state.last_prompt, M.config, true) - end - - -- Reinsert sticky prompts from last prompt and last response - local lines = {} - if state.last_prompt then - lines = vim.split(state.last_prompt, '\n') - end - if state.last_response then - for _, line in ipairs(vim.split(state.last_response, '\n')) do - table.insert(lines, line) + if not utils.empty(M.chat.tool_calls) then + for _, tool_call in ipairs(M.chat.tool_calls) do + M.chat:append(string.format('#%s:%s\n', tool_call.name, tool_call.id)) end + M.chat:append('\n') end - local has_sticky = false - local in_code_block = false - for _, line in ipairs(lines) do - if line:match('^```') then - in_code_block = not in_code_block - end - if vim.startswith(line, '> ') and not in_code_block then - M.chat:append(line .. '\n') - has_sticky = true + + if not utils.empty(state.sticky) then + for _, sticky in ipairs(state.sticky) do + M.chat:append('> ' .. sticky .. '\n') end - end - if has_sticky then M.chat:append('\n') end @@ -199,19 +158,7 @@ end ---@param err string|table|nil local function show_error(err) err = err or 'Unknown error' - - if type(err) == 'string' then - while true do - local new_err = err:gsub('^[^:]+:%d+: ', '') - if new_err == err then - break - end - err = new_err - end - else - err = utils.make_string(err) - end - + err = utils.make_string(err) M.chat:append('\n' .. M.config.error_header .. '\n```error\n' .. err .. '\n```') finish() end @@ -261,10 +208,148 @@ local function update_source() M.set_source(use_prev_window and vim.fn.win_getid(vim.fn.winnr('#')) or vim.api.nvim_get_current_win()) end +--- Call and resolve tools from the prompt. +---@param prompt string? +---@param config CopilotChat.config.Shared? +---@return table, table, string +---@async +function M.resolve_tools(prompt, config) + config, prompt = M.resolve_prompt(prompt, config) + local enabled_tools = {} + local resolved_resources = {} + local matches = utils.to_table(config.tools) + + -- Check for @tool pattern to find enabled tools + prompt = prompt:gsub('@' .. WORD, function(match) + for name, tool in pairs(M.config.functions) do + if name == match or tool.group == match then + table.insert(matches, match) + return '' + end + end + return '@' .. match + end) + for _, match in ipairs(matches) do + for name, tool in pairs(M.config.functions) do + if name == match or tool.group == match then + enabled_tools[name] = tool + end + end + end + + local matches = utils.ordered_map() + + -- Check for #word:`input` pattern + for word, input in prompt:gmatch('#' .. WORD_WITH_INPUT_QUOTED) do + local pattern = string.format('#%s:`%s`', word, input) + matches:set(pattern, { + word = word, + input = input, + }) + end + + -- Check for #word:input pattern + for word, input in prompt:gmatch('#' .. WORD_WITH_INPUT_UNQUOTED) do + local pattern = utils.empty(input) and string.format('#%s', word) or string.format('#%s:%s', word, input) + matches:set(pattern, { + word = word, + input = input, + }) + end + + -- Check for ##word:input pattern + for word in prompt:gmatch('##' .. WORD_NO_INPUT) do + local pattern = string.format('##%s', word) + matches:set(pattern, { + word = word, + }) + end + + -- Resolve each tool reference + local function expand_tool(name, input) + notify.publish(notify.STATUS, 'Running function: ' .. name) + + local tool_id = nil + if not utils.empty(M.chat.tool_calls) then + for _, tool_call in ipairs(M.chat.tool_calls) do + if tool_call.name == name and vim.trim(tool_call.id) == vim.trim(input) then + input = utils.empty(tool_call.arguments) and {} or utils.json_decode(tool_call.arguments) + tool_id = tool_call.id + break + end + end + end + + local tool = tool_id and enabled_tools[name] + if not tool then + -- Check if tool is resource and call it even when not enabled + tool = M.config.functions[name] + if tool and not tool.uri then + return nil + end + end + if not tool then + -- Check if input matches uri + for tool_name, tool_spec in pairs(M.config.functions) do + if tool_spec.uri then + local match = functions.match_uri(name, tool_spec.uri) + if match then + name = tool_name + tool = tool_spec + input = match + break + end + end + end + end + if not tool then + return nil + end + + local ok, output = pcall(tool.resolve, functions.parse_input(input, tool.schema), state.source or {}, prompt) + if not ok then + return string.format(TOOL_OUTPUT_FORMAT, 'error', name, tool_id or '', utils.make_string(output)) .. '\n' + end + + local result = '' + for _, content in ipairs(output) do + if content then + local content_out = nil + if content.uri then + content_out = '##' .. content.uri + table.insert(resolved_resources, resources.to_resource(content)) + if tool_id then + table.insert(state.sticky, content_out) + end + else + local ft = utils.mimetype_to_filetype(content.mimetype) + content_out = string.format(TOOL_OUTPUT_FORMAT, ft, name, tool_id or '', content.data) + end + + if not utils.empty(result) then + result = result .. '\n' + end + result = result .. content_out + end + end + + return result + end + + -- Resolve and process all tools + for _, pattern in ipairs(matches:keys()) do + local match = matches:get(pattern) + local out = expand_tool(match.word, match.input) or pattern + prompt = prompt:gsub(vim.pesc(pattern), out, 1) + end + + return functions.parse_tools(enabled_tools), resolved_resources, prompt +end + --- Resolve the final prompt and config from prompt template. ---@param prompt string? ----@param config CopilotChat.config.shared? ----@return CopilotChat.config.prompt, string +---@param config CopilotChat.config.Shared? +---@return CopilotChat.config.prompts.Prompt, string function M.resolve_prompt(prompt, config) if not prompt then local section = M.chat:get_prompt() @@ -303,107 +388,20 @@ function M.resolve_prompt(prompt, config) if prompts_to_use[config.system_prompt] then config.system_prompt = prompts_to_use[config.system_prompt].system_prompt end - return config, prompt -end - ---- Resolve the context embeddings from the prompt. ----@param prompt string? ----@param config CopilotChat.config.shared? ----@return table, string ----@async -function M.resolve_context(prompt, config) - config, prompt = M.resolve_prompt(prompt, config) - local contexts = {} - local function parse_context(prompt_context) - local split = vim.split(prompt_context, ':') - local context_name = table.remove(split, 1) - local context_input = vim.trim(table.concat(split, ':')) - if vim.startswith(context_input, '`') and vim.endswith(context_input, '`') then - context_input = context_input:sub(2, -2) - end - - if M.config.contexts[context_name] then - table.insert(contexts, { - name = context_name, - input = (context_input ~= '' and context_input or nil), - }) - - return true - end - - return false - end - - prompt = prompt:gsub('#' .. WORD_INPUT, function(match) - return parse_context(match) and '' or '#' .. match - end) - - prompt = prompt:gsub('#' .. WORD, function(match) - return parse_context(match) and '' or '#' .. match - end) - - if config.context then - if type(config.context) == 'table' then - ---@diagnostic disable-next-line: param-type-mismatch - for _, config_context in ipairs(config.context) do - parse_context(config_context) - end - else - parse_context(config.context) - end - end - - local embeddings = utils.ordered_map() - for _, context_data in ipairs(contexts) do - local context_value = M.config.contexts[context_data.name] - notify.publish( - notify.STATUS, - 'Resolving context: ' .. context_data.name .. (context_data.input and ' with input: ' .. context_data.input or '') - ) - - local ok, resolved_embeddings = pcall(context_value.resolve, context_data.input, state.source or {}, prompt) - if ok then - for _, embedding in ipairs(resolved_embeddings) do - if embedding then - embeddings:set(embedding.filename, embedding) - end - end - else - log.error('Failed to resolve context: ' .. context_data.name, resolved_embeddings) + if config.system_prompt then + config.system_prompt = config.system_prompt:gsub('{OS_NAME}', jit.os) + if state.source then + config.system_prompt = config.system_prompt:gsub('{DIR}', state.source.cwd()) end end - return embeddings:values(), prompt -end - ---- Resolve the agent from the prompt. ----@param prompt string? ----@param config CopilotChat.config.shared? ----@return string, string ----@async -function M.resolve_agent(prompt, config) - config, prompt = M.resolve_prompt(prompt, config) - - local agents = vim.tbl_map(function(agent) - return agent.id - end, client:list_agents()) - - local selected_agent = config.agent or '' - prompt = prompt:gsub('@' .. WORD, function(match) - if vim.tbl_contains(agents, match) then - selected_agent = match - return '' - end - return '@' .. match - end) - - return selected_agent, prompt + return config, prompt end --- Resolve the model from the prompt. ---@param prompt string? ----@param config CopilotChat.config.shared? +---@param config CopilotChat.config.Shared? ---@return string, string ---@async function M.resolve_model(prompt, config) @@ -457,7 +455,7 @@ function M.set_source(source_winnr) end --- Get the selection from the source buffer. ----@return CopilotChat.select.selection? +---@return CopilotChat.select.Selection? function M.get_selection() local config = vim.tbl_deep_extend('force', M.config, M.chat.config) local selection = config.selection @@ -506,8 +504,8 @@ function M.set_selection(bufnr, start_line, end_line, clear) end --- Trigger the completion for the chat window. ----@param without_context boolean? -function M.trigger_complete(without_context) +---@param without_input boolean? +function M.trigger_complete(without_input) local info = M.complete_info() local bufnr = vim.api.nvim_get_current_buf() local line = vim.api.nvim_get_current_line() @@ -523,23 +521,18 @@ function M.trigger_complete(without_context) return end - if not without_context and vim.startswith(prefix, '#') and vim.endswith(prefix, ':') then - local found_context = M.config.contexts[prefix:sub(2, -2)] - if found_context and found_context.input then + if not without_input and vim.startswith(prefix, '#') and vim.endswith(prefix, ':') then + local found_tool = M.config.functions[prefix:sub(2, -2)] + if found_tool and found_tool.schema then async.run(function() - found_context.input(function(value) - if not value then - return - end - - local value_str = vim.trim(tostring(value)) - if value_str:find('%s') then - value_str = '`' .. value_str .. '`' - end + local value = functions.enter_input(found_tool.schema, state.source) + if not value then + return + end - vim.api.nvim_buf_set_text(bufnr, row - 1, col, row - 1, col, { value_str }) - vim.api.nvim_win_set_cursor(0, { row, col + #value_str }) - end, state.source or {}) + utils.schedule_main() + vim.api.nvim_buf_set_text(bufnr, row - 1, col, row - 1, col, { value }) + vim.api.nvim_win_set_cursor(0, { row, col + #value }) end) end @@ -577,7 +570,6 @@ end ---@async function M.complete_items() local models = client:list_models() - local agents = client:list_agents() local prompts_to_use = M.prompts() local items = {} @@ -616,32 +608,59 @@ function M.complete_items() } end - for _, agent in pairs(agents) do + local groups = {} + for name, tool in pairs(M.config.functions) do + if tool.group then + groups[tool.group] = groups[tool.group] or {} + groups[tool.group][name] = tool + end + end + for name, group in pairs(groups) do + local group_tools = vim.tbl_keys(group) items[#items + 1] = { - word = '@' .. agent.id, - abbr = agent.id, - kind = agent.provider, - info = agent.description, - menu = agent.name, + word = '@' .. name, + abbr = name, + kind = 'group', + info = table.concat(group_tools, '\n'), + menu = string.format('%s tools', #group_tools), icase = 1, dup = 0, empty = 0, } end - - for name, value in pairs(M.config.contexts) do + for name, tool in pairs(M.config.functions) do items[#items + 1] = { - word = '#' .. name, + word = '@' .. name, abbr = name, - kind = 'context', - info = value.description or '', - menu = value.input and string.format('#%s:', name) or string.format('#%s', name), + kind = 'tool', + info = tool.description, + menu = tool.group or '', icase = 1, dup = 0, empty = 0, } end + local tools_to_use = functions.parse_tools(M.config.functions) + for _, tool in pairs(tools_to_use) do + local uri = M.config.functions[tool.name].uri + if uri then + local info = + string.format('%s\n\n%s', tool.description, tool.schema and vim.inspect(tool.schema, { indent = ' ' }) or '') + + items[#items + 1] = { + word = '#' .. tool.name, + abbr = tool.name, + kind = 'resource', + info = info, + menu = uri, + icase = 1, + dup = 0, + empty = 0, + } + end + end + table.sort(items, function(a, b) if a.kind == b.kind then return a.word < b.word @@ -653,7 +672,7 @@ function M.complete_items() end --- Get the prompts to use. ----@return table +---@return table function M.prompts() local prompts_to_use = {} @@ -676,7 +695,7 @@ function M.prompts() end --- Open the chat window. ----@param config CopilotChat.config.shared? +---@param config CopilotChat.config.Shared? function M.open(config) config = vim.tbl_deep_extend('force', M.config, config or {}) utils.return_to_normal_mode() @@ -701,7 +720,7 @@ function M.close() end --- Toggle the chat window. ----@param config CopilotChat.config.shared? +---@param config CopilotChat.config.Shared? function M.toggle(config) if M.chat:visible() then M.close() @@ -710,12 +729,6 @@ function M.toggle(config) end end ---- Get the last response. ---- @returns string -function M.response() - return state.last_response -end - --- Select default Copilot GPT model. function M.select_model() async.run(function() @@ -725,6 +738,8 @@ function M.select_model() id = model.id, name = model.name, provider = model.provider, + streaming = model.streaming, + tools = model.tools, selected = model.id == M.config.model, } end, models) @@ -733,53 +748,39 @@ function M.select_model() vim.ui.select(choices, { prompt = 'Select a model> ', format_item = function(item) - local out = string.format('%s (%s:%s)', item.name, item.provider, item.id) + local indicators = {} + local out = item.name + if item.selected then out = '* ' .. out end - return out - end, - }, function(choice) - if choice then - M.config.model = choice.id - end - end) - end) -end ---- Select default Copilot agent. -function M.select_agent() - async.run(function() - local agents = client:list_agents() - local choices = vim.tbl_map(function(agent) - return { - id = agent.id, - name = agent.name, - provider = agent.provider, - selected = agent.id == M.config.agent, - } - end, agents) + if item.provider then + table.insert(indicators, item.provider) + end + if item.streaming then + table.insert(indicators, 'streaming') + end + if item.tools then + table.insert(indicators, 'tools') + end - utils.schedule_main() - vim.ui.select(choices, { - prompt = 'Select an agent> ', - format_item = function(item) - local out = string.format('%s (%s:%s)', item.name, item.provider, item.id) - if item.selected then - out = '* ' .. out + if #indicators > 0 then + out = out .. ' [' .. table.concat(indicators, ', ') .. ']' end + return out end, }, function(choice) if choice then - M.config.agent = choice.id + M.config.model = choice.id end end) end) end --- Select a prompt template to use. ----@param config CopilotChat.config.shared? +---@param config CopilotChat.config.Shared? function M.select_prompt(config) local prompts = M.prompts() local keys = vim.tbl_keys(prompts) @@ -813,7 +814,7 @@ end --- Ask a question to the Copilot model. ---@param prompt string? ----@param config CopilotChat.config.shared? +---@param config CopilotChat.config.Shared? function M.ask(prompt, config) prompt = prompt or '' if prompt == '' then @@ -836,9 +837,20 @@ function M.ask(prompt, config) M.open(config) end - state.last_prompt = prompt + local sticky = {} + local in_code_block = false + for _, line in ipairs(vim.split(prompt, '\n')) do + if line:match('^```') then + in_code_block = not in_code_block + end + if vim.startswith(line, '> ') and not in_code_block then + table.insert(sticky, line:sub(3)) + end + end + + state.sticky = sticky M.chat:set_prompt(prompt) - M.chat:append('\n\n' .. M.config.answer_header .. M.config.separator .. '\n\n') + M.chat:append('\n\n') M.chat:follow() else update_source() @@ -848,16 +860,6 @@ function M.ask(prompt, config) config, prompt = M.resolve_prompt(prompt, config) local system_prompt = config.system_prompt or '' - -- Resolve context name and description - local contexts = {} - if config.include_contexts_in_prompt then - for name, context in pairs(M.config.contexts) do - if context.description then - contexts[name] = context.description - end - end - end - -- Remove sticky prefix prompt = table.concat( vim.tbl_map(function(l) @@ -870,30 +872,30 @@ function M.ask(prompt, config) local selection = M.get_selection() local ok, err = pcall(async.run, function() - local selected_agent, prompt = M.resolve_agent(prompt, config) + local selected_tools, selected_resources, prompt = M.resolve_tools(prompt, config) local selected_model, prompt = M.resolve_model(prompt, config) - local embeddings, prompt = M.resolve_context(prompt, config) - - local query_ok, filtered_embeddings = - pcall(context.filter_embeddings, prompt, selected_model, config.headless, embeddings) + local query_ok, processed_resources = + pcall(resources.process_resources, prompt, selected_model, config.headless, selected_resources) + if query_ok then + selected_resources = processed_resources + else + log.warn('Failed to process resources', processed_resources) + end - if not query_ok then + if not config.headless then utils.schedule_main() - log.error(filtered_embeddings) - if not config.headless then - show_error(filtered_embeddings) - end - return + M.chat:set_prompt(vim.trim(prompt)) + M.chat:append('\n\n' .. M.config.answer_header .. M.config.separator .. '\n\n') + M.chat:follow() end - local ask_ok, response, references, token_count, token_max_count = pcall(client.ask, client, prompt, { + local ask_ok, ask_response = pcall(client.ask, client, prompt, { headless = config.headless, - contexts = contexts, selection = selection, - embeddings = filtered_embeddings, + resources = selected_resources, + tools = selected_tools, system_prompt = system_prompt, model = selected_model, - agent = selected_agent, temperature = config.temperature, on_progress = vim.schedule_wrap(function(token) local out = config.stream and config.stream(token, state.source) or nil @@ -910,18 +912,23 @@ function M.ask(prompt, config) utils.schedule_main() if not ask_ok then - log.error(response) + log.error(ask_response) if not config.headless then - show_error(response) + show_error(ask_response) end return end -- If there was no error and no response, it means job was cancelled - if response == nil then + if ask_response == nil then return end + local response = ask_response.content + local token_count = ask_response.token_count + local token_max_count = ask_response.token_max_count + local tool_calls = ask_response.tool_calls + -- Call the callback function and store to history local out = config.callback and config.callback(response, state.source) or nil if out == nil then @@ -940,18 +947,9 @@ function M.ask(prompt, config) end if not config.headless then - state.last_response = response - M.chat.references = references + M.chat.tool_calls = tool_calls M.chat.token_count = token_count M.chat.token_max_count = token_max_count - - if not utils.empty(references) and config.references_display == 'write' then - M.chat:append('\n\n**`References`**:') - for _, ref in ipairs(references) do - M.chat:append(string.format('\n[%s](%s)', ref.name, ref.url)) - end - end - finish() end end) @@ -973,8 +971,6 @@ function M.stop(reset) client:reset() M.chat:clear() vim.diagnostic.reset(vim.api.nvim_create_namespace('copilot-chat-diagnostics')) - state.last_prompt = nil - state.last_response = nil -- Clear the selection if state.source then @@ -1104,7 +1100,7 @@ function M.log_level(level) end --- Set up the plugin ----@param config CopilotChat.config? +---@param config CopilotChat.config.Config? function M.setup(config) M.config = vim.tbl_deep_extend('force', require('CopilotChat.config'), config or {}) state.highlights_loaded = false diff --git a/lua/CopilotChat/integrations/fzflua.lua b/lua/CopilotChat/integrations/fzflua.lua deleted file mode 100644 index 174d0139..00000000 --- a/lua/CopilotChat/integrations/fzflua.lua +++ /dev/null @@ -1,42 +0,0 @@ -local fzflua = require('fzf-lua') -local chat = require('CopilotChat') -local utils = require('CopilotChat.utils') - -local M = {} - ---- Pick an action from a list of actions ----@param pick_actions CopilotChat.integrations.actions?: A table with the actions to pick from ----@param opts table?: fzf-lua options ----@deprecated Use |CopilotChat.select_prompt| instead -function M.pick(pick_actions, opts) - if not pick_actions or not pick_actions.actions or vim.tbl_isempty(pick_actions.actions) then - return - end - - utils.return_to_normal_mode() - opts = vim.tbl_extend('force', { - prompt = pick_actions.prompt .. '> ', - preview = function(items) - return pick_actions.actions[items[1]].prompt - end, - winopts = { - preview = { - wrap = 'wrap', - }, - }, - actions = { - ['default'] = function(selected) - if not selected or vim.tbl_isempty(selected) then - return - end - vim.defer_fn(function() - chat.ask(pick_actions.actions[selected[1]].prompt, pick_actions.actions[selected[1]]) - end, 100) - end, - }, - }, opts or {}) - - fzflua.fzf_exec(vim.tbl_keys(pick_actions.actions), opts) -end - -return M diff --git a/lua/CopilotChat/integrations/snacks.lua b/lua/CopilotChat/integrations/snacks.lua deleted file mode 100644 index 14a2daaa..00000000 --- a/lua/CopilotChat/integrations/snacks.lua +++ /dev/null @@ -1,54 +0,0 @@ -local snacks = require('snacks') -local chat = require('CopilotChat') -local utils = require('CopilotChat.utils') - -local M = {} - ---- Pick an action from a list of actions ----@param pick_actions CopilotChat.integrations.actions?: A table with the actions to pick from ----@param opts table?: snacks options ----@deprecated Use |CopilotChat.select_prompt| instead -function M.pick(pick_actions, opts) - if not pick_actions or not pick_actions.actions or vim.tbl_isempty(pick_actions.actions) then - return - end - - utils.return_to_normal_mode() - opts = vim.tbl_extend('force', { - items = vim.tbl_map(function(name) - return { - id = name, - text = name, - file = name, - preview = { - text = pick_actions.actions[name].prompt, - ft = 'text', - }, - } - end, vim.tbl_keys(pick_actions.actions)), - preview = 'preview', - win = { - preview = { - wo = { - wrap = true, - linebreak = true, - }, - }, - }, - title = pick_actions.prompt, - confirm = function(picker) - local selected = picker:current() - if selected then - local action = pick_actions.actions[selected.id] - vim.defer_fn(function() - chat.ask(action.prompt, action) - end, 100) - end - picker:close() - end, - }, opts or {}) - - snacks.picker(opts) -end - -return M diff --git a/lua/CopilotChat/integrations/telescope.lua b/lua/CopilotChat/integrations/telescope.lua deleted file mode 100644 index 5e14d913..00000000 --- a/lua/CopilotChat/integrations/telescope.lua +++ /dev/null @@ -1,65 +0,0 @@ -local actions = require('telescope.actions') -local action_state = require('telescope.actions.state') -local pickers = require('telescope.pickers') -local finders = require('telescope.finders') -local themes = require('telescope.themes') -local conf = require('telescope.config').values -local previewers = require('telescope.previewers') -local chat = require('CopilotChat') -local utils = require('CopilotChat.utils') - -local M = {} - ---- Pick an action from a list of actions ----@param pick_actions CopilotChat.integrations.actions?: A table with the actions to pick from ----@param opts table?: Telescope options ----@deprecated Use |CopilotChat.select_prompt| instead -function M.pick(pick_actions, opts) - if not pick_actions or not pick_actions.actions or vim.tbl_isempty(pick_actions.actions) then - return - end - - utils.return_to_normal_mode() - - if not (opts and opts.theme) then - opts = themes.get_dropdown(opts or {}) - end - - pickers - .new(opts, { - prompt_title = pick_actions.prompt, - finder = finders.new_table({ - results = vim.tbl_keys(pick_actions.actions), - }), - previewer = previewers.new_buffer_previewer({ - define_preview = function(self, entry) - vim.api.nvim_win_set_option(self.state.winid, 'wrap', true) - vim.api.nvim_buf_set_lines( - self.state.bufnr, - 0, - -1, - false, - vim.split(pick_actions.actions[entry[1]].prompt or '', '\n') - ) - end, - }), - sorter = conf.generic_sorter(opts), - attach_mappings = function(prompt_bufnr) - actions.select_default:replace(function() - actions.close(prompt_bufnr) - local selected = action_state.get_selected_entry() - if not selected or vim.tbl_isempty(selected) then - return - end - - vim.defer_fn(function() - chat.ask(pick_actions.actions[selected[1]].prompt, pick_actions.actions[selected[1]]) - end, 100) - end) - return true - end, - }) - :find() -end - -return M diff --git a/lua/CopilotChat/context.lua b/lua/CopilotChat/resources.lua similarity index 83% rename from lua/CopilotChat/context.lua rename to lua/CopilotChat/resources.lua index 3d50ca3d..6fcf68dc 100644 --- a/lua/CopilotChat/context.lua +++ b/lua/CopilotChat/resources.lua @@ -1,4 +1,4 @@ ----@class CopilotChat.context.symbol +---@class CopilotChat.resources.Symbol ---@field name string? ---@field signature string ---@field type string @@ -7,16 +7,6 @@ ---@field end_row number ---@field end_col number ----@class CopilotChat.context.embed ----@field content string ----@field filename string ----@field filetype string ----@field outline string? ----@field diagnostics table? ----@field symbols table? ----@field embedding table? ----@field score number? - local async = require('plenary.async') local log = require('plenary.log') local client = require('CopilotChat.client') @@ -94,9 +84,9 @@ local function spatial_distance_cosine(a, b) end --- Rank data by relatedness to the query ----@param query CopilotChat.context.embed ----@param data table ----@return table +---@param query CopilotChat.client.EmbeddedResource +---@param data table +---@return table local function data_ranked_by_relatedness(query, data) for _, item in ipairs(data) do local score = spatial_distance_cosine(item.embedding, query.embedding) @@ -189,8 +179,8 @@ end --- Rank data by symbols and filenames ---@param query string ----@param data table ----@return table +---@param data table +---@return table local function data_ranked_by_symbols(query, data) -- Get query trigrams including compound versions local query_trigrams = {} @@ -211,7 +201,7 @@ local function data_ranked_by_symbols(query, data) local max_score = 0 for _, entry in ipairs(data) do - local basename = utils.filename(entry.filename):gsub('%..*$', '') + local basename = utils.filename(entry.name):gsub('%..*$', '') -- Get trigrams for basename and compound version local file_trigrams = get_trigrams(basename) @@ -327,9 +317,9 @@ end --- Build an outline and symbols from a string ---@param content string ---@param ft string ----@return string?, table? +---@return string?, table? local function get_outline(content, ft) - if not ft or ft == '' or ft == 'text' or ft == 'raw' then + if not ft or ft == '' then return nil end @@ -399,47 +389,36 @@ end --- Get data for a file ---@param filename string ----@param filetype string? ----@return CopilotChat.context.embed? -function M.get_file(filename, filetype) +---@return string?, string? +function M.get_file(filename) + local filetype = utils.filetype(filename) if not filetype then return nil end - local modified = utils.file_mtime(filename) if not modified then return nil end - local cached = file_cache[filename] - if cached and cached._modified >= modified then - return { - content = cached.content, - _modified = cached._modified, - filename = filename, - filetype = filetype, + local data = file_cache[filename] + if not data or data._modified < modified then + local content = utils.read_file(filename) + if not content or content == '' then + return nil + end + data = { + content = content, + _modified = modified, } + file_cache[filename] = data end - local content = utils.read_file(filename) - if not content or content == '' then - return nil - end - - local out = { - content = content, - filename = filename, - filetype = filetype, - _modified = modified, - } - - file_cache[filename] = out - return out + return data.content, utils.filetype_to_mimetype(filetype) end --- Get data for a buffer ---@param bufnr number ----@return CopilotChat.context.embed? +---@return string?, string? function M.get_buffer(bufnr) if not utils.buf_valid(bufnr) then return nil @@ -450,23 +429,18 @@ function M.get_buffer(bufnr) return nil end - return { - content = table.concat(content, '\n'), - filename = utils.filepath(vim.api.nvim_buf_get_name(bufnr)), - filetype = vim.bo[bufnr].filetype, - score = 0.1, - diagnostics = utils.diagnostics(bufnr), - } + return table.concat(content, '\n'), utils.filetype_to_mimetype(vim.bo[bufnr].filetype) end --- Get the content of an URL ---@param url string ----@return CopilotChat.context.embed? +---@return string?, string? function M.get_url(url) if not url or url == '' then return nil end + local ft = utils.filetype(url) local content = url_cache[url] if not content then local ok, out = async.util.apcall(utils.system, { 'lynx', '-dump', url }) @@ -504,37 +478,42 @@ function M.get_url(url) url_cache[url] = content end + return content, utils.filetype_to_mimetype(ft) +end + +--- Transform a resource into a format suitable for the client +---@param resource CopilotChat.config.functions.Result +---@return CopilotChat.client.Resource +function M.to_resource(resource) return { - content = content, - filename = url, - filetype = 'text', + name = utils.uri_to_filename(resource.uri), + type = utils.mimetype_to_filetype(resource.mimetype), + data = resource.data, } end ---- Filter embeddings based on the query +--- Process resources based on the query ---@param prompt string ---@param model string ---@param headless boolean ----@param embeddings table ----@return table -function M.filter_embeddings(prompt, model, headless, embeddings) +---@param resources table +---@return table +function M.process_resources(prompt, model, headless, resources) -- If we dont need to embed anything, just return directly - if #embeddings < MULTI_FILE_THRESHOLD then - return embeddings + if #resources < MULTI_FILE_THRESHOLD then + return resources end notify.publish(notify.STATUS, 'Preparing embedding outline') - for _, input in ipairs(embeddings) do - -- Precalculate hash and attributes for caching - local hash = input.filename .. utils.quick_hash(input.content) + -- Get the outlines for each resource + for _, input in ipairs(resources) do + local hash = input.name .. utils.quick_hash(input.data) input._hash = hash - input.filename = input.filename or 'unknown' - input.filetype = input.filetype or 'text' local outline = outline_cache[hash] if not outline then - local outline_text, symbols = get_outline(input.content, input.filetype) + local outline_text, symbols = get_outline(input.data, input.type) if outline_text then outline = { outline = outline_text, @@ -571,16 +550,16 @@ function M.filter_embeddings(prompt, model, headless, embeddings) end -- Rank embeddings by symbols - embeddings = data_ranked_by_symbols(query, embeddings) - log.debug('Ranked data:', #embeddings) - for i, item in ipairs(embeddings) do - log.debug(string.format('%s: %s - %s', i, item.score, item.filename)) + resources = data_ranked_by_symbols(query, resources) + log.debug('Ranked data:', #resources) + for i, item in ipairs(resources) do + log.debug(string.format('%s: %s - %s', i, item.score, item.name)) end -- Prepare embeddings for processing local to_process = {} local results = {} - for _, input in ipairs(embeddings) do + for _, input in ipairs(resources) do local hash = input._hash local embed = embedding_cache[hash] if embed then @@ -591,14 +570,13 @@ function M.filter_embeddings(prompt, model, headless, embeddings) end end table.insert(to_process, { - content = query, - filename = 'query', - filetype = 'raw', + type = 'text', + data = query, }) -- Embed the data and process the results for _, input in ipairs(client:embed(to_process, model)) do - if input.filetype ~= 'raw' then + if input._hash then embedding_cache[input._hash] = input.embedding end table.insert(results, input) diff --git a/lua/CopilotChat/select.lua b/lua/CopilotChat/select.lua index 2254c913..8bef366c 100644 --- a/lua/CopilotChat/select.lua +++ b/lua/CopilotChat/select.lua @@ -1,18 +1,16 @@ ----@class CopilotChat.select.selection +---@class CopilotChat.select.Selection ---@field content string ---@field start_line number ---@field end_line number ---@field filename string ---@field filetype string ---@field bufnr number ----@field diagnostics table? -local utils = require('CopilotChat.utils') local M = {} --- Select and process current visual selection --- @param source CopilotChat.source ---- @return CopilotChat.select.selection|nil +--- @return CopilotChat.select.Selection|nil function M.visual(source) local bufnr = source.bufnr local start_line = unpack(vim.api.nvim_buf_get_mark(bufnr, '<')) @@ -35,18 +33,17 @@ function M.visual(source) return { content = lines_content, - filename = utils.filepath(vim.api.nvim_buf_get_name(bufnr)), + filename = vim.api.nvim_buf_get_name(bufnr), filetype = vim.bo[bufnr].filetype, start_line = start_line, end_line = finish_line, bufnr = bufnr, - diagnostics = utils.diagnostics(bufnr, start_line, finish_line), } end --- Select and process whole buffer --- @param source CopilotChat.source ---- @return CopilotChat.select.selection|nil +--- @return CopilotChat.select.Selection|nil function M.buffer(source) local bufnr = source.bufnr local lines = vim.api.nvim_buf_get_lines(bufnr, 0, -1, false) @@ -54,22 +51,19 @@ function M.buffer(source) return nil end - local out = { + return { content = table.concat(lines, '\n'), - filename = utils.filepath(vim.api.nvim_buf_get_name(bufnr)), + filename = vim.api.nvim_buf_get_name(bufnr), filetype = vim.bo[bufnr].filetype, start_line = 1, end_line = #lines, bufnr = bufnr, } - - out.diagnostics = utils.diagnostics(bufnr, out.start_line, out.end_line) - return out end --- Select and process current line --- @param source CopilotChat.source ---- @return CopilotChat.select.selection|nil +--- @return CopilotChat.select.Selection|nil function M.line(source) local bufnr = source.bufnr local winnr = source.winnr @@ -79,22 +73,19 @@ function M.line(source) return nil end - local out = { + return { content = line, - filename = utils.filepath(vim.api.nvim_buf_get_name(bufnr)), + filename = vim.api.nvim_buf_get_name(bufnr), filetype = vim.bo[bufnr].filetype, start_line = cursor[1], end_line = cursor[1], bufnr = bufnr, } - - out.diagnostics = utils.diagnostics(bufnr, out.start_line, out.end_line) - return out end --- Select and process contents of unnamed register ("). This register contains last deleted, changed or yanked content. --- @param source CopilotChat.source ---- @return CopilotChat.select.selection|nil +--- @return CopilotChat.select.Selection|nil function M.unnamed(source) local bufnr = source.bufnr local start_line = unpack(vim.api.nvim_buf_get_mark(bufnr, '[')) @@ -117,12 +108,11 @@ function M.unnamed(source) return { content = lines_content, - filename = utils.filepath(vim.api.nvim_buf_get_name(bufnr)), + filename = vim.api.nvim_buf_get_name(bufnr), filetype = vim.bo[bufnr].filetype, start_line = start_line, end_line = finish_line, bufnr = bufnr, - diagnostics = utils.diagnostics(bufnr, start_line, finish_line), } end diff --git a/lua/CopilotChat/ui/chat.lua b/lua/CopilotChat/ui/chat.lua index 82af9789..de9b0acf 100644 --- a/lua/CopilotChat/ui/chat.lua +++ b/lua/CopilotChat/ui/chat.lua @@ -13,70 +13,72 @@ function CopilotChatFoldExpr(lnum, separator) return '=' end +local TOOL_PATTERN = '^```?(%w+)%s+tool=(%S+)%s+id=(%S*)$' local HEADER_PATTERNS = { - '%[file:.+%]%((.+)%) line:(%d+)-?(%d*)', - '%[file:(.+)%] line:(%d+)-?(%d*)', + '^```?(%w+)%s+path=(%S+)%s+start_line=(%d+)%s+end_line=(%d+)$', + '^```(%w+)$', } ---@param header? string ----@return string?, number?, number? +---@return string?, string?, number?, number? local function match_header(header) if not header then return end for _, pattern in ipairs(HEADER_PATTERNS) do - local filename, start_line, end_line = header:match(pattern) - if filename then - return utils.filepath(filename), tonumber(start_line) or 1, tonumber(end_line) or tonumber(start_line) or 1 + local type, path, start_line, end_line = header:match(pattern) + if path then + return type, path, tonumber(start_line) or 1, tonumber(end_line) or tonumber(start_line) or 1 + elseif type then + return type, 'block' end end end ----@class CopilotChat.ui.Chat.Section.Block.Header +---@class CopilotChat.ui.chat.Header ---@field filename string ---@field start_line number ---@field end_line number ---@field filetype string ----@class CopilotChat.ui.Chat.Section.Block ----@field header CopilotChat.ui.Chat.Section.Block.Header +---@class CopilotChat.ui.chat.Block +---@field header CopilotChat.ui.chat.Header ---@field start_line number ---@field end_line number ---@field content string? ----@class CopilotChat.ui.Chat.Section +---@class CopilotChat.ui.chat.Section ---@field answer boolean ---@field start_line number ---@field end_line number ----@field blocks table +---@field blocks table ---@field content string? ----@class CopilotChat.ui.Chat : CopilotChat.ui.Overlay +---@class CopilotChat.ui.chat.Chat : CopilotChat.ui.overlay.Overlay ---@field winnr number? ----@field config CopilotChat.config.shared ----@field layout CopilotChat.config.Layout? ----@field sections table ----@field references table +---@field config CopilotChat.config.Shared +---@field sections table +---@field tool_calls table ---@field token_count number? ---@field token_max_count number? +---@field private layout CopilotChat.config.Layout? ---@field private question_header string ---@field private answer_header string ---@field private separator string ---@field private header_ns number ----@field private spinner CopilotChat.ui.Spinner ----@field private chat_overlay CopilotChat.ui.Overlay +---@field private spinner CopilotChat.ui.spinner.Spinner +---@field private chat_overlay CopilotChat.ui.overlay.Overlay local Chat = class(function(self, question_header, answer_header, separator, help, on_buf_create) Overlay.init(self, 'copilot-chat', help, on_buf_create) self.winnr = nil self.sections = {} self.config = {} - self.layout = nil - self.references = {} self.token_count = nil self.token_max_count = nil + self.layout = nil self.question_header = question_header self.answer_header = answer_header self.separator = separator @@ -112,7 +114,7 @@ end --- Get the closest section to the cursor. ---@param type? "answer"|"question" If specified, only considers sections of the given type ----@return CopilotChat.ui.Chat.Section? +---@return CopilotChat.ui.chat.Section? function Chat:get_closest_section(type) if not self:visible() then return nil @@ -139,7 +141,7 @@ function Chat:get_closest_section(type) end --- Get the closest code block to the cursor. ----@return CopilotChat.ui.Chat.Section.Block? +---@return CopilotChat.ui.chat.Block? function Chat:get_closest_block() if not self:visible() then return nil @@ -164,7 +166,7 @@ function Chat:get_closest_block() end --- Get the prompt in the chat window. ----@return CopilotChat.ui.Chat.Section? +---@return CopilotChat.ui.chat.Section? function Chat:get_prompt() if not self:visible() then return @@ -265,7 +267,7 @@ function Chat:overlay(opts) end --- Open the chat window. ----@param config CopilotChat.config.shared +---@param config CopilotChat.config.Shared function Chat:open(config) self:validate() @@ -451,7 +453,7 @@ end --- Clear the chat window. function Chat:clear() self:validate() - self.references = {} + self.tool_calls = nil self.token_count = nil self.token_max_count = nil vim.bo[self.bufnr].modifiable = true @@ -558,16 +560,10 @@ function Chat:render() }) end - -- Parse code blocks if current_section and current_section.answer then - local filetype = line:match('^```(%w+)$') - if filetype and not current_block then - local filename, start_line, end_line = match_header(lines[l - 1]) - if not filename then - filename, start_line, end_line = match_header(lines[l - 2]) - end - filename = filename or 'code-block' - + -- Parse code block headers + local filetype, filename, start_line, end_line = match_header(line) + if filetype and filename and not current_block then current_block = { header = { filename = filename, @@ -577,6 +573,18 @@ function Chat:render() }, start_line = l + 1, } + + local text = string.format('[%s] %s', filetype, filename) + if start_line and end_line then + text = text .. string.format(' lines %d-%d', start_line, end_line) + end + + vim.api.nvim_buf_set_extmark(self.bufnr, self.header_ns, l, 0, { + virt_lines_above = true, + virt_lines = { { { text, 'CopilotChatAnnotationHeader' } } }, + priority = 100, + strict = false, + }) elseif line == '```' and current_block then current_block.end_line = l - 1 current_block.content = @@ -585,6 +593,43 @@ function Chat:render() current_block = nil end end + + -- Parse tool headers + local tool_type, tool_name, tool_id = line:match(TOOL_PATTERN) + if tool_type and tool_name then + local text = string.format('[%s] %s', tool_type, tool_name) + if tool_id then + text = text .. ' ' .. tool_id + end + + vim.api.nvim_buf_set_extmark(self.bufnr, self.header_ns, l, 0, { + virt_lines_above = true, + virt_lines = { { { text, 'CopilotChatAnnotationHeader' } } }, + priority = 100, + strict = false, + }) + end + + -- Parse tool calls + if self.tool_calls then + for _, tool_call in ipairs(self.tool_calls) do + if line:match(string.format('#%s:%s', tool_call.name, vim.pesc(tool_call.id))) then + vim.api.nvim_buf_add_highlight(self.bufnr, self.header_ns, 'CopilotChatAnnotationHeader', l - 1, 0, #line) + + if not utils.empty(tool_call.arguments) then + vim.api.nvim_buf_set_extmark(self.bufnr, self.header_ns, l - 1, 0, { + virt_lines = vim.tbl_map(function(json_line) + return { { json_line, 'CopilotChatAnnotation' } } + end, vim.split(vim.inspect(vim.json.decode(tool_call.arguments)), '\n')), + priority = 100, + strict = false, + }) + end + + break + end + end + end end local last_section = sections[#sections] @@ -598,22 +643,6 @@ function Chat:render() end self:show_help(msg, last_section.start_line - last_section.end_line - 1) - - if not utils.empty(self.references) and self.config.references_display == 'virtual' then - msg = 'References:\n' - for _, ref in ipairs(self.references) do - msg = msg .. ' ' .. ref.name .. '\n' - end - - vim.api.nvim_buf_set_extmark(self.bufnr, self.header_ns, last_section.start_line - 2, 0, { - hl_mode = 'combine', - priority = 100, - virt_lines_above = true, - virt_lines = vim.tbl_map(function(t) - return { { t, 'CopilotChatHelp' } } - end, vim.split(msg, '\n')), - }) - end else self:show_help() end diff --git a/lua/CopilotChat/ui/overlay.lua b/lua/CopilotChat/ui/overlay.lua index 9b70cb5e..a23c022e 100644 --- a/lua/CopilotChat/ui/overlay.lua +++ b/lua/CopilotChat/ui/overlay.lua @@ -1,7 +1,7 @@ local utils = require('CopilotChat.utils') local class = utils.class ----@class CopilotChat.ui.Overlay : Class +---@class CopilotChat.ui.overlay.Overlay : Class ---@field bufnr number? ---@field protected name string ---@field protected help string @@ -19,10 +19,6 @@ local Overlay = class(function(self, name, help, on_buf_create) self.on_hide = nil self.help_ns = vim.api.nvim_create_namespace('copilot-chat-help') - self.hl_ns = vim.api.nvim_create_namespace('copilot-chat-highlights') - vim.api.nvim_set_hl(self.hl_ns, '@diff.plus', { bg = utils.blend_color('DiffAdd', 20) }) - vim.api.nvim_set_hl(self.hl_ns, '@diff.minus', { bg = utils.blend_color('DiffDelete', 20) }) - vim.api.nvim_set_hl(self.hl_ns, '@diff.delta', { bg = utils.blend_color('DiffChange', 20) }) end) --- Show the overlay buffer @@ -38,7 +34,6 @@ function Overlay:show(text, winnr, filetype, syntax, on_show, on_hide) end self:validate() - vim.api.nvim_win_set_hl_ns(winnr, self.hl_ns) text = text .. '\n' self.cursor = vim.api.nvim_win_get_cursor(winnr) @@ -122,7 +117,6 @@ function Overlay:restore(winnr, bufnr) end vim.api.nvim_win_set_buf(winnr, bufnr) - vim.api.nvim_win_set_hl_ns(winnr, 0) if self.cursor then vim.api.nvim_win_set_cursor(winnr, self.cursor) diff --git a/lua/CopilotChat/ui/spinner.lua b/lua/CopilotChat/ui/spinner.lua index 4b69ae44..0f582032 100644 --- a/lua/CopilotChat/ui/spinner.lua +++ b/lua/CopilotChat/ui/spinner.lua @@ -14,7 +14,7 @@ local spinner_frames = { '⠏', } ----@class CopilotChat.ui.Spinner : Class +---@class CopilotChat.ui.spinner.Spinner : Class ---@field bufnr number ---@field status string? ---@field private index number diff --git a/lua/CopilotChat/utils.lua b/lua/CopilotChat/utils.lua index 1be2504b..5a51ff5f 100644 --- a/lua/CopilotChat/utils.lua +++ b/lua/CopilotChat/utils.lua @@ -103,6 +103,24 @@ function M.ordered_map() } end +--- Convert arguments to a table +---@param ... any The arguments +---@return table +function M.to_table(...) + local result = {} + for i = 1, select('#', ...) do + local x = select(i, ...) + if type(x) == 'table' then + for _, v in ipairs(x) do + table.insert(result, v) + end + elseif x ~= nil then + table.insert(result, x) + end + end + return result +end + ---@class StringBuffer ---@field add fun(self:StringBuffer, s:string) ---@field set fun(self:StringBuffer, s:string) @@ -150,26 +168,6 @@ function M.temp_file(text) return temp_file end ---- Blend a color with the neovim background ----@param color_name string The color name ----@param blend number The blend percentage ----@return string? -function M.blend_color(color_name, blend) - local color_int = vim.api.nvim_get_hl(0, { name = color_name }).fg - local bg_int = vim.api.nvim_get_hl(0, { name = 'Normal' }).bg - - if not color_int or not bg_int then - return - end - - local color = { (color_int / 65536) % 256, (color_int / 256) % 256, color_int % 256 } - local bg = { (bg_int / 65536) % 256, (bg_int / 256) % 256, bg_int % 256 } - local r = math.floor((color[1] * blend + bg[1] * (100 - blend)) / 100) - local g = math.floor((color[2] * blend + bg[2] * (100 - blend)) / 100) - local b = math.floor((color[3] * blend + bg[3] * (100 - blend)) / 100) - return string.format('#%02x%02x%02x', r, g, b) -end - --- Return to normal mode function M.return_to_normal_mode() local mode = vim.fn.mode():lower() @@ -180,11 +178,6 @@ function M.return_to_normal_mode() end end ---- Mark a function as deprecated -function M.deprecate(old, new) - vim.deprecate(old, new, '3.0.X', 'CopilotChat.nvim', false) -end - --- Debounce a function function M.debounce(id, fn, delay) if M.timers[id] then @@ -194,21 +187,6 @@ function M.debounce(id, fn, delay) M.timers[id] = vim.defer_fn(fn, delay) end ---- Create key-value list from table ----@param tbl table The table ----@return table -function M.kv_list(tbl) - local result = {} - for k, v in pairs(tbl) do - table.insert(result, { - key = k, - value = v, - }) - end - - return result -end - --- Check if a buffer is valid --- Check if the buffer is not a terminal ---@param bufnr number? The buffer number @@ -246,6 +224,53 @@ function M.filetype(filename) return ft end +--- Get the mimetype from filetype +---@param filetype string? +---@return string +function M.filetype_to_mimetype(filetype) + if not filetype or filetype == '' then + return 'text/plain' + end + if filetype == 'json' or filetype == 'yaml' then + return 'application/' .. filetype + end + if filetype == 'html' or filetype == 'css' then + return 'text/' .. filetype + end + return 'text/x-' .. filetype +end + +--- Get the filetype from mimetype +---@param mimetype string? +---@return string +function M.mimetype_to_filetype(mimetype) + if not mimetype or mimetype == '' then + return 'text' + end + + local out = mimetype:gsub('^text/x%-', '') + out = out:gsub('^text/', '') + out = out:gsub('^application/', '') + out = out:gsub('^image/', '') + out = out:gsub('^video/', '') + out = out:gsub('^audio/', '') + return out +end + +--- Convert a URI to a file name +---@param uri string The URI +---@return string +function M.uri_to_filename(uri) + if not uri or uri == '' then + return uri + end + local ok, fname = pcall(vim.uri_to_fname, uri) + if not ok or M.empty(fname) then + return uri + end + return fname +end + --- Get the file name ---@param filepath string The file path ---@return string @@ -253,13 +278,6 @@ function M.filename(filepath) return vim.fs.basename(filepath) end ---- Get the file path ----@param filename string The file name ----@return string -function M.filepath(filename) - return vim.fn.fnamemodify(filename, ':p:.') -end - --- Generate a UUID ---@return string function M.uuid() @@ -291,6 +309,13 @@ function M.make_string(...) x = vim.inspect(x) else x = tostring(x) + while true do + local new_x = x:gsub('^[^:]+:%d+: ', '') + if new_x == x then + break + end + x = new_x + end end t[#t + 1] = x @@ -413,7 +438,18 @@ M.curl_post = async.wrap(function(url, opts, callback) curl.post(url, args) end, 3) ----@class CopilotChat.utils.scan_dir_opts +local function filter_files(files, max_count) + files = vim.tbl_filter(function(file) + return file ~= '' and M.filetype(file) ~= nil + end, files) + if max_count and max_count > 0 then + files = vim.list_slice(files, 1, max_count) + end + + return files +end + +---@class CopilotChat.utils.ScanOpts ---@field max_count number? The maximum number of files to scan ---@field max_depth number? The maximum depth to scan ---@field glob? string The glob pattern to match files @@ -421,30 +457,19 @@ end, 3) ---@field no_ignore? boolean Whether to respect or ignore .gitignore --- Scan a directory ----@param path string The directory path ----@param opts CopilotChat.utils.scan_dir_opts? The options +---@param path string +---@param opts CopilotChat.utils.ScanOpts? ---@async -M.scan_dir = async.wrap(function(path, opts, callback) +M.glob = async.wrap(function(path, opts, callback) opts = vim.tbl_deep_extend('force', M.scan_args, opts or {}) - local function filter_files(files) - files = vim.tbl_filter(function(file) - return file ~= '' and M.filetype(file) ~= nil - end, files) - if opts.max_count and opts.max_count > 0 then - files = vim.list_slice(files, 1, opts.max_count) - end - - return files - end - -- Use ripgrep if available if vim.fn.executable('rg') == 1 then local cmd = { 'rg' } - if opts.glob then + if opts.pattern then table.insert(cmd, '-g') - table.insert(cmd, opts.glob) + table.insert(cmd, opts.pattern) end if opts.max_depth then @@ -466,7 +491,7 @@ M.scan_dir = async.wrap(function(path, opts, callback) vim.system(cmd, { text = true }, function(result) local files = {} if result and result.code == 0 and result.stdout ~= '' then - files = filter_files(vim.split(result.stdout, '\n')) + files = filter_files(vim.split(result.stdout, '\n'), opts.max_count) end callback(files) @@ -481,15 +506,76 @@ M.scan_dir = async.wrap(function(path, opts, callback) vim.tbl_deep_extend('force', opts, { depth = opts.max_depth, add_dirs = false, - search_pattern = M.glob_to_pattern(opts.glob), + search_pattern = M.glob_to_pattern(opts.pattern), respect_gitignore = not opts.no_ignore, on_exit = function(files) - callback(filter_files(files)) + callback(filter_files(files, opts.max_count)) end, }) ) end, 3) +--- Grep a directory +---@param path string The path to search +---@param opts CopilotChat.utils.ScanOpts? +M.grep = async.wrap(function(path, opts, callback) + opts = vim.tbl_deep_extend('force', M.scan_args, opts or {}) + local cmd = {} + + if vim.fn.executable('rg') == 1 then + table.insert(cmd, 'rg') + + if opts.max_depth then + table.insert(cmd, '--max-depth') + table.insert(cmd, tostring(opts.max_depth)) + end + + if opts.no_ignore then + table.insert(cmd, '--no-ignore') + end + + if opts.hidden then + table.insert(cmd, '--hidden') + end + + table.insert(cmd, '--files-with-matches') + table.insert(cmd, '--ignore-case') + + if opts.pattern then + table.insert(cmd, '-e') + table.insert(cmd, "'" .. opts.pattern .. "'") + end + + table.insert(cmd, path) + elseif vim.fn.executable('grep') == 1 then + table.insert(cmd, 'grep') + table.insert(cmd, '-rli') + + if opts.pattern then + table.insert(cmd, '-e') + table.insert(cmd, "'" .. opts.pattern .. "'") + end + + table.insert(cmd, path) + end + + if M.empty(cmd) then + error('No executable found for grep') + return + end + + vim.print(table.concat(cmd, ' ')) + + vim.system(cmd, { text = true }, function(result) + local files = {} + if result and result.code == 0 and result.stdout ~= '' then + files = filter_files(vim.split(result.stdout, '\n'), opts.max_count) + end + + callback(files) + end) +end, 3) + --- Get last modified time of a file ---@param path string The file path ---@return number? @@ -587,6 +673,46 @@ M.ts_parse = async.wrap(function(parser, callback) end end, 2) +--- Wait for a user input +M.input = async.wrap(function(opts, callback) + local fn = function() + vim.ui.input(opts, function(input) + if input == nil or input == '' then + callback(nil) + return + end + + callback(input) + end) + end + + if vim.in_fast_event() then + vim.schedule(fn) + else + fn() + end +end, 2) + +--- Select an item from a list +M.select = async.wrap(function(choices, opts, callback) + local fn = function() + vim.ui.select(choices, opts, function(item) + if item == nil or item == '' then + callback(nil) + return + end + + callback(item) + end) + end + + if vim.in_fast_event() then + vim.schedule(fn) + else + fn() + end +end, 3) + --- Get the info for a key. ---@param name string ---@param key table @@ -770,40 +896,4 @@ function M.glob_to_pattern(g) return p end ----@class CopilotChat.Diagnostic ----@field content string ----@field start_line number ----@field end_line number ----@field severity string - ---- Get diagnostics in a given range ---- @param bufnr number ---- @param start_line number? ---- @param end_line number? ---- @return table|nil -function M.diagnostics(bufnr, start_line, end_line) - local diagnostics = vim.diagnostic.get(bufnr) - local range_diagnostics = {} - local severity = { - [1] = 'ERROR', - [2] = 'WARNING', - [3] = 'INFORMATION', - [4] = 'HINT', - } - - for _, diagnostic in ipairs(diagnostics) do - local lnum = diagnostic.lnum + 1 - if (not start_line or lnum >= start_line) and (not end_line or lnum <= end_line) then - table.insert(range_diagnostics, { - severity = severity[diagnostic.severity], - content = diagnostic.message, - start_line = lnum, - end_line = diagnostic.end_lnum and diagnostic.end_lnum + 1 or lnum, - }) - end - end - - return #range_diagnostics > 0 and range_diagnostics or nil -end - return M diff --git a/plugin/CopilotChat.lua b/plugin/CopilotChat.lua index 5674d6bc..de5e0158 100644 --- a/plugin/CopilotChat.lua +++ b/plugin/CopilotChat.lua @@ -8,14 +8,29 @@ if vim.fn.has('nvim-' .. min_version) ~= 1 then return end +local group = vim.api.nvim_create_augroup('CopilotChat', {}) + -- Setup highlights -vim.api.nvim_set_hl(0, 'CopilotChatStatus', { link = 'DiagnosticHint', default = true }) -vim.api.nvim_set_hl(0, 'CopilotChatHelp', { link = 'DiagnosticInfo', default = true }) -vim.api.nvim_set_hl(0, 'CopilotChatKeyword', { link = 'Keyword', default = true }) -vim.api.nvim_set_hl(0, 'CopilotChatInput', { link = 'Special', default = true }) -vim.api.nvim_set_hl(0, 'CopilotChatSelection', { link = 'Visual', default = true }) -vim.api.nvim_set_hl(0, 'CopilotChatHeader', { link = '@markup.heading.2.markdown', default = true }) -vim.api.nvim_set_hl(0, 'CopilotChatSeparator', { link = '@punctuation.special.markdown', default = true }) +local function setup_highlights() + vim.api.nvim_set_hl(0, 'CopilotChatHeader', { link = '@markup.heading.2.markdown', default = true }) + vim.api.nvim_set_hl(0, 'CopilotChatSeparator', { link = '@punctuation.special.markdown', default = true }) + vim.api.nvim_set_hl(0, 'CopilotChatStatus', { link = 'DiagnosticHint', default = true }) + vim.api.nvim_set_hl(0, 'CopilotChatHelp', { link = 'DiagnosticInfo', default = true }) + vim.api.nvim_set_hl(0, 'CopilotChatKeyword', { link = 'Keyword', default = true }) + vim.api.nvim_set_hl(0, 'CopilotChatSelection', { link = 'Visual', default = true }) + vim.api.nvim_set_hl(0, 'CopilotChatAnnotation', { link = 'ColorColumn', default = true }) + + local fg = vim.api.nvim_get_hl(0, { name = 'CopilotChatStatus', link = false }).fg + local bg = vim.api.nvim_get_hl(0, { name = 'CopilotChatAnnotation', link = false }).bg + vim.api.nvim_set_hl(0, 'CopilotChatAnnotationHeader', { fg = fg, bg = bg }) +end +vim.api.nvim_create_autocmd('ColorScheme', { + group = group, + callback = function() + setup_highlights() + end, +}) +setup_highlights() -- Setup commands vim.api.nvim_create_user_command('CopilotChat', function(args) @@ -39,10 +54,6 @@ vim.api.nvim_create_user_command('CopilotChatModels', function() local chat = require('CopilotChat') chat.select_model() end, { force = true }) -vim.api.nvim_create_user_command('CopilotChatAgents', function() - local chat = require('CopilotChat') - chat.select_agent() -end, { force = true }) vim.api.nvim_create_user_command('CopilotChatOpen', function() local chat = require('CopilotChat') chat.open() @@ -90,7 +101,7 @@ end, { nargs = '*', force = true, complete = complete_load }) -- with "rooter" plugins, LSP and stuff as vim.fn.getcwd() when -- i pass window number inside doesnt work vim.api.nvim_create_autocmd({ 'VimEnter', 'WinEnter', 'DirChanged' }, { - group = vim.api.nvim_create_augroup('CopilotChat', {}), + group = group, callback = function() vim.w.cchat_cwd = vim.fn.getcwd() end,