Rust/Cargo and nvim-dap/codelldb interaction

[DerekSauer]

Hello,

I am unsure if this is a bug, functionality that doesn't exist, I am just using it wrong, or codelldb is actually at fault so I started a discussion rather than filing an issue.

I've got nvim-dap working well to debug any executable I point it at using the following config, based on the Wiki's install instructions for codelldb, with the extension install via Mason:

local dap = require("dap")

-- CodeLLDB debug adapter location
codelldb_path = vim.fn.stdpath("data") .. "/mason/packages/codelldb/extension/adapter/codelldb"
if vim.fn.has("win32") then
    codelldb_path = vim.fn.stdpath("data") .. "/mason/packages/codelldb/extension/adapter/codelldb.exe"
end

-- Configure LLDB adapter
dap.adapters.lldb = {
    type = "server",
    port = "${port}",
    executable = {
        command = codelldb_path,
        args = { "--port", "${port}" },
        detached = false,
    }
}

-- Default debug configuration for C, C++
dap.configurations.c = {
    {
        name = "Debug an Executable",
        type = "lldb",
        request = "launch",
        program = function()
            return vim.fn.input("Path to executable: ", vim.fn.getcwd() .. '/', "file")
        end,
        cwd = "${workspaceFolder}",
        stopOnEntry = false,
    },
}

dap.configurations.cpp = dap.configurations.c
dap.configurations.rust = dap.configurations.c

-- Override default configurations with `launch.json`
require("dap.ext.vscode").load_launchjs(".nvim/launch.json", { lldb = { "c", "cpp", "rust" } })

I would prefer to define debug configurations per project using a launch.json, hence the last line in my config. I am using the following launch.json in a simple project lifted from my working vsCode workspace:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Debug `Test App`",
            "type": "lldb",
            "request": "launch",
            "program": "${cargo:program}",
            "cargo": {
                "args": [
                    "build"
                ],
                "problemMatcher": "$rustc"
            },
            "cwd": "${workspaceFolder}",
            "sourceLanguages": [
                "rust"
            ],
            "stopOnEntry": false
        }
    ]
}

This crate only has a single executable and no libs so a simple cargo build prior to launching the debugger will build the app. I know nvim is parsing the file as it appears in the list of debuggables when launching the debugger.

When using the codelldb extension in vsCode it interprets the Cargo section of launch.json to run Cargo with the passed in arguments and other options, and expands the ${cargo:program} placeholder with the location of the binary to debug as retrieved from Cargo. This makes it trivial to create a launch.json to debug a variety of build targets including tests, examples and multiple binaries in the same workspace. Codelldb's documentation for this functionality can be found here.

Unfortunately when launching the debugger in Neovim I get the following error:

Error on launch: unable to find executable for '${cargo:\program}.exe'

The placeholder is not being expanded. There is no delay before attempting to launch the debugger so it would seem the Cargo section of the file is not being run either (Rust has rather long build times). Omitting the program field to allow codelldb to fill it in automatically from Cargo also fails with:

Error on launch: "program" property is required for launch

Unfortunately the dap.log file is empty so there is no more detailed error information, and nvim's own log file is not reporting any irregularities either which makes me suspicious that whatever is going on is happening on codelldb's side of things rather than nvim's.

Would anyone know if there is a way to get this working or if I should look at a more involved solution? I am aware of rust-tools.nvim but it seems to conflict with another plugin I am using to configure the variety of LSPs I use (lsp-zero.nvim).

[mfussenegger]

The ${cargo:program} placeholder is not supported by nvim-dap. It's specific to the vscode extension:

https://github.com/vadimcn/vscode-lldb/blob/bc36db0bafd1b77d1cc9aa2dc9d69b221c7d4ae8/extension/main.ts#L397

Given that nvim-dap aims to be debug-adapter agnostic it's out of scope for the core, but would have to go into an extension

There is currently no first-class extension points specifically to place-holders, but I think it should be possible to use the enrich_config mechanism of the adapter definition to resolve custom placeholders.

Along the lines of:

local function expand_cargo(option)
  if option == "${cargo:program}" then
    -- ...
  end
  return option
end

dap.adapters.codelldb = {
  type = 'server',
  port = "${port}",
  executable = {
    command = HOME .. '/apps/codelldb/extension/adapter/codelldb',
    args = {"--port", "${port}"},
  },
  enrich_config = function(config, on_config)
    on_config(vim.tbl_map(expand_cargo, config))
  end
}

[DerekSauer]

That sounds like a good idea. I'll start experimenting and update the thread if I get a something working. Thank you very much for your help.

[DerekSauer]

Been a while. Life got in the way of playing around with nvim-dap. Finally got some time to work on this issue and got a minimum viable product functioning. @mfussenegger's suggestion to use the adapter's enrich_config field was excellent and allowed me to achieve what I desired; thank you for your advice.

I am hardly a Lua expert so the following code is certainly not clean or idiomatic but it does work (on my machine, heh) and I know even less about the workings of Neovim under the hood. The LLDB adapter is configured as the wiki suggested, with the enrich_config field calling out to an additional function if the configuration loaded from launch.json includes a cargo section.

-- Get the path to `codelldb` installed by Mason.nvim
local codelldb_path = require("mason-registry").get_package("codelldb"):get_install_path() .. "/extension"
local codelldb_bin = codelldb_path .. "/adapter/codelldb"

-- Configure the LLDB adapter
dap.adapters.codelldb = {
  type = "server",
  port = "${port}",
  executable = {
    command = codelldb_bin,
    args = { "--port", "${port}" },
  },
  enrich_config = function(config, on_config)
    -- If the configuration(s) in `launch.json` contains a `cargo` section
    -- send the configuration off to the cargo_inspector.
    if config["cargo"] ~= nil then on_config(cargo_inspector(config)) end
  end,
}

The cargo_inspector() then runs Cargo with any additional args and environment variables defined in the launch.json and extracts the location of the binary to debug, filling in the program field of the configuration. This function is rather long and rambling but I kept tripping over myself dealing with the asynchronous operation of vim.fn.jobstart() and breaking functionality into supporting functions was complicating affairs. The last half of the function, building a sourcemap, is not strictly necessary but allows the debugger to step into the source for Rust's standard library rather than emitting assembly.

-- Parse the `cargo` section of a DAP configuration and add any needed
-- information to the final configuration to be handed back to the adapter.
-- E.g.: When debugging a test, cargo generates a random executable name.
-- We need to ask cargo for the name and add it to the `program` config field
-- so LLDB can find it.
local function cargo_inspector(config)
  local final_config = vim.deepcopy(config)

  -- Create a buffer to receive compiler progress messages
  local compiler_msg_buf = vim.api.nvim_create_buf(false, true)
  vim.api.nvim_buf_set_option(compiler_msg_buf, "buftype", "nofile")

  -- And a floating window in the corner to display those messages
  local window_width = math.max(#final_config.name + 1, 50)
  local window_height = 12
  local compiler_msg_window = vim.api.nvim_open_win(compiler_msg_buf, false, {
    relative = "editor",
    width = window_width,
    height = window_height,
    col = vim.api.nvim_get_option "columns" - window_width - 1,
    row = vim.api.nvim_get_option "lines" - window_height - 1,
    border = "rounded",
    style = "minimal",
  })

  -- Let the user know what's going on
  vim.fn.appendbufline(compiler_msg_buf, "$", "Compiling: ")
  vim.fn.appendbufline(compiler_msg_buf, "$", final_config.name)
  vim.fn.appendbufline(compiler_msg_buf, "$", string.rep("=", window_width - 1))

  -- Instruct cargo to emit compiler metadata as JSON
  local message_format = "--message-format=json"
  if final_config.cargo.args ~= nil then
    table.insert(final_config.cargo.args, message_format)
  else
    final_config.cargo.args = { message_format }
  end

  -- Build final `cargo` command to be executed
  local cargo_cmd = { "cargo" }
  for _, value in pairs(final_config.cargo.args) do
    table.insert(cargo_cmd, value)
  end

  -- Run `cargo`, retaining buffered `stdout` for later processing,
  -- and emitting compiler messages to to a window
  local compiler_metadata = {}
  local cargo_job = vim.fn.jobstart(cargo_cmd, {
    clear_env = false,
    env = final_config.cargo.env,
    cwd = final_config.cwd,

    -- Cargo emits compiler metadata to `stdout`
    stdout_buffered = true,
    on_stdout = function(_, data) compiler_metadata = data end,

    -- Cargo emits compiler messages to `stderr`
    on_stderr = function(_, data)
      local complete_line = ""

      -- `data` might contain partial lines, glue data together until
      -- the stream indicates the line is complete with an empty string
      for _, partial_line in ipairs(data) do
        if string.len(partial_line) ~= 0 then complete_line = complete_line .. partial_line end
      end

      if vim.api.nvim_buf_is_valid(compiler_msg_buf) then
        vim.fn.appendbufline(compiler_msg_buf, "$", complete_line)
        vim.api.nvim_win_set_cursor(compiler_msg_window, { vim.api.nvim_buf_line_count(compiler_msg_buf), 1 })
        vim.cmd "redraw"
      end
    end,

    on_exit = function(_, exit_code)
      -- Cleanup the compile message window and buffer
      if vim.api.nvim_win_is_valid(compiler_msg_window) then
        vim.api.nvim_win_close(compiler_msg_window, { force = true })
      end

      if vim.api.nvim_buf_is_valid(compiler_msg_buf) then
        vim.api.nvim_buf_delete(compiler_msg_buf, { force = true })
      end

      -- If compiling succeeed, send the compile metadata off for processing
      -- and add the resulting executable name to the `program` field of the final config
      if exit_code == 0 then
        local executable_name = parse_cargo_metadata(compiler_metadata)
        if executable_name ~= nil then
          final_config.program = executable_name
        else
          vim.notify(
            "Cargo could not find an executable for debug configuration:\n\n\t" .. final_config.name,
            vim.log.levels.ERROR
          )
        end
      else
        vim.notify("Cargo failed to compile debug configuration:\n\n\t" .. final_config.name, vim.log.levels.ERROR)
      end
    end,
  })

  -- Get the rust compiler's commit hash for the source map
  local rust_hash = ""
  local rust_hash_stdout = {}
  local rust_hash_job = vim.fn.jobstart({ "rustc", "--version", "--verbose" }, {
    clear_env = false,
    stdout_buffered = true,
    on_stdout = function(_, data) rust_hash_stdout = data end,
    on_exit = function()
      for _, line in pairs(rust_hash_stdout) do
        local start, finish = string.find(line, "commit-hash: ", 1, true)

        if start ~= nil then rust_hash = string.sub(line, finish + 1) end
      end
    end,
  })

  -- Get the location of the rust toolchain's source code for the source map
  local rust_source_path = ""
  local rust_source_job = vim.fn.jobstart({ "rustc", "--print", "sysroot" }, {
    clear_env = false,
    stdout_buffered = true,
    on_stdout = function(_, data) rust_source_path = data[1] end,
  })

  -- Wait until compiling and parsing are done
  -- This blocks the UI (except for the :redraw above) and I haven't figured
  -- out how to avoid it, yet
  -- Regardless, not much point in debugging if the binary isn't ready yet
  vim.fn.jobwait { cargo_job, rust_hash_job, rust_source_job }

  -- Enable visualization of built in Rust datatypes
  final_config.sourceLanguages = { "rust" }

  -- Build sourcemap to rust's source code so we can step into stdlib
  rust_hash = "/rustc/" .. rust_hash .. "/"
  rust_source_path = rust_source_path .. "/lib/rustlib/src/rust/"
  if final_config.sourceMap == nil then final_config["sourceMap"] = {} end
  final_config.sourceMap[rust_hash] = rust_source_path

  -- Cargo section is no longer needed
  final_config.cargo = nil

  return final_config
end

Parsing Cargo's output is rather simple since it composed of JSON tables.

-- After extracting cargo's compiler metadata with the cargo inspector
-- parse it to find the binary to debug
local function parse_cargo_metadata(cargo_metadata)
  -- Iterate backwards through the metadata list since the binary
  -- we're interested will be near the end (usually second to last)
  for i = 1, #cargo_metadata do
    local json_table = cargo_metadata[#cargo_metadata + 1 - i]

    -- Some metadata lines may be blank, skip those
    if string.len(json_table) ~= 0 then
      -- Each matadata line is a JSON table,
      -- parse it into a data structure we can work with
      json_table = vim.fn.json_decode(json_table)

      -- Our binary will be the compiler artifact with an executable defined
      if json_table["reason"] == "compiler-artifact" and json_table["executable"] ~= vim.NIL then
        return json_table["executable"]
      end
    end
  end

  return nil
end

It all seems to work fine and adds no noticeable cost to starting the debugger other than waiting for Rust's sometimes excessive build times. It even allows the debugging of test builds defined in launch.json, such as:

{
      "type": "codelldb",
      "request": "launch",
      "name": "Debug tests 'nvim_test'",
      "cargo": {
        "args": ["test", "--no-run"]
      },
      "cwd": "${workspaceFolder}"
    }

Thank you again for your assistance @mfussenegger !

[mfussenegger]

Maybe worth putting in an extension or contributing to https://github.com/simrat39/rust-tools.nvim ?