Collecting ideas for libkrun 2.x

[slp]

We plan to start working on 2.x on 2026Q1, with the goal of releasing it by 2026Q3. Being a major version, we're free to break the API and the behavior of the library in ways that simply aren't possible on minor releases.

We know for sure we're going to:

• Revamp the whole API to make it more flexible and make the configuration of the microVM completely explicit.
• Provide native Rust bindings.
• De-vendorize rutabaga_gfx, and potentially other crates.

We'd like to invite everyone in our Community to provide ideas of things they would like to see implemented in 2.x. If you have a cool proposal, please let us know here! 😁

[nohajc]

This has been discussed before but I would definitely like if the new version implemented proper cleanup of VM resources so that the entrypoint function doesn't just exit the process but instead gracefully returns to the caller.

I suppose this might already be in the milestone but just wanted to mention it explicitly.

It would simplify some use cases where additional custom cleanup is needed after a VM terminates. As a workaround I currently have to spawn a child process in my solution.

[ggoodman]

I would like to see the option for the host program to have the option to run the microvm and get back a handle to it, instead of the library systematically execing and taking over. Some outcomes this could enable:

1. Ability to pause the vm
2. Ability to ergonomically measure resource usage / perform metering
3. Ability to snapshot the vm
4. ... and I'm sure many more

This would represent somewhat of a philosophical shift but worth considering.

[ggoodman]

I would love it if the host program could supply a memory range to the library that would act as the init binary when using a root virtio-fs.

[slp]

This one is getting in for sure.

[ggoodman]

I think it would be great if the embedding program could expose host filesystem virtio-fs mounts as read-only. A pie-in-the sky wish was if we had further sophistication to have a sort of overlay-style option where we could even have the delta layer in size-constrained memory. In that world, locking it and snapshotting just the delta would be super efficient and it would guard against guests exhausting host disk space when running untrusted workloads.

[handpickencounter]

Not strictly related to libkrun, but for krunvm it would be nice to have the option of mirroring the host system it finds itself in as-is.

This would allow to bypass the OCI spec and rely on more CLI friendly tooling such as bubblewrap/bwrap: https://github.com/containers/bubblewrap

for example bwrap [setup + KVM] -- krunvm do

This would also be friendly for testing and commands/do.rs would be a good rust example.

[slp]

Isn't this basically what muvm does? Or do you mean something else?

[handpickencounter]

I was going to say that they didn't appear interested in serving x86-64 (x86 emulator was on by default) but it looks like some recent commits/PRs are addressing this.

So I don't know.

[mtjhrc]

One of the things, that we currently don't handle well is returning errors from API functions. We are limited to just the errors defined by errno, and just log::error! the details. This can lead to confusion where you don't know why something failed. In general for most API functions the failure is often relatively obvious, but consider something like krun_start_enter which could for a lot of random reasons.

As mentioned we want to also have a Rust API for version 2.x, but we still want to keep the C API. In both versions we should return a rich error type, with our custom list of error codes.

The error type can be something similar to io::Error in Rust, or glib

The error type should contain an integer describing the error code, that can be programmatically checked and a string that is meant to be reported to the user/developer. The string should be optionally heap allocated, because we might want to return a custom formatted string. (If the string is static, it doesn't need to be heap allocated and can be returned based on matching on the error code like with errno). To access the error msg you would call a function which would either lookup the static string in a table or return the formatted error msg.

[ggoodman]

Host-injected, pluggable virtio-fs drivers or hooks

I think it would be quite compelling if the host app were able to register a virtio-fs driver or hooks on a per-vm basis. This would give the host app considerable flexibility in how virtio-fs-powered mounts behaved. A host app could, for example, track changes or implement reliable I/O policies. With a complete interface, the virtio-fs could be entirely virtual and operate from memory or the network.

I think the key values to this would be the following:

1. This is implemented on the host and therefore not something that can be bypassed by privilege escalation or clever work-arounds within the guest.
2. This gives flexibility to embedders of libkrun to explore different concepts without having to fork or get them merged into main.

On the other hand, I acknowledge that this is a pretty huge new surface area to start supporting. Would it be ergonomic / feasible in the c api or just at the rust crate level? I don't know enough of either to have an informed opinion.

[valpackett]

There is a well-established vhost-user protocol for implementing virtio devices in external processes that are portable between hypervisors (see crosvm, cloud hypervisor, and even xen). There's a Rust library fuse-backend-rs for writing filesystems that run as both FUSE and vhost-user-virtiofs.

For Clan.lol we'd love to have a Rustified xdg-document-portal directly attached via vhost-user without mounting on the host, so big +1 here.

[mtjhrc]

Regarding having more in-process hooks - recently I have added some APIs (virtio-gpu display, and virtio-input) that have a user provided vtable - C structs with a bunch of function pointers inside them. Since I wanted to implement a display backend in Rust, I also wanted bindings for that, so we currently have bidirectional bindings (krun_input, krun_display)

For 2.x we would have a Rust first API and I'm hoping that will allow me to automatically generate the bindings more automatically via rust macros. This should allow implementing things like this more easily.

This gives flexibility to embedders of libkrun to explore different concepts without having to fork or get them merged into main.

The main thing I would be worried about exposing something like a full ViritioDevice trait for library users to implement would be having to support this as a stable API, which I don't think we will be ready for at all. It might need to be an "experimantal" API, with less guarantees about the stability (more frequent user breaking changes).