Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows and bindings support #5

Merged
merged 33 commits into from
Jan 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
153a5c8
initial support for making windows plugins
tocklime Jan 4, 2024
378d0c7
hide the windows generation stuff behind cfg(win).
tocklime Jan 4, 2024
5d0e901
Format, add docstrings, move weak to more descriptive module
novafacing Jan 5, 2024
51f2cd6
Use new weak feature name in examples
novafacing Jan 5, 2024
a0fcc99
Fix link
novafacing Jan 5, 2024
849183f
Fix unused import, update bindings
novafacing Jan 5, 2024
69b2509
Add blocklist for bindings to reduce unused definitions
novafacing Jan 5, 2024
32622cf
Update workflow with all dependencies
novafacing Jan 7, 2024
7a1b54c
Disable space freeing
novafacing Jan 7, 2024
1c82542
Re-enable space shaving
novafacing Jan 7, 2024
d143927
Enable g_free on Windows
novafacing Jan 8, 2024
fa780c2
Test in container
novafacing Jan 8, 2024
b30a5f7
Checkout before build
novafacing Jan 8, 2024
cf1f1a1
Dont strip space for docker building
novafacing Jan 8, 2024
d9bdab5
Fix build then run to avoid missing libtracer.so
novafacing Jan 8, 2024
1f459a1
Escape braces
novafacing Jan 8, 2024
ed0b335
Dont build qemu in workflow
novafacing Jan 9, 2024
46bfd9d
Add dependencies
novafacing Jan 9, 2024
1df37f9
Add linker dep
novafacing Jan 9, 2024
66a63cf
Add ignores and seed img generation
novafacing Jan 9, 2024
bb8c683
Add config line to cloud init
novafacing Jan 9, 2024
576aead
Add working cloud-init config
novafacing Jan 10, 2024
8709fe0
Use powershell
novafacing Jan 10, 2024
bd63c72
Add system syscall trace plugin
novafacing Jan 10, 2024
19e486c
Use Start-Process instead of Start-Job
novafacing Jan 10, 2024
d58ed84
Locating qemu
novafacing Jan 10, 2024
ece4c6f
Use custom msys path
novafacing Jan 10, 2024
28d40ae
Test in powershell
novafacing Jan 10, 2024
01ae201
Fix path to cloud img
novafacing Jan 10, 2024
e688b97
Wait for process to exit
novafacing Jan 10, 2024
917500b
Wait with sleep and increase time before shutdown
novafacing Jan 10, 2024
fab7ac5
Add debug messages
novafacing Jan 10, 2024
30d0d7b
Switch system example to actually do things in system mode, remove SS…
novafacing Jan 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Generated by Cargo
# will have compiled files and executables
target/

# Remove Cargo.lock from gitignore if creating an executable, leave it for libraries
# More information here https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html
Cargo.lock

# These are backup files generated by rustfmt
**/*.rs.bk
1 change: 1 addition & 0 deletions .github/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
logs
1 change: 1 addition & 0 deletions .github/rsrc/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*.yml
38 changes: 38 additions & 0 deletions .github/rsrc/id_rsa
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABlwAAAAdzc2gtcn
NhAAAAAwEAAQAAAYEA2bW01Jm9ujWY0VoDuL3D4XmQ09D1Tkyg98MQoXNg7nZ6IJP2F8PW
N4OHRwrd6/flcDj5c8YfgkwqCryu8f/qq+nlgaQ8dskJIUnaLa3YtVax2KrQu/2zHH6StI
abiQYKM+6L0OPZxwrbX+uSfs/QEAF0KCxcXVDjvXjpvOla+MWunm9B+s11yj+qesSc+MWp
gZKvZYH8FmIIy+P77cnnjDgXx51vHHdO8rI4uLuoNpaMQjqbtpGP3pFYbqAId1lrQSBzVJ
4ybrhZw/+pfzhoXrHDCd4lTd6fihj0IHQkPmuxgd09EZONfsiDGX7iUoTrRvGt9WjCVClz
vjPgCIy/0hpUG4/+1GAtX4V3rN84gT1e+qThFL8GimiPzUXL1abT5YQiWCbN/eJ90kd/Tj
fhsAhD1xOOK8TlHi/28COF+czhSektncm0Wrng5wWYpZkJG7yX4GE5MKWikgCXB4SvCW81
0+Ievgk1g7nL2c8oynYixG2JfE4q3gcFmOf9UwixAAAFiE9S/jRPUv40AAAAB3NzaC1yc2
EAAAGBANm1tNSZvbo1mNFaA7i9w+F5kNPQ9U5MoPfDEKFzYO52eiCT9hfD1jeDh0cK3ev3
5XA4+XPGH4JMKgq8rvH/6qvp5YGkPHbJCSFJ2i2t2LVWsdiq0Lv9sxx+krSGm4kGCjPui9
Dj2ccK21/rkn7P0BABdCgsXF1Q47146bzpWvjFrp5vQfrNdco/qnrEnPjFqYGSr2WB/BZi
CMvj++3J54w4F8edbxx3TvKyOLi7qDaWjEI6m7aRj96RWG6gCHdZa0Egc1SeMm64WcP/qX
84aF6xwwneJU3en4oY9CB0JD5rsYHdPRGTjX7Igxl+4lKE60bxrfVowlQpc74z4AiMv9Ia
VBuP/tRgLV+Fd6zfOIE9Xvqk4RS/Bopoj81Fy9Wm0+WEIlgmzf3ifdJHf0434bAIQ9cTji
vE5R4v9vAjhfnM4UnpLZ3JtFq54OcFmKWZCRu8l+BhOTClopIAlweErwlvNdPiHr4JNYO5
y9nPKMp2IsRtiXxOKt4HBZjn/VMIsQAAAAMBAAEAAAGAB2nBl1zoDgRz0HqcvnRXPHjsyJ
qglLbFIySdLwswR3RpMI1rO3hNlbi0wTN2dqnLciqde17JXUDhzFlsCVVcELgjWoqMrMSI
Cx4yN7yYAMY7wm+AEauoBvoMHamo94WpMKcgc4ejp2x4J3QsegUSXg4nnfPTPhHyXEX3Cw
nE4UxNZ6uNCwjGmFE6SUmDOREVdlIX5vhh7KLwTt2Dqz9VEyQokqFqqnTpTzgoeGla5ydx
jZKEcXczx3nKYaNxQFue/ElJw9KabSAZ3v3T9rzw0z3rP7n0Bvn/xs7+i2cKJhJIkQtPUa
Z1r/GMrE+yp3BMYw5vcqKx+OOvNPkoIiRSzuyJ3qJK4okibtDCkYfPk/FxuCX/7nVlL/Ak
n3CaVWWWGTGOyohW6+MxJ/z16gqgXNy8gwKRnsIEUtpc7RoGIO8MT5fVrMOhlwmJHnPnEm
2/185hraTHLrb8V6suTwCvbzB65e3TSQaJXyMqOyYSdOXM3fmpZrSmHw7y8DqJ74lhAAAA
wDIxqOfvRc7j736AKgp2qDhJftc/mb8osZsbPgApCbVLZmI9Zvy1PFDEcNfDunU06Mue+2
yaWhaeQ180UAqJ4E3vgE1OkBMae+bPyk/v5HeGmZfxvcFhxP87/zMxqARxVLy2ECDS2NR5
A/G5bL4VJatmFJiAa5N137X5FDlctvy5h+ZEhMJdy/LYCJld5mM7EMoa3AKErkSnhdI+mH
v9x34mWSzeB198qGb7QMqvhGLu7ZEVqRAIaAZ5HhkIx8oO0QAAAMEA97+ZMXEbNQFTyFMO
GuT42A3wU0RJmKqRD3wQDeW5Ua/RF+WeaMxAWRZG5Pj8X0p4WAsupJEfU4FZFsQhmu0Kzr
vvTEHmFBmupnYdwmQSDk4SkJLrXNQSpkFoxTdSDZ3/P+WgDAdC1pU1/ClAXS7gQZhWLzCo
IGDIFkUEvcUCxh0Jf8YszFWWb3WZ7lOBFdQCyS6SmvQtyq+M8ghi0Ce+btU3TvpFPRKa0p
LT9vw3LMhjeYidBrzZu/gvWWuhJb3hAAAAwQDg9fyPweE++XeHXtyzqDZIRRw5O/fsEGow
cgSDfII2zJhiPOzO55HuguS92uak7oLsoFx5+Ulw9jdFu+spn2W2+iAaSlg9CTQVSt/Z/W
gyN48rYziYtoC0eKcKNdXoMX+ovAdwRA7LfGVc3MKm1jX+tlJ6lcWT5ZAodGdDX+mzMJfA
8U6K4Z/lgId5D/xZjNMm5n+dZ7rnLcQoUsa7nnrJ6wob4nDDEYDIGx4FvDIyyEupRmesmt
2S9f5i0kz7hNEAAAAQZmVkb3JhQGxvY2FsaG9zdAECAw==
-----END OPENSSH PRIVATE KEY-----
1 change: 1 addition & 0 deletions .github/rsrc/id_rsa.pub
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZtbTUmb26NZjRWgO4vcPheZDT0PVOTKD3wxChc2Dudnogk/YXw9Y3g4dHCt3r9+VwOPlzxh+CTCoKvK7x/+qr6eWBpDx2yQkhSdotrdi1VrHYqtC7/bMcfpK0hpuJBgoz7ovQ49nHCttf65J+z9AQAXQoLFxdUOO9eOm86Vr4xa6eb0H6zXXKP6p6xJz4xamBkq9lgfwWYgjL4/vtyeeMOBfHnW8cd07ysji4u6g2loxCOpu2kY/ekVhuoAh3WWtBIHNUnjJuuFnD/6l/OGhescMJ3iVN3p+KGPQgdCQ+a7GB3T0Rk41+yIMZfuJShOtG8a31aMJUKXO+M+AIjL/SGlQbj/7UYC1fhXes3ziBPV76pOEUvwaKaI/NRcvVptPlhCJYJs394n3SR39ON+GwCEPXE44rxOUeL/bwI4X5zOFJ6S2dybRaueDnBZilmQkbvJfgYTkwpaKSAJcHhK8JbzXT4h6+CTWDucvZzyjKdiLEbYl8TireBwWY5/1TCLE= fedora@localhost
Binary file added .github/rsrc/seed.img
Binary file not shown.
96 changes: 77 additions & 19 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,26 +10,84 @@ env:
CARGO_TERM_COLOR: always

jobs:
build:
test_plugins_linux:
name: Build and Test Plugins (Linux)
runs-on: ubuntu-latest
container: ubuntu:22.04
steps:
- name: Install QEMU User
run: |
apt-get -y update && \
apt-get -y install git curl qemu-user build-essential
- uses: dtolnay/rust-toolchain@nightly
- uses: actions/checkout@v4
- name: Test QEMU Install
run: |
qemu-x86_64 --help
- name: Build and Test Tracer
run: |
cd plugins/tracer
cargo build -r || exit 0
cargo build -r
cargo run -r --bin tracer -- -a /bin/ls -- -lah
cd ../..
- name: Build and Test Tiny
run: |
cd plugins/tiny
cargo build -r
qemu-x86_64 -plugin ../../target/release/libtiny.so /bin/ls -lah
cd ../..

test_plugins_windows:
name: Build and Test Plugins (Windows)
runs-on: windows-latest
env:
# QEMU 8.1.0
QEMU_URL: "https://qemu.weilnetz.de/w64/2023/qemu-w64-setup-20230822.exe"
RUSTUP_URL: "https://win.rustup.rs/x86_64"
FEDORA_CLOUDIMG_URL: "https://download.fedoraproject.org/pub/fedora/linux/releases/39/Cloud/x86_64/images/Fedora-Cloud-Base-39-1.5.x86_64.qcow2"

steps:
- name: Free build space
uses: easimon/maximize-build-space@master
- uses: msys2/setup-msys2@v2
with:
root-reserve-mb: 2048
temp-reserve-mb: 256
swap-size-mb: 1024
remove-dotnet: 'true'
remove-android: 'true'
remove-haskell: 'true'
remove-codeql: 'true'
remove-docker-images: 'true'
- name: Install QEMU Build Dependencies
run: |
sudo apt-get -y install git libglib2.0-dev libfdt-dev \
libpixman-1-dev zlib1g-dev ninja-build
- uses: dtolnay/rust-toolchain@nightly
- uses: actions/checkout@v3
- name: Build
run: cargo build -vv -p tracer
msystem: UCRT64
update: true
install: git mingw-w64-ucrt-x86_64-gcc
location: C:\msys-custom

- name: Download and Install Rust
run: |
$ProgressPreference = 'SilentlyContinue'
Invoke-WebRequest -Uri ${{ env.RUSTUP_URL }} -OutFile rustup-init.exe
./rustup-init.exe --default-toolchain nightly --default-host x86_64-pc-windows-gnu -y

- name: Install QEMU
shell: msys2 {0}
run: |
pacman -Syu --noconfirm
pacman -Sy mingw-w64-ucrt-x86_64-qemu --noconfirm

- name: Test QEMU
run: |
C:\msys-custom\msys64\ucrt64\bin\qemu-system-x86_64.exe --version

- uses: actions/checkout@v4

- name: Download Cloud Image
run: |
$ProgressPreference = 'SilentlyContinue'
Invoke-WebRequest -Uri ${{ env.FEDORA_CLOUDIMG_URL }} -OutFile Fedora-Cloud-Base-39-1.5.x86_64.qcow2
ls

- name: Build and Test Tiny
run: |
cd plugins/tiny-system
cargo build -r
cd ../..
$process = Start-Process PowerShell.exe -NoNewWindow -RedirectStandardOutput out.txt -RedirectStandardError err.txt -PassThru -ArgumentList "-Command", "C:\msys-custom\msys64\ucrt64\bin\qemu-system-x86_64.exe -machine type=q35 -m 2G -nographic -device virtio-net-pci,netdev=net0 -netdev user,id=net0,hostfwd=tcp::2222-:22 -drive if=virtio,format=qcow2,file=Fedora-Cloud-Base-39-1.5.x86_64.qcow2 -drive if=virtio,format=raw,file=.github/rsrc/seed.img -plugin target/release/tiny_system.dll"
echo "Sleeping 180.0 seconds until booted (boot process took 118s first time)"
Start-Sleep -Seconds 180.0
echo "Stopping process"
Stop-Process -Id $process.id
cat out.txt
cat err.txt
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,6 @@ Cargo.lock

# These are backup files generated by rustfmt
**/*.rs.bk
.env
.secrets
*~
7 changes: 5 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,12 @@ resolver = "2"
members = [
"qemu",
"qemu-plugin",
"qemu-plugin/examples/tiny",
"qemu-plugin/examples/tracer",
"qemu-plugin-sys",
"plugins/tiny",
"plugins/tiny-system",
"plugins/tracer",
]
default-members = ["qemu-plugin", "qemu-plugin-sys"]

[workspace.dependencies]
qemu-plugin-sys = { version = "8.1.3-v3", path = "qemu-plugin-sys" }
Expand Down
17 changes: 17 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
FROM ubuntu:22.04

ENV PATH="${PATH}:/root/.cargo/bin"

RUN apt-get -y update && \
apt-get -y install bison flex git curl libglib2.0-dev libfdt-dev \
libpixman-1-dev zlib1g-dev ninja-build build-essential python3 python3-pip python3-venv && \
python3 -m pip install sphinx sphinx_rtd_theme && \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain nightly

COPY . /qemu-rs

WORKDIR /qemu-rs

RUN cargo build -r && \
cargo run -r --bin tracer -- -a /bin/ls -- -lah

20 changes: 18 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ QEMU for Rust, and Rust for QEMU!

This repository provides several QEMU-related crates:

* [qemu](https://github.com/novaafcing/qemu-rs/tree/main/qemu): QEMU binary installer
* [qemu](https://github.com/novafacing/qemu-rs/tree/main/qemu): QEMU binary installer
* [qemu-plugin-sys](https://github.com/novafacing/qemu-rs/tree/main/qemu-plugin-sys): Low level bindings to the QEMU plugin API
* [qemu-plugin](https://github.com/novafacing/qemu-rs/tree/main/qemu-plugin): High level bindings to the QEMU plugin API

Expand All @@ -21,4 +21,20 @@ and instructions) of a program like:

```sh
cargo run -r --bin tracer -- -a /bin/ls -- -lah
```
```

## Installing QEMU

This repository also provides a crate (`qemu`) which builds QEMU from source and
installs Rust wrappers for QEMU as binaries.

You can install QEMU with (add any additional features you need, e.g. `plugins`):

```sh
cargo install qemu@8.1.3-v3 --features=binaries
```

On some systems, particularly BTRFS systems, `/tmp` may not be large enough for the
temporary build directory (QEMU is quite large to build). In this case, create a
directory on your root filesystem (e.g. `$HOME/.cargo/tmp`) and set
`CARGO_TARGET_DIR=$HOME/.cargo/tmp` when running the install command.
13 changes: 13 additions & 0 deletions plugins/tiny-system/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[package]
name = "tiny-system"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["cdylib"]

[dependencies]
qemu-plugin.workspace = true
anyhow = "1.0.75"
ffi = "0.1.0"
ctor = "0.2.6"
58 changes: 58 additions & 0 deletions plugins/tiny-system/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
use anyhow::{anyhow, Result};
use ctor::ctor;
use qemu_plugin::{
plugin::{HasCallbacks, Plugin, Register, PLUGIN},
PluginId,
};
use std::sync::Mutex;

struct TinyTrace {}

impl Plugin for TinyTrace {}
impl Register for TinyTrace {}

impl HasCallbacks for TinyTrace {
fn on_vcpu_init(
&mut self,
id: PluginId,
vcpu_id: qemu_plugin::VCPUIndex,
) -> std::prelude::v1::Result<(), anyhow::Error> {
println!("on_vcpu_init: id: {:?}, vcpu_id: {:?}", id, vcpu_id);
Ok(())
}

fn on_vcpu_idle(
&mut self,
id: PluginId,
vcpu_id: qemu_plugin::VCPUIndex,
) -> std::prelude::v1::Result<(), anyhow::Error> {
println!("on_vcpu_idle: id: {:?}, vcpu_id: {:?}", id, vcpu_id);
Ok(())
}

fn on_vcpu_exit(
&mut self,
id: PluginId,
vcpu_id: qemu_plugin::VCPUIndex,
) -> std::prelude::v1::Result<(), anyhow::Error> {
println!("on_vcpu_exit: id: {:?}, vcpu_id: {:?}", id, vcpu_id);
Ok(())
}

fn on_vcpu_resume(
&mut self,
id: PluginId,
vcpu_id: qemu_plugin::VCPUIndex,
) -> std::prelude::v1::Result<(), anyhow::Error> {
println!("on_vcpu_resume: id: {:?}, vcpu_id: {:?}", id, vcpu_id);
Ok(())
}
}

#[ctor]
fn init() {
PLUGIN
.set(Mutex::new(Box::new(TinyTrace {})))
.map_err(|_| anyhow!("Failed to set plugin"))
.expect("Failed to set plugin");
}
File renamed without changes.
File renamed without changes.
35 changes: 35 additions & 0 deletions plugins/tracer/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
[package]
name = "tracer"
version = "0.1.0"
edition = "2021"

[lib]
name = "tracer"
crate-type = ["cdylib", "lib"]

[dependencies]
anyhow = "1.0.75"
ctor = "0.2.6"
qemu-plugin = { workspace = true, features = ["unix-weak-link"] }
serde = { version = "1.0.193", features = ["derive"] }
serde_cbor = "0.11.2"
tokio = { version = "1.35.0", features = ["full"] }
typed-builder = "0.18.0"
yaxpeax-x86 = "1.2.2"

# Dependencies only used by this crate's `tracer` binary. We do not use dev-dependencies
# because they cannot be optional.
clap = { version = "4.4.11", features = ["derive", "string"] }
# Enable the `qemu` feature to build and install QEMU with the `qemu` crate instead
# of trying to use the system QEMU.
qemu = { workspace = true, features = [
"plugins",
"debug-info",
], optional = true }
memfd-exec = { version = "0.2.1", optional = true }
rand = "0.8.5"
serde_json = "1.0.108"

[features]
qemu = ["dep:memfd-exec", "dep:qemu"]
default = []
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ name = "tracer"
crate-type = ["cdylib", "lib"]

[dependencies]
qemu = { workspace = true, features = ["plugins", "debug-info"] }
qemu-plugin = { workspace = true, features = ["weak"] }
qemu-plugin = { workspace = true, features = ["unix-weak-link"] }
anyhow = "1.0.75"
ffi = "0.1.0"
ctor = "0.2.6"
Expand All @@ -22,3 +21,8 @@ serde = { version = "1.0.193", features = ["derive"] }
memfd-exec = "0.2.1"
serde_json = "1.0.108"
yaxpeax-x86 = "1.2.2"

# Enable the `qemu` feature to build and install QEMU with the `qemu` crate instead
# of trying to use the system QEMU.
qemu = { workspace = true, features = ["plugins", "debug-info"], optional = true }

Loading