Build EBPF module for opensnitch, at present selecting the ebpf monitor method fails.

[Slime90]

Describe the bug

The opensnitch nixos module allows the selection of the process monitor method, and currently the most preferred option ebpf is listed as valid. When configured via the module option, the service fails back to the "proc" process monitor method, since it cannot find the opensnitch.o ebpf module which is required for it to funciton.

Steps To Reproduce

Steps to reproduce the behavior:

1. set services.opensnitch.settings.ProcMonitorMethod = "ebpf";
2. check the journal logs for the message "open /etc/opensnitchd/opensnitch.o: no such file or directory"

Expected behavior

I would expect that if the "ebpf" ProcMonitorMethod is utilized as documented, that the module would be installed.

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

I have taken a stab at building the module based on the opensnitch source and my current kernel source. The module builds successfully and properly identifies kworker threads which the other methods cannot accomplish. For example, kernel wireguard is silently dropped by opensnitch unless ebpf is used. Below is what I have successfully building:

{ lib
, pkgs
, stdenv
, fetchFromGitHub
, pkg-config
, clang-tools
, llvmPackages
, libllvm
, elfutils
, flex
, bison
, libressl
, bc
, rsync
, python3
, which
, zlib
}:

stdenv.mkDerivation rec {
  pname = "opensnitch_ebpf";
  version = "1.5.2";

  srcs = [
    (pkgs.opensnitch.src.overrideAttrs (old: { name = "opensnitch-src"; }))
    (pkgs.linuxPackages_zen.kernel.src.overrideAttrs (old: { name = "kernel-src"; }))
  ];

  sourceRoot = ".";

  nativeBuildInputs = [
    pkg-config
    clang-tools
    llvmPackages.clang
    libllvm # llvm-strip
    elfutils
    flex
    bison
    libressl
    bc
    rsync
    python3
    which
    zlib
  ];

  #https://aur.archlinux.org/cgit/aur.git/?h=opensnitch-ebpf-module-git
  #docs in the repo for patching/building aren't 100 percent up to date

  patchPhase = ''
    runHook prePatch
    substituteInPlace kernel-src/tools/lib/bpf/bpf_helpers.h \
      --replace 'struct bpf_map_def' 'struct bpf_map_defold'
    substituteInPlace opensnitch-src/ebpf_prog/Makefile \
      --replace '/bin/rm' 'rm'
      runHook postPatch
  '';

  configurePhase = ''
    runHook preConfigure
    ebpf="opensnitch-src/ebpf_prog"
    cp $ebpf/opensnitch*.c $ebpf/common*.h $ebpf/Makefile kernel-src/samples/bpf
    patchShebangs .
    cd kernel-src
    ( set +o pipefail; yes "" | make oldconfig )
    make prepare
    runHook postConfigure
  '';

  buildPhase = ''
    runHook preBuild
    make headers_install
    cd samples/bpf && make
    objdump -h opensnitch*.o
    llvm-strip -g opensnitch*.o
    runHook postBuild
  '';

  installPhase = ''
    runHook preInstall
    mkdir -p $out/etc/opensnitchd
    install -Dm644 opensnitch.o $out/etc/opensnitchd/
    runHook postInstall
  '';
}

Notify maintainers

@onny @raboof @zowoq @pennae

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

❯ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.0.10-zen2, NixOS, 22.11 (Raccoon), 22.11.20230417.cd749f5`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.11.1`

[onny]

Oh nice that you've managed to package the kernel module. Would you like to open a pull request for it?

[Slime90]

Oh nice that you've managed to package the kernel module. Would you like to open a pull request for it?

@onny It isn't 100 percent ready to go, there are a few issues.

1. Should this even be a separate package? It seems to me that it could either be its own package, which could be conditionally included in the opensnitch nixos module if the ebpf method is chosen. The other way could be to integrate this code into the opensnitch package itself.
2. The package would ideally be compiled utilizing the specific kernel sources of the users nixos configuration. Is it possible to add some code to the opensnitch nixos module which could automatically use whatever kernel src the user has configured?
3. The resulting opensnitch.o file has to be installed into /etc/opensnitchd, which would need to be done in the nixos module using environment.etc

Let me know what you all think about these thoughts. I am fairly new to packaging with nix, and feel a little over my head with how to implement this, so at this time I don't have a PR with a concrete suggestion of what to do.

[onny]

@Slime90 thank you for your thoughts about this.

I would suggest packaging at in a new file inside the opensnitch directory, for example pkgs/tools/networking/opensnitch/module.nix.

Really don't know yet how these kernel modules are packaged in Nix but it would be nice if it would choose the correct kernel headers and sources.

We could install this module by default or only if its configured in the module. This might also depend on how other packages are doing this.

Maybe we don't have to place it in /etc and could configure OpenSnitch to look at an other directory for the module.

[Slime90]

@onny Thanks for the followup.

Offhand, the location in the opensnitch project dir structure makes perfect sense.

Looking around, I see I think a good representative example of a third party kernel module, and its inclusion in a nixpkg project. Take a look at:

nixos/modules/programs/droidcam.nix

pkgs/os-specific/linux/v4l2loopback/default.nix

Finally you will be able to see that kernel modules get added to: pkgs/top-level/linux-kernels.nix instead of pkgs/top-level/all-packages.nix

Of special note is the use of kernel.moduleBuildDependencies as a native build dependency.

My confusion with all of this is that even though we are buliding an ebpf module, we are not loading it into the kernel config using boot.kernelModules , this ebpf module is called by the opensnitch application at run time. Maybe you can take a look at some of the stuff above and let me know.

[Slime90]

The below now builds with nix build -L .#linuxPackages_zen.opensnitch-ebpf and nix build -L .#linuxPackages.opensnitch-ebpf after being added to pkgs/top-level/linux-kernels.nix. I tried a few other newer kernel versions as well.

This should mean that the config option in the nixos module for opensnitch should in theory be able to install the package with the boot.extraModulePackages = config.boot.kernelPackages.opensnitch-ebpf; option set in the configuration, although I have not yet tried importing this local nixpkgs source into my flake to actually test it.

At this point I think the ebpf procmon option in the service module can conditionally set config.boot.extraModulePackages, and perhaps use environment.etc to link it to the correct path

We could certainly open an issue with the maintainer of Opensnitch to add a flag for the daemon which would allow specifying a path to the ebpf file, which could then just be set to the store path in the future. I might try to do that this week, since the docs state that the /etc path will be removed in the future from the code. I have checked the release notes from the 1.6.0 release candidates, and the new preferred path will be /usr/lib/opensnitchd/ebpf/ but it does look for now like the /etc path is being retained for the foreseeable future as a fallback: evilsocket/opensnitch@474a637

{ lib
, kernel
, pkgs
, stdenv
, fetchFromGitHub
, pkg-config
, clang-tools
, llvmPackages
, libllvm
, elfutils
, flex
, bison
, libressl
, bc
, rsync
, python3
, which
, zlib
, ...
}:


stdenv.mkDerivation rec {
  pname = "opensnitch_ebpf";
  version = "1.5.2-bump-${kernel.version}";

  srcs = [
    kernel.src
    (pkgs.opensnitch.src.overrideAttrs (old: { name = "opensnitch-source"; }))
  ];

  sourceRoot = ".";

  nativeBuildInputs = [
    pkg-config
    clang-tools
    llvmPackages.clang
    libllvm # llvm-strip
    elfutils
    flex
    bison
    libressl
    bc
    rsync
    python3
    which
    zlib
  ];

  #https://aur.archlinux.org/cgit/aur.git/?h=opensnitch-ebpf-module-git
  #docs in the repo for patching/building aren't 100 percent up to date

  patchPhase = ''
    runHook prePatch
    substituteInPlace ${kernel.name}/tools/lib/bpf/bpf_helpers.h \
      --replace 'struct bpf_map_def' 'struct bpf_map_defold'
    substituteInPlace opensnitch-source/ebpf_prog/Makefile \
      --replace '/bin/rm' 'rm'
      runHook postPatch
  '';

  configurePhase = ''
    runHook preConfigure
    ebpf="opensnitch-source/ebpf_prog"
    cp $ebpf/opensnitch*.c $ebpf/common*.h $ebpf/Makefile ${kernel.name}/samples/bpf
    patchShebangs .
    cd ${kernel.name}
    ( set +o pipefail; yes "" | make oldconfig )
    make prepare
    runHook postConfigure
  '';

  buildPhase = ''
    runHook preBuild
    make headers_install
    cd samples/bpf && make
    objdump -h opensnitch*.o
    llvm-strip -g opensnitch*.o
    runHook postBuild
  '';

  installPhase = ''
    runHook preInstall
    mkdir -p $out/etc/opensnitchd
    install -Dm644 opensnitch.o $out/etc/opensnitchd/
    runHook postInstall
  '';
}

[onny]

Thank you @Slime90, i started creating a PR with your changes. I'll add you as co-author but also feel free to open a new PR if you want #229627

Further I opened an upstream ticket asking for a configuration option for the kernel module path evilsocket/opensnitch#928

[Slime90]

Should a new issue be created for actually modifying the opensnitch nixos module to conditionally include the extra module package, as well as link to etc?

I was thinking we could try

environment.etc."opensnitchd/opensnitch.o".source = "${config.boot.kernelPackages.opensnitch-ebpf}/etc/opensnitchd/opensnitch.o";

Is that valid?