sunshine cannot find encoder, not even software ones

[SuperKenVery]

Describe the bug

sunshine (a game streaming server) cannot find any video encoder, not even software ones.

Steps To Reproduce

Steps to reproduce the behavior:

1. Install sunshine with nix
2. Invoke sunshine from terminal
3. Access Web UI from browser, and some errors are shown at main page, stating that no encoders available, not even software ones.

Expected behavior

Use appropriate hardware encoders. Even if this fails, software encoder should work.

Screenshots

Additional context

Flatpak install works (with hardware encoding).

I'm using an Intel 12th gen CPU with integrated graphics, with no discrete GPU.

Notify maintainers

@devusb Thanks for you great work! It's a pity that this didn't work on my machine.

I'm still a newbie to nix, but I guess that we need to put libraries like ffmpeg and vaapi into wrapProgram so that sunshine can find them. (They are currently only in buildInputs)

Metadata

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.6.2, NixOS, 24.05 (Uakari), 24.05pre552293.5a09cb4b393d`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.1`
 - channels(root): `"nixos"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Priorities

Add a 👍 reaction to issues you find important.

[devusb]

Something changed in this build that broke it -- before this it was working with hardware encoding for me, after it does not. Still working on a bisect to see if I can track down what broke it. If I revert to a generation that uses an older nixpkgs it works again.

FWIW sunshine brings its own vendored ffmpeg which is why it's not in buildInputs anymore -- could be something with that too but will see if I can find anything. Any other help is also appreciated :)

[ReenigneArcher]

FYI... we have several patches in our version of ffmpeg that improve streaming performance for game streaming applications.

Also regarding this specific issue there are a whole lot of errors in that log. I'd start at the top and work your way through them. Likely a lot more at the top as well.

https://docs.lizardbyte.dev/projects/sunshine/en/latest/about/usage.html#linux

[devusb]

@ReenigneArcher thanks -- I actually started using the vendored ffmpeg here with 0.21 to take advantage of those patches -- no longer patching out the pre-built version and replacing with the nixpkgs ffmpeg.

@SuperKenVery I think what happened here was that libglvnd used to be propagated from one of the other inputs but now is not -- I've opened #271352 to add it, which resolves the issues I was having and allows this to work with both software and vaapi encoding for me. The other thing I had to make sure I was doing was giving the binary cap_sys_admin -- I have a quick-and-dirty module in my nix-config here that does it for me in the context of a systemd user unit, but YMMV -- I should probably upstream this into nixpkgs someday :)

[ReenigneArcher]

The setcap is tricky if you need to worry about KMS and X11 because for one it needs to be enabled, and for the other disabled.

I wonder if we're going to face the same thing in our Linux packages. We don't explicitly define libglvnd anywhere. https://docs.lizardbyte.dev/projects/sunshine/en/latest/building/linux.html#requirements

P.S. if you notice anything wrong with upstream that would help with packaging here (or anywhere), PRs would be welcome. Any help is appreciated as we're just a small team.

[devusb]

Resolved by #271352.

[peperunas]

I am going to re-open this issue as I'm having the same problem. Using Sunshine 0.21.0 on NixOS 23.11.

This is the log:

Feb 24 01:24:18 architect sunshine[49469]: [capture] -- [kms]
Feb 24 01:24:18 architect sunshine[49469]: [2024:02:24:01:24:18]: Info: Sunshine version: 0.21.0
Feb 24 01:24:18 architect sunshine[49469]: [2024:02:24:01:24:18]: Error: Environment variable WAYLAND_DISPLAY has not been defined
Feb 24 01:24:18 architect sunshine[49469]: [2024:02:24:01:24:18]: Error: Unable to initialize capture method
Feb 24 01:24:18 architect sunshine[49469]: [2024:02:24:01:24:18]: Error: Platform failed to initialize
Feb 24 01:24:18 architect sunshine[49469]: [2024:02:24:01:24:18]: Info: // Testing for available encoders, this may generate errors. You can safely i>
Feb 24 01:24:18 architect sunshine[49469]: [2024:02:24:01:24:18]: Info: Trying encoder [nvenc]
Feb 24 01:24:18 architect sunshine[49469]: [2024:02:24:01:24:18]: Info: System tray created
Feb 24 01:24:19 architect sunshine[49469]: [2024:02:24:01:24:19]: Info: Encoder [nvenc] failed
Feb 24 01:24:19 architect sunshine[49469]: [2024:02:24:01:24:19]: Info: Trying encoder [vaapi]
Feb 24 01:24:19 architect sunshine[49469]: [2024:02:24:01:24:19]: Info: Encoder [vaapi] failed
Feb 24 01:24:19 architect sunshine[49469]: [2024:02:24:01:24:19]: Info: Trying encoder [software]
Feb 24 01:24:19 architect sunshine[49469]: [2024:02:24:01:24:19]: Info: Encoder [software] failed
Feb 24 01:24:19 architect sunshine[49469]: [2024:02:24:01:24:19]: Fatal: Couldn't find any working encoder
Feb 24 01:24:19 architect sunshine[49469]: [2024:02:24:01:24:19]: Error: Video failed to find working encoder
Feb 24 01:24:19 architect sunshine[49469]: [2024:02:24:01:24:19]: Error: Failed to create client: Daemon not running
Feb 24 01:24:19 architect sunshine[49469]: [2024:02:24:01:24:19]: Info: Configuration UI available at [https://localhost:47990]

Config:

let
  sunshineOverride = (pkgs.sunshine.override { cudaSupport = true; stdenv = pkgs.cudaPackages.backendStdenv; }).overrideAttrs (prev: {
    runtimeDependencies = prev.runtimeDependencies ++ [ pkgs.libglvnd ];
  });
  ...
  in {
    # for KMS >:(
  security.wrappers.sunshine = {
    owner = "root";
    group = "root";
    capabilities = "cap_sys_admin+p";
    source = "${sunshineOverride}/bin/sunshine";
  };
  ...
  }

The bug is present ONLY when giving Sunshine CAP_SYS_ADMIN. I tried to have NVFBC working but failed, and now I'm trying to have KMS working instead.

Note: passing backedStdenv is a fix needed to have Sunshine working on the current state of Nixpkgs. See https://github.com/NixOS/nixpkgs/pull/235655/files

[jottr]

Should this be re-opened @devusb?

[SuperKenVery]

Recently I tried sunshine in nixpkgs unstable again. It finds the encoders, but it cannot pair with devices with PIN. When I enter the PIN on admin web page, it either says my numbers are wrong or right and nothing happens. Haven't debugged further, but flatpak version doesn't have this issue. Have you encountered that?

[devusb]

@jottr the stdenv fix was added in #293938
@SuperKenVery haven't had that issue -- have paired a bunch of times with current version (and in fact the test in #294641 also pairs successfully)

[SuperKenVery]

Thanks, I'll try to investigate my own environment.

[ReenigneArcher]

Pin/pairing was broken in the localization update, but this never made it to a stable release/tag. It was fixed by LizardByte/Sunshine#2324

[TakodaS]

I still have this issue with sunshine 0.23.1 under wayland plasma 6 using nvidia's unfree drivers for a GeForce GTX 980 Ti

❯ nix-shell -p nix-info --run "nix-info -m" 
 - system: `"x86_64-linux"`
 - host os: `Linux 6.6.59, NixOS, 24.11 (Vicuna),  
 - multi-user?: `yes` 
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.24.10`                                                                                                 - nixpkgs: `/etc/nix/inputs/nixpkgs`

EDIT: Looks like ERROR: unable to initialize capture method is the key. See this issue