NixOS requires other distro to attach

[573]

[wsl2]
memory=8GB   # Limits VM memory in WSL 2 up to 3GB
processors=2 # Makes the WSL 2 VM use two virtual processors

$ uname -sr
Linux 5.15.146.1-microsoft-standard-WSL2

admin powershell:

usbipd bind --busid 1-2
usbipd: info: Device with busid '1-2' was already shared.

non-admin powershell:

usbipd list
Connected:
BUSID  VID:PID    DEVICE                                                        STATE
1-2    04a9:220e  CanoScan                                                      Shared

usbipd attach --wsl --busid 1-2
usbipd: info: Using WSL distribution 'NixOS-Groundtruth' to attach; the device will be available in all WSL 2 distributions.
usbipd: error: WSL kernel is not USBIP capable; update with 'wsl --update'.

wsl -v
WSL-Version: 2.1.5.0
Kernelversion: 5.15.146.1-2
WSLg-Version: 1.0.60
MSRDC-Version: 1.2.5105
Direct3D-Version: 1.611.1-81528511
DXCore-Version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows-Version: 10.0.19045.4291

Using custom kernel https://github.com/LGUG2Z/custom-wsl2-linux-kernel doesn't change anything.

[573]

Steps using Troubleshooting so far

# non-admin pwsh
winget uninstall usbipd-win
winget install --interactive --exact dorssel.usbipd-win -v 3.1.0   # this worked so far but has firewall rule warning

# admin pwsh
usbipd bind --busid 1-2

# non-admin pwsh
usbipd wsl attach -a --busid 1-2
usbipd: info: Using default WSL distribution 'NixOS-Groundtruth'; specify the '--distribution' option to select a different one.
usbipd: warning: A third-party firewall may be blocking the connection; ensure TCP port 3240 is allowed.
usbipd: info: Starting endless attach loop; press Ctrl+C to quit.

# admin pwsh
sc.exe stop usbipd
 usbipd server Logging:LogLevel:Default=Trace
dbug: Microsoft.Extensions.Hosting.Internal.Host[1]
      Hosting starting
dbug: Usbipd.PcapNg[1000]
      usbipd:PcapNg:Path = ''
dbug: Usbipd.Server[1000]
      3.1.0+66.Branch.master.Sha.47e0500785e1ad4bd6574da02dae33744ea23f52
dbug: Usbipd.Server[1000]
      usbipd:TcpKeepAliveInterval = 500 ms
dbug: Usbipd.Server[1000]
      usbipd:TcpKeepAliveTime = 10000 ms
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Production
info: Microsoft.Hosting.Lifetime[0]
      Content root path: C:\Program Files\usbipd-win\
dbug: Microsoft.Extensions.Hosting.Internal.Host[2]
      Hosting started
dbug: Usbipd.Server[1000]
      new connection from 127.0.0.1
dbug: Usbipd.ConnectedClient[1000]
      Received opcode: OP_REQ_IMPORT
dbug: Usbipd.ConnectedClient[1000]
      Claiming took 10 ms
info: Usbipd.ConnectedClient[1]
      Client 127.0.0.1 claimed device at 1-2 (USB\VID_04A9&PID_220E\5&665DA22&0&2).
dbug: Usbipd.AttachedClient[1000]
      Masked USB_CONFIG_REMOTE_WAKEUP
dbug: Usbipd.AttachedClient[1000]
      Masked USB_CONFIG_REMOTE_WAKEUP
dbug: Usbipd.AttachedClient[1000]
      Trapped SET_CONFIGURATION: 1

# non-admin pwsh (usbipd wsl attach -a --busid 1-2 still running)
usbip: error: tcp connect
usbip: error: Attach Request for 1-2 failed - Device in error state
usbip: error: Attach Request for 1-2 failed - Device not found
Attached
Detached
usbip: error: tcp connect
Attached

# again in wsl now
lsusb
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 04a9:220e Canon, Inc. CanoScan N1240U/LiDE 30
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

[dorssel]

usbipd: info: Using WSL distribution 'NixOS-Groundtruth' to attach; the device will be available in all WSL 2 distributions.
usbipd: error: WSL kernel is not USBIP capable; update with 'wsl --update'.

I'm sure your kernel is WSL capable. It's just the test for it that fails due to an uncommon distro. To solve it, you could simply install another distro (e.g. Ubuntu) and use that to attach. The device will be available in every distro.

However, I would like to find out why the test fails on your distro. This is the command that fails:

wsl --distribution NixOS-Groundtruth --user root --cd / --exec cat /sys/devices/platform/vhci_hcd.0/status

What is the output if you run that in a (Windows) command prompt?

[573]

@dorssel wsl --distribution NixOS-Groundtruth --user root --cd / --exec /run/current-system/sw/bin/cat /sys/devices/platform/vhci_hcd.0/status

hub port sta spd dev sockfd local_busid
hs 0000 004 000 00000000 000000 0-0
hs 0001 004 000 00000000 000000 0-0
hs 0002 004 000 00000000 000000 0-0
hs 0003 004 000 00000000 000000 0-0
hs 0004 004 000 00000000 000000 0-0
hs 0005 004 000 00000000 000000 0-0
hs 0006 004 000 00000000 000000 0-0
hs 0007 004 000 00000000 000000 0-0
ss 0008 004 000 00000000 000000 0-0
ss 0009 004 000 00000000 000000 0-0
ss 0010 004 000 00000000 000000 0-0
ss 0011 004 000 00000000 000000 0-0
ss 0012 004 000 00000000 000000 0-0
ss 0013 004 000 00000000 000000 0-0
ss 0014 004 000 00000000 000000 0-0
ss 0015 004 000 00000000 000000 0-0

You're right the problem might be cat is not on the path by default in my distro, unmodified wsl --distribution NixOS-Groundtruth --user root --cd / --exec cat /sys/devices/platform/vhci_hcd.0/status gives:

<3>WSL (2897) ERROR: CreateProcessEntryCommon:505: execvpe cat failed 2
<3>WSL (2897) ERROR: CreateProcessEntryCommon:508: Create process not expected to return

[dorssel]

Does the following work?

wsl --distribution NixOS-Groundtruth --user root --cd / --exec sh -c "cat /sys/devices/platform/vhci_hcd.0/status"

and/or this one?

wsl --distribution NixOS-Groundtruth --user root --cd / --exec /bin/sh -c "cat /sys/devices/platform/vhci_hcd.0/status"

[Pheoxy]

Does the following work?

wsl --distribution NixOS-Groundtruth --user root --cd / --exec sh -c "cat /sys/devices/platform/vhci_hcd.0/status"

and/or this one?

wsl --distribution NixOS-Groundtruth --user root --cd / --exec /bin/sh -c "cat /sys/devices/platform/vhci_hcd.0/status"

To further this as I am using NixOS as well and having the same problem, here is the results of each command.

$ wsl --distribution NixOS --user root --cd / --exec sh -c "cat /sys/devices/platform/vhci_hcd.0/status"
/nix/store/rdd4pnr4x9rqc9wgbibhngv217w2xvxl-bash-interactive-5.2p26/bin/sh: line 1: cat: command not found

$ wsl --distribution NixOS --user root --cd / --exec /bin/sh -c "cat /sys/devices/platform/vhci_hcd.0/status"
/nix/store/rdd4pnr4x9rqc9wgbibhngv217w2xvxl-bash-interactive-5.2p26/bin/sh: line 1: cat: command not found

[dorssel]

If NixOS cannot even run Linux Standard Base Core applications, then it is not going to be supported. Use another distro to attach.

[Pheoxy]

If NixOS cannot even run Linux Standard Base Core applications, then it is not going to be supported. Use another distro to attach.

As this seems to be a issue with how NixOS uses packages I normally get around it by using paths like /usr/bin/env bash in my scripts so it still works within other distros, it being a problem with usbipd not finding cat may mean other commands might be unavailable as well.

I agree on not going out of your way to support this, in fact I'm more of a scripter than programmer so I couldn't way in on the complexity of supporting NixOS but don't waste your time.

I'm sure that theirs probably a NixOS option that can be used to allow support and as long as the standard distro support works there may be a NixOS specific option we can enable when using usbipd like we do with vscode.

[dorssel]

One option would be (if NixOS somehow is capable) to ensure the required commands are in the default path; on regular OSs this is in /etc/environment. The commands used by usbipd-win are:

• cat
• /bin/sh
• /bin/wslinfo (injected by WSL, should be available always)

Wikipedia explicitly says:

An implication of this is that NixOS does not follow the Filesystem Hierarchy Standard.

But like I said, the easiest is probably to just install Ubuntu as well, and use that for attaching. The device should then be available in NixOS as well.

[saippua]

I believe this issue is related to nix-community/NixOS-WSL#284 (comment)

For me the line that fails is in Wsl.cs the bit where it tries to read the kernel version using a command like:

wsl --distribution NixOS --user root --cd / --exec cat /proc/config.gz

I've confirmed this produces an error when executed in PS.
the following works (i.e. same without --exec):

wsl --distribution NixOS --user root --cd / cat /proc/config.gz

Booting up another distro and attaching through that works, however I'm not super thrilled about having a random ubuntu stub in my configurations, and it's also tedious to boot the stub every time I need to reattach a device.

is there any chance we could use a login shell to read the kernel version, or perhaps fallback to one if a non-login shell fails?

I'd be interested in fiddling around with this a bit more in a fork, but I haven't been able to find any build instructions.

[dorssel]

@saippua
The login shell doesn't have to be a sh-type shell. Could be PowerShell, which has a different syntax (not for cat, but for some of the other commands).

Does the following work?

wsl --distribution NixOS --user root --cd / --exec sh -c 'cat /proc/config.gz'

and this one?

wsl --distribution NixOS --user root --cd / --exec /bin/sh -c 'cat /proc/config.gz'

If we are going the "fallback" route (I like the idea), then we might as well go all the way:

1. --exec cat
2. --exec sh -c cat (with or without prefix /bin)
3. cat (using whatever login shell happens to be configured)

And this for all the commands, of course.

[saippua]

wsl --distribution NixOS --user root --cd / --exec sh -c 'cat /proc/config.gz'
wsl --distribution NixOS --user root --cd / --exec /bin/sh -c 'cat /proc/config.gz'

Neither of these work (the error is cat: command not found), but

wsl --distribution NixOS --user root --cd / --exec sh -lc 'cat /proc/config.gz'
wsl --distribution NixOS --user root --cd / --exec /bin/sh -lc 'cat /proc/config.gz'

does work (added -l flag)

[dorssel]

From https://pubs.opengroup.org/onlinepubs/9799919799/utilities/sh.html, the -l option isn't actually standardized. And the /bin prefix assumes FHS, which if it is available isn't actually necessary (so we can skip that version). Therefore, I will make any command xxx use the following fallback sequence:

1. --exec xxx (best for performance, if it works)
2. --exec sh -c xxx (POSIX)
3. --exec sh -lc xxx (for non-FHS, such as NixOS)
4. xxx (using whatever default shell is configured)

[qwqawawow]

Sorry to bother
Small hack for other who need solve this

sudo ln -s /run/current-system/sw/bin/cat /usr/bin
sudo ln -s /run/current-system/sw/bin/usbip /usr/bin

From nix-community/NixOS-WSL#111 (comment)