shiftfs error on /usr/bin/nvidia-persistenced with NVIDIA GPU Operator #111
Description
{"output":"Failed to run envbox: start container: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: failed to request shiftfs marking to sysbox-mgr: failed to invoke ReqShiftfsMark via grpc: rpc error: code = Unknown desc = failed to mark shiftfs on /usr/bin/nvidia-persistenced at /var/lib/sysbox/shiftfs/5b2aad2e-4651-49bd-aed9-40665640bc00: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type","time":"2024-11-25T16:47:05.945875989Z","type":"error"}
Customer reporting this is using the Helm installed https://helm.ngc.nvidia.com/nvidia "gpu-operator" at version v24.6.1
They are using ghcr.io/coder/envbox:latest as of last week.