You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
when deploy the comfy deploy ,and the pod can not running. and the error log that is following:
pod.
11m Warning FailedMount pod/comfyui-774b78d8c4-ss44v MountVolume.SetUp failed for volume "comfyui-outputs-pv" : rpc error: code = Internal desc = Could not mount "comfyui-outputs-112233445566-us-east-1" at "/var/lib/kubelet/pods/83046055-0ec9-40ec-8448-67afd9608fa0/volumes/kubernetes.iocsi/comfyui-outputs-pv/mount": Could not check if "/var/lib/kubelet/pods/83046055-0ec9-40ec-8448-67afd9608fa0/volumes/kubernetes.iocsi/comfyui-outputs-pv/mount" is a mount point: stat /var/lib/kubelet/pods/83046055-0ec9-40ec-8448-67afd9608fa0/volumes/kubernetes.iocsi/comfyui-outputs-pv/mount: no such file or directory, Failed to read /host/proc/mounts: open /host/proc/mounts: invalid argument
17m Warning FailedMount pod/comfyui-774b78d8c4-ss44v MountVolume.SetUp failed for volume "comfyui-inputs-pv" : rpc error: code = Internal desc = Could not mount "comfyui-inputs-112233445566-us-east-1" at "/var/lib/kubelet/pods/83046055-0ec9-40ec-8448-67afd9608fa0/volumes/kubernetes.iocsi/comfyui-inputs-pv/mount": Could not check if "/var/lib/kubelet/pods/83046055-0ec9-40ec-8448-67afd9608fa0/volumes/kubernetes.iocsi/comfyui-inputs-pv/mount" is a mount point: stat /var/lib/kubelet/pods/83046055-0ec9-40ec-8448-67afd9608fa0/volumes/kubernetes.iocsi/comfyui-inputs-pv/mount: no such file or directory, Failed to read /host/proc/mounts: open /host/proc/mounts: invalid argument
27m Normal Scheduled pod/comfyui-774b78d8c4-ss44v Successfully assigned default/comfyui-774b78d8c4-ss44v to ip-10-2-178-17.ec2.internal
27m Normal Nominated pod/comfyui-774b78d8c4-ss44v Pod should schedule on: nodeclaim/karpenter-nodepool-xlmf5
27m Normal DisruptionBlocked nodeclaim/karpenter-nodepool-ht9tg Cannot disrupt NodeClaim: Nominated for a pending pod
27m Normal DisruptionBlocked node/ip-10-2-178-17.ec2.internal Cannot disrupt Node: Nominated for a pending pod
27m Warning FailedDraining node/ip-10-2-166-48.ec2.internal Failed to drain node, 12 pods are waiting to be evicted
27m Normal DisruptionTerminating node/ip-10-2-166-48.ec2.internal Disrupting Node: Emptiness/Delete
27m Normal DisruptionTerminating nodeclaim/karpenter-nodepool-hbv2q Disrupting NodeClaim: Emptiness/Delete
26m Warning Failed pod/comfyui-774b78d8c4-n42vk Failed to pull image "112233445566.dkr.ecr.us-east-1.amazonaws.com/comfyui-images:latest": rpc error: code = Unavailable desc = error reading from server: EOF
And check the url awslabs/mountpoint-s3-csi-driver#174, it show that is a bug for s3-csi-addon with karpenter. and I deploy the k8s-device-plugin plugin and restart s3-csi and then it resolve
The text was updated successfully, but these errors were encountered:
when deploy the comfy deploy ,and the pod can not running. and the error log that is following:
pod.
11m Warning FailedMount pod/comfyui-774b78d8c4-ss44v MountVolume.SetUp failed for volume "comfyui-outputs-pv" : rpc error: code = Internal desc = Could not mount "comfyui-outputs-112233445566-us-east-1" at "/var/lib/kubelet/pods/83046055-0ec9-40ec-8448-67afd9608fa0/volumes/kubernetes.io
csi/comfyui-outputs-pv/mount": Could not check if "/var/lib/kubelet/pods/83046055-0ec9-40ec-8448-67afd9608fa0/volumes/kubernetes.iocsi/comfyui-outputs-pv/mount" is a mount point: stat /var/lib/kubelet/pods/83046055-0ec9-40ec-8448-67afd9608fa0/volumes/kubernetes.iocsi/comfyui-outputs-pv/mount: no such file or directory, Failed to read /host/proc/mounts: open /host/proc/mounts: invalid argumentcsi/comfyui-inputs-pv/mount": Could not check if "/var/lib/kubelet/pods/83046055-0ec9-40ec-8448-67afd9608fa0/volumes/kubernetes.io17m Warning FailedMount pod/comfyui-774b78d8c4-ss44v MountVolume.SetUp failed for volume "comfyui-inputs-pv" : rpc error: code = Internal desc = Could not mount "comfyui-inputs-112233445566-us-east-1" at "/var/lib/kubelet/pods/83046055-0ec9-40ec-8448-67afd9608fa0/volumes/kubernetes.io
csi/comfyui-inputs-pv/mount" is a mount point: stat /var/lib/kubelet/pods/83046055-0ec9-40ec-8448-67afd9608fa0/volumes/kubernetes.iocsi/comfyui-inputs-pv/mount: no such file or directory, Failed to read /host/proc/mounts: open /host/proc/mounts: invalid argument27m Normal Scheduled pod/comfyui-774b78d8c4-ss44v Successfully assigned default/comfyui-774b78d8c4-ss44v to ip-10-2-178-17.ec2.internal
27m Normal Nominated pod/comfyui-774b78d8c4-ss44v Pod should schedule on: nodeclaim/karpenter-nodepool-xlmf5
27m Normal DisruptionBlocked nodeclaim/karpenter-nodepool-ht9tg Cannot disrupt NodeClaim: Nominated for a pending pod
27m Normal DisruptionBlocked node/ip-10-2-178-17.ec2.internal Cannot disrupt Node: Nominated for a pending pod
27m Warning FailedDraining node/ip-10-2-166-48.ec2.internal Failed to drain node, 12 pods are waiting to be evicted
27m Normal DisruptionTerminating node/ip-10-2-166-48.ec2.internal Disrupting Node: Emptiness/Delete
27m Normal DisruptionTerminating nodeclaim/karpenter-nodepool-hbv2q Disrupting NodeClaim: Emptiness/Delete
26m Warning Failed pod/comfyui-774b78d8c4-n42vk Failed to pull image "112233445566.dkr.ecr.us-east-1.amazonaws.com/comfyui-images:latest": rpc error: code = Unavailable desc = error reading from server: EOF
And check the url
awslabs/mountpoint-s3-csi-driver#174, it show that is a bug for s3-csi-addon with karpenter. and I deploy the k8s-device-plugin plugin and restart s3-csi and then it resolve
The text was updated successfully, but these errors were encountered: