-
Notifications
You must be signed in to change notification settings - Fork 124
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
no namespace "nvidia-device-plugin" when attempting to provision GPU nodes #515
Comments
I ran into the same issue in EKS 1.23 / KF 1.6.1 deploy-eks-blueprints-k8s-addons:
kubectl create ns nvidia-device-plugin --dry-run=client -o yaml | kubectl apply -f -
terraform apply -target="module.eks_blueprints_kubernetes_addons" -auto-approve |
This does not working |
Hey folks, as a workaround, can you create the namespace as @jbgerth suggested or using terraform namespace module? Let me know if any of you interested in contributing for the fix? I will try to reproduce and file a PR to fix it by this week |
@chiennh2 can you please provide details on what is not working |
Looks like we have a PR to fix it, its a known issue - #516, aws-ia/terraform-aws-eks-blueprints#1019 |
closing resolved |
Describe the bug
When modifying local terraform variables to provision GPU nodes, when tf attempts to provision the nvidia-device-plugin, it fails saying that there is no namespace for the plugin. The variables in the terraform code suggest that a namespace should be provisioned, but it's failing.
Steps To Reproduce
Modify the variable "node_instance_type_gpu" to a gpu based node,
terraform init && plan
make deploy
Expected behavior
I expect an autoscaled cluster of GPU nodes to be provisioned by terraform on the EKS cluster.
Environment
Screenshots
The text was updated successfully, but these errors were encountered: