Skip to content

Commit

Permalink
fix(k8s): default to NFS for in-cluster builder storage for local k8s
Browse files Browse the repository at this point in the history
Docker Desktop and minikube can't run docker-in-docker with the
default storage class (something to do with nested overlayfs).

Note: This is terribly slow, but at least works.
  • Loading branch information
edvald committed Dec 12, 2019
1 parent a967154 commit 10b90e1
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 5 deletions.
7 changes: 7 additions & 0 deletions garden-service/src/plugins/kubernetes/local/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import { configureMicrok8sAddons } from "./microk8s"
import { setMinikubeDockerEnv } from "./minikube"
import { exec } from "../../../util/util"
import { remove } from "lodash"
import { getNfsStorageClass } from "../init"

// TODO: split this into separate plugins to handle Docker for Mac and Minikube

Expand Down Expand Up @@ -129,6 +130,12 @@ export async function configureProvider(params: ConfigureProviderParams<LocalKub
}
}

// Docker Desktop, minikube and others are unable to run docker-in-docker overlayfs
// on top of their default storage class, so we override the default here to use the NFS storage class.
if (config.buildMode !== "local-docker" && !config.storage.builder.storageClass) {
config.storage.builder.storageClass = getNfsStorageClass(config)
}

if (!config.defaultHostname) {
config.defaultHostname = `${projectName}.local.app.garden`
}
Expand Down
5 changes: 0 additions & 5 deletions garden-service/test/data/test-projects/container/garden.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,6 @@ environments:
imagePullSecrets:
# Note: We populate this secret in the test code
- name: test-docker-auth
# This is currently necessary for Docker Desktop, and potentially other local K8s distros
# Note: It is _terribly_ slow, so should only be used for testing.
storage:
builder:
storageClass: garden-system-nfs-v2
- name: cluster-docker-buildkit
providers:
- <<: *clusterDocker
Expand Down

0 comments on commit 10b90e1

Please sign in to comment.