Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[doc] Improve documentation for concurrency settings. #3491

Merged
merged 1 commit into from
Jan 16, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 45 additions & 8 deletions docs/content/en/docs/pipeline-stages/builders/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,24 +28,37 @@ see [skaffold.yaml References]({{< relref "/docs/references/yaml" >}}).

## Local Build
Local build execution is the default execution context.
Skaffold will use the build tools locally installed on your machine to execute the build.
Skaffold will use your locally-installed build tools (such as Docker, Bazel, Maven or Gradle) to execute the build.

**Configuration**

To configure the local execution explicitly, add build type `local` to the build section of `skaffold.yaml`

```yaml
build:
local:
...
local: {}
```

The following options can optionally be configured:

{{< schema root="LocalBuild" >}}

If you are deploying to [local cluster]({{<relref "/docs/environment/local-cluster" >}}), you can additional set `push` to `false` to speed up builds.
**Faster builds**

When deploying to a [local cluster]({{<relref "/docs/environment/local-cluster" >}}),
Skaffold will default `push` to `false` to speed up builds.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should expand on this paragraph because there are a few implications.

  • certain clusters, like minikube, kind, Docker Kubernetes, are considered as local clusters
  • Skaffold can usually load the images directly into a local cluster, instead of via an image registry
  • local clusters change the default for push to false to avoid pushing the image to the remote repository
  • this means that the same builds can fail when switching the kube context to a remote cluster. but the speed tradeoff is worth it.

And maybe Faster builds isn't the right section heading either.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy to add some more documentation when most of this is merged :-)


Skaffold can build artifacts in parallel by setting `concurrency` to a value other than `1`, and `0` means there are no limits.
For local builds, this is however disabled by default since local builds could have side effects that are
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we should mention specific builders that are believed to be safe (docker, Bazel) or that aren't safe (and the circumstances)?

not compatible with parallel builds. Feel free to increase the `concurrency` if you know that your builds
can run in parallel.

{{<alert title="Note">}}
When artifacts are built in parallel, the build logs are still printed in sequence to make them easier to read.
{{</alert>}}

## In Cluster Build

Skaffold supports building in cluster via [Kaniko]({{< relref "/docs/pipeline-stages/builders/docker#dockerfile-in-cluster-with-kaniko" >}})
or [Custom Build Script]({{<relref "/docs/pipeline-stages/builders/custom#custom-build-script-in-cluster" >}}).

Expand All @@ -55,14 +68,23 @@ To configure in-cluster Build, add build type `cluster` to the build section of

```yaml
build:
cluster:
...
cluster: {}
```

The following options can optionally be configured:

{{< schema root="ClusterDetails" >}}

**Faster builds**

Skaffold can build multiple artifacts in parallel, by settings a value higher than `1` to `concurrency`.
For in-cluster builds, the default is to build all the artifacts in parallel. If your cluster is too
small, you might want to reduce the `concurrency`. Setting `concurrency` to `1` will cause artifacts to be built sequentially.

{{<alert title="Note">}}
When artifacts are built in parallel, the build logs are still printed in sequence to make them easier to read.
{{</alert>}}

## Remotely on Google Cloud Build

Skaffold supports building remotely with Google Cloud Build.
Expand Down Expand Up @@ -91,10 +113,25 @@ section of `skaffold.yaml`.

```yaml
build:
googleCloudBuild:
...
googleCloudBuild: {}
```

The following options can optionally be configured:

{{< schema root="GoogleCloudBuild" >}}

**Faster builds**

Skaffold can build multiple artifacts in parallel, by settings a value higher than `1` to `concurrency`.
For Google Cloud Build, the default is to build all the artifacts in parallel. If you hit a quota restriction,
you might want to reduce the `concurrency`.

{{<alert title="Note">}}
When artifacts are built in parallel, the build logs are still printed in sequence to make them easier to read.
{{</alert>}}

**Restrictions**

Skaffold currently supports [Docker]({{<relref "/docs/pipeline-stages/builders/docker#dockerfile-remotely-with-google-cloud-build">}}),
[Jib]({{<relref "/docs/pipeline-stages/builders/jib#remotely-with-google-cloud-build">}})
on Google Cloud Build.
15 changes: 9 additions & 6 deletions docs/content/en/schemas/v2alpha2.json
Original file line number Diff line number Diff line change
Expand Up @@ -570,8 +570,9 @@
},
"concurrency": {
"type": "integer",
"description": "how many artifacts can be built concurrently. 0 means \"no-limit\" Defaults to 0.",
"x-intellij-html-description": "how many artifacts can be built concurrently. 0 means &quot;no-limit&quot; Defaults to 0."
"description": "how many artifacts can be built concurrently. 0 means \"no-limit\".",
"x-intellij-html-description": "how many artifacts can be built concurrently. 0 means &quot;no-limit&quot;.",
"default": "0"
},
"dockerConfig": {
"$ref": "#/definitions/DockerConfig",
Expand Down Expand Up @@ -914,8 +915,9 @@
"properties": {
"concurrency": {
"type": "integer",
"description": "how many artifacts can be built concurrently. 0 means \"no-limit\" Defaults to 0.",
"x-intellij-html-description": "how many artifacts can be built concurrently. 0 means &quot;no-limit&quot; Defaults to 0."
"description": "how many artifacts can be built concurrently. 0 means \"no-limit\".",
"x-intellij-html-description": "how many artifacts can be built concurrently. 0 means &quot;no-limit&quot;.",
"default": "0"
},
"diskSizeGb": {
"type": "integer",
Expand Down Expand Up @@ -1570,8 +1572,9 @@
"properties": {
"concurrency": {
"type": "integer",
"description": "how many artifacts can be built concurrently. 0 means \"no-limit\" Defaults to 1.",
"x-intellij-html-description": "how many artifacts can be built concurrently. 0 means &quot;no-limit&quot; Defaults to 1."
"description": "how many artifacts can be built concurrently. 0 means \"no-limit\".",
"x-intellij-html-description": "how many artifacts can be built concurrently. 0 means &quot;no-limit&quot;.",
"default": "1"
},
"push": {
"type": "boolean",
Expand Down
12 changes: 6 additions & 6 deletions pkg/skaffold/schema/latest/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -197,8 +197,8 @@ type LocalBuild struct {
// UseBuildkit use BuildKit to build Docker images.
UseBuildkit bool `yaml:"useBuildkit,omitempty"`

// Concurrency is how many artifacts can be built concurrently. 0 means "no-limit"
// Defaults to 1.
// Concurrency is how many artifacts can be built concurrently. 0 means "no-limit".
// Defaults to `1`.
Concurrency *int `yaml:"concurrency,omitempty"`
}

Expand Down Expand Up @@ -251,8 +251,8 @@ type GoogleCloudBuild struct {
// Defaults to `gcr.io/k8s-skaffold/pack`.
PackImage string `yaml:"packImage,omitempty"`

// Concurrency is how many artifacts can be built concurrently. 0 means "no-limit"
// Defaults to 0.
// Concurrency is how many artifacts can be built concurrently. 0 means "no-limit".
// Defaults to `0`.
Concurrency int `yaml:"concurrency,omitempty"`
}

Expand Down Expand Up @@ -301,8 +301,8 @@ type ClusterDetails struct {
// Resources define the resource requirements for the kaniko pod.
Resources *ResourceRequirements `yaml:"resources,omitempty"`

// Concurrency is how many artifacts can be built concurrently. 0 means "no-limit"
// Defaults to 0.
// Concurrency is how many artifacts can be built concurrently. 0 means "no-limit".
// Defaults to `0`.
Concurrency int `yaml:"concurrency,omitempty"`
}

Expand Down