Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

v0.13.0: FailedCreate Pod affinity-assistant: must be no more than 63 characters #2766

Closed
r0bj opened this issue Jun 5, 2020 · 3 comments · Fixed by #2768
Closed

v0.13.0: FailedCreate Pod affinity-assistant: must be no more than 63 characters #2766

r0bj opened this issue Jun 5, 2020 · 3 comments · Fixed by #2768
Assignees
@jlpettersson
Labels
kind/bug Categorizes issue or PR as related to a bug.
Milestone
Pipelines 0.13.1 🐱

Comments

@r0bj
Copy link

r0bj commented Jun 5, 2020 •
edited
Loading

Expected Behavior

Pod affinity-assistant is created successfully.

Actual Behavior

Pod affinity-assistant fail to create so the whole pipelinerun fail.

Steps to Reproduce the Problem

  1. My workspace definition in TriggerBinding:
---
apiVersion: triggers.tekton.dev/v1alpha1
kind: TriggerBinding
metadata:
  name: build-images
[...]
      workspaces:
      - name: build-images-workspace
        volumeClaimTemplate:
          metadata:
            name: build-images
          spec:
            accessModes:
            - ReadWriteOnce
            resources:
              requests:
                storage: 100Mi
  1. PipelineRun starts with name build-images-75hz8
  2. PipelineRun creates PVC with name: build-images-build-images-workspace-build-images-75hz8
  3. Creating Pod affinity-assistant failing with:
  Warning  FailedCreate  54s (x16 over 3m38s)  statefulset-controller  create Pod affinity-assistant-build-images-workspace-build-images-75hz8-0 in StatefulSet affinity-assistant-build-images-workspace-build-images-75hz8 failed error: Pod "affinity-assistant-build-images-workspace-build-images-75hz8-0" is invalid: metadata.labels: Invalid value: "affinity-assistant-build-images-workspace-build-images-75hz8-66f4857895": must be no more than 63 characters

Additional Info

  • Kubernetes version:
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.3", GitCommit:"2e7996e3e2712684bc73f0dec0200d64eec7fe40", GitTreeState:"clean", BuildDate:"2020-05-21T14:51:23Z", GoVersion:"go1.14.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.4", GitCommit:"8d8aa39598534325ad77120c120a22b3a990b5ea", GitTreeState:"clean", BuildDate:"2020-03-12T20:55:23Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
  • Tekton Pipeline version: v0.13.0
@jlpettersson
Copy link
Member

/kind bug
/assign jlpettersson

@tekton-robot tekton-robot added the kind/bug Categorizes issue or PR as related to a bug. label Jun 5, 2020
jlpettersson added a commit to jlpettersson/pipeline that referenced this issue Jun 6, 2020
@jlpettersson
StatefulSet is sensitive to long names - remove prefix
b923e18 
Names in Kubernetes can be up to 253 chars, but labels can only be up to 63 chars.

We are relatively conservative with the two labels we introduce for the Affinity Assistant

    app.kubernetes.io/component: affinity-assistant
    app.kubernetes.io/instance: ws-parallel-pipelinerun-bbx6w

But apparently, StatefulSets adds a label with the full StatefulSet Name + 10 chars (for a hash) as a label

    controller-revision-hash: affinity-assistant-ws-parallel-pipelinerun-bbx6w-dd64c6c8d

This only leave users to use StatefulSet Names up to 53 chars. We use a prefix of 19 chars (affinity-assistant-)
on the Affinity Assistant StatefulSet. This leaves Tekton users with only 34 chars for a combination of
Workspace Name and the PipelineRun Name.

This commit removes the prefix (affinity-assistant-), so that users can use up to 53 chars for
Workspace Name and PipelineRun Name. Also the unnecessary name of the PVC in the volumeClaimTemplate-example is removed.

This limitation of StatefulSet is apparently a known problem kubernetes/kubernetes#64023 but I was not aware of it.

/kind bug
Fixes tektoncd#2766
@jlpettersson jlpettersson mentioned this issue Jun 6, 2020
Merged
3 tasks
@jlpettersson
Copy link
Member

@r0bj Thanks for reporting this!

It appears as the new version is more sensitive to long names for Workspaces+PipelineRuns, because of the introduction of an "Affinity Assistant" that is a StatefulSet and seem to be more sensitive to long names.

In your example

      workspaces:
      - name: build-images-workspace
        volumeClaimTemplate:
          metadata:
            name: build-images

I would recommend to remove name: build-images in the volumeClaimTemplate, since the PVC will be named from the Workspace Name (build-images-workspace in your case) in addition to the PipelineRun Name.

@r0bj
Copy link
Author

r0bj commented Jun 6, 2020

@jlpettersson thanks!
BTW, thanks for really useful Affinity Assistant feature!

jlpettersson added a commit to jlpettersson/pipeline that referenced this issue Jun 6, 2020
@jlpettersson
StatefulSet is sensitive to long names - use hashed name
6bc75f5 
Names in Kubernetes can be up to 253 chars, but labels can only be up to 63 chars.

We are relatively conservative with the two labels we introduce for the Affinity Assistant

    app.kubernetes.io/component: affinity-assistant
    app.kubernetes.io/instance: ws-parallel-pipelinerun-bbx6w

But apparently, StatefulSets adds a label with the full StatefulSet Name + 10 chars (for a hash) as a label

    controller-revision-hash: affinity-assistant-ws-parallel-pipelinerun-bbx6w-dd64c6c8d

This only leave users to use StatefulSet Names up to 53 chars. We use a prefix of 19 chars (affinity-assistant-)
on the Affinity Assistant StatefulSet. This leaves Tekton users with only 34 chars for a combination of
Workspace Name and the PipelineRun Name.

This commit use a hash of the Workspace Name and the PipelineRun Name to make sure that the name is not too long.
Typical labels after this commit will be:

    labels:
      app.kubernetes.io/component: affinity-assistant
      app.kubernetes.io/instance: affinity-assistant-e067465fc0
      controller-revision-hash: affinity-assistant-e067465fc0-b78cb9478
      statefulset.kubernetes.io/pod-name: affinity-assistant-e067465fc0-0
      tekton.dev/pipeline: parallel-pipeline
      tekton.dev/pipelineRun: parallel-pipelinerun-wr9wd

Also the unnecessary name of the PVC in the volumeClaimTemplate-example is removed.

This limitation of StatefulSet is apparently a known problem kubernetes/kubernetes#64023 but I was not aware of it.

/kind bug
Fixes tektoncd#2766
jlpettersson added a commit to jlpettersson/pipeline that referenced this issue Jun 6, 2020
@jlpettersson
StatefulSet is sensitive to long names - use hashed name
2aecd8c 
Names in Kubernetes can be up to 253 chars, but labels can only be up to 63 chars.

We are relatively conservative with the two labels we introduce for the Affinity Assistant

    app.kubernetes.io/component: affinity-assistant
    app.kubernetes.io/instance: ws-parallel-pipelinerun-bbx6w

But apparently, StatefulSets adds a label with the full StatefulSet Name + 10 chars (for a hash) as a label

    controller-revision-hash: affinity-assistant-ws-parallel-pipelinerun-bbx6w-dd64c6c8d

This only leave users to use StatefulSet Names up to 53 chars. We use a prefix of 19 chars (affinity-assistant-)
on the Affinity Assistant StatefulSet. This leaves Tekton users with only 34 chars for a combination of
Workspace Name and the PipelineRun Name.

This commit use a hash of the Workspace Name and the PipelineRun Name to make sure that the name is not too long.
Typical labels after this commit will be:

    labels:
      app.kubernetes.io/component: affinity-assistant
      app.kubernetes.io/instance: affinity-assistant-e067465fc0
      controller-revision-hash: affinity-assistant-e067465fc0-b78cb9478
      statefulset.kubernetes.io/pod-name: affinity-assistant-e067465fc0-0
      tekton.dev/pipeline: parallel-pipeline
      tekton.dev/pipelineRun: parallel-pipelinerun-wr9wd

Also the unnecessary name of the PVC in the volumeClaimTemplate-example is removed.

This limitation of StatefulSet is apparently a known problem kubernetes/kubernetes#64023 but I was not aware of it.

/kind bug
Fixes tektoncd#2766
This was referenced Jun 9, 2020
Merged
Closed
bobcatfish pushed a commit to jlpettersson/pipeline that referenced this issue Jun 10, 2020
@jlpettersson @bobcatfish
StatefulSet is sensitive to long names - use hashed name
fc79022 
Names in Kubernetes can be up to 253 chars, but labels can only be up to 63 chars.

We are relatively conservative with the two labels we introduce for the Affinity Assistant

    app.kubernetes.io/component: affinity-assistant
    app.kubernetes.io/instance: ws-parallel-pipelinerun-bbx6w

But apparently, StatefulSets adds a label with the full StatefulSet Name + 10 chars (for a hash) as a label

    controller-revision-hash: affinity-assistant-ws-parallel-pipelinerun-bbx6w-dd64c6c8d

This only leave users to use StatefulSet Names up to 53 chars. We use a prefix of 19 chars (affinity-assistant-)
on the Affinity Assistant StatefulSet. This leaves Tekton users with only 34 chars for a combination of
Workspace Name and the PipelineRun Name.

This commit use a hash of the Workspace Name and the PipelineRun Name to make sure that the name is not too long.
Typical labels after this commit will be:

    labels:
      app.kubernetes.io/component: affinity-assistant
      app.kubernetes.io/instance: affinity-assistant-e067465fc0
      controller-revision-hash: affinity-assistant-e067465fc0-b78cb9478
      statefulset.kubernetes.io/pod-name: affinity-assistant-e067465fc0-0
      tekton.dev/pipeline: parallel-pipeline
      tekton.dev/pipelineRun: parallel-pipelinerun-wr9wd

Also the unnecessary name of the PVC in the volumeClaimTemplate-example is removed.

This limitation of StatefulSet is apparently a known problem kubernetes/kubernetes#64023 but I was not aware of it.

/kind bug
Fixes tektoncd#2766
tekton-robot pushed a commit that referenced this issue Jun 10, 2020
@jlpettersson @tekton-robot
StatefulSet is sensitive to long names - use hashed name
5627024 
Names in Kubernetes can be up to 253 chars, but labels can only be up to 63 chars.

We are relatively conservative with the two labels we introduce for the Affinity Assistant

    app.kubernetes.io/component: affinity-assistant
    app.kubernetes.io/instance: ws-parallel-pipelinerun-bbx6w

But apparently, StatefulSets adds a label with the full StatefulSet Name + 10 chars (for a hash) as a label

    controller-revision-hash: affinity-assistant-ws-parallel-pipelinerun-bbx6w-dd64c6c8d

This only leave users to use StatefulSet Names up to 53 chars. We use a prefix of 19 chars (affinity-assistant-)
on the Affinity Assistant StatefulSet. This leaves Tekton users with only 34 chars for a combination of
Workspace Name and the PipelineRun Name.

This commit use a hash of the Workspace Name and the PipelineRun Name to make sure that the name is not too long.
Typical labels after this commit will be:

    labels:
      app.kubernetes.io/component: affinity-assistant
      app.kubernetes.io/instance: affinity-assistant-e067465fc0
      controller-revision-hash: affinity-assistant-e067465fc0-b78cb9478
      statefulset.kubernetes.io/pod-name: affinity-assistant-e067465fc0-0
      tekton.dev/pipeline: parallel-pipeline
      tekton.dev/pipelineRun: parallel-pipelinerun-wr9wd

Also the unnecessary name of the PVC in the volumeClaimTemplate-example is removed.

This limitation of StatefulSet is apparently a known problem kubernetes/kubernetes#64023 but I was not aware of it.

/kind bug
Fixes #2766
jlpettersson added a commit to jlpettersson/pipeline that referenced this issue Jun 10, 2020
@jlpettersson
StatefulSet is sensitive to long names - use hashed name
b01d860 
Names in Kubernetes can be up to 253 chars, but labels can only be up to 63 chars.

We are relatively conservative with the two labels we introduce for the Affinity Assistant

    app.kubernetes.io/component: affinity-assistant
    app.kubernetes.io/instance: ws-parallel-pipelinerun-bbx6w

But apparently, StatefulSets adds a label with the full StatefulSet Name + 10 chars (for a hash) as a label

    controller-revision-hash: affinity-assistant-ws-parallel-pipelinerun-bbx6w-dd64c6c8d

This only leave users to use StatefulSet Names up to 53 chars. We use a prefix of 19 chars (affinity-assistant-)
on the Affinity Assistant StatefulSet. This leaves Tekton users with only 34 chars for a combination of
Workspace Name and the PipelineRun Name.

This commit use a hash of the Workspace Name and the PipelineRun Name to make sure that the name is not too long.
Typical labels after this commit will be:

    labels:
      app.kubernetes.io/component: affinity-assistant
      app.kubernetes.io/instance: affinity-assistant-e067465fc0
      controller-revision-hash: affinity-assistant-e067465fc0-b78cb9478
      statefulset.kubernetes.io/pod-name: affinity-assistant-e067465fc0-0
      tekton.dev/pipeline: parallel-pipeline
      tekton.dev/pipelineRun: parallel-pipelinerun-wr9wd

Also the unnecessary name of the PVC in the volumeClaimTemplate-example is removed.

This limitation of StatefulSet is apparently a known problem kubernetes/kubernetes#64023 but I was not aware of it.

/kind bug
Fixes tektoncd#2766
tekton-robot pushed a commit that referenced this issue Jun 10, 2020
@jlpettersson @tekton-robot
StatefulSet is sensitive to long names - use hashed name
7e74e96 
Names in Kubernetes can be up to 253 chars, but labels can only be up to 63 chars.

We are relatively conservative with the two labels we introduce for the Affinity Assistant

    app.kubernetes.io/component: affinity-assistant
    app.kubernetes.io/instance: ws-parallel-pipelinerun-bbx6w

But apparently, StatefulSets adds a label with the full StatefulSet Name + 10 chars (for a hash) as a label

    controller-revision-hash: affinity-assistant-ws-parallel-pipelinerun-bbx6w-dd64c6c8d

This only leave users to use StatefulSet Names up to 53 chars. We use a prefix of 19 chars (affinity-assistant-)
on the Affinity Assistant StatefulSet. This leaves Tekton users with only 34 chars for a combination of
Workspace Name and the PipelineRun Name.

This commit use a hash of the Workspace Name and the PipelineRun Name to make sure that the name is not too long.
Typical labels after this commit will be:

    labels:
      app.kubernetes.io/component: affinity-assistant
      app.kubernetes.io/instance: affinity-assistant-e067465fc0
      controller-revision-hash: affinity-assistant-e067465fc0-b78cb9478
      statefulset.kubernetes.io/pod-name: affinity-assistant-e067465fc0-0
      tekton.dev/pipeline: parallel-pipeline
      tekton.dev/pipelineRun: parallel-pipelinerun-wr9wd

Also the unnecessary name of the PVC in the volumeClaimTemplate-example is removed.

This limitation of StatefulSet is apparently a known problem kubernetes/kubernetes#64023 but I was not aware of it.

/kind bug
Fixes #2766
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jlpettersson jlpettersson

Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Milestone
Pipelines 0.13.1 🐱
Development

Successfully merging a pull request may close this issue.

StatefulSet is sensitive to long names - use a hashed name jlpettersson/pipeline
4 participants
@vdemeester @r0bj @jlpettersson @tekton-robot