Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --no-trunc flag to maintain original annotation length #19102

Conversation

jakecorrenti
Copy link
Member

@jakecorrenti jakecorrenti commented Jul 3, 2023

Adds a --no-trunc flag to podman kube generate preventing the
annotations from being trimmed at 63 characters. However, due to
the fact the annotations will not be trimmed, any annotation that is
longer than 63 characters means this YAML will no longer be Kubernetes
compatible. However, these YAML files can still be used with podman kube play due to the addition of the new flag below.

Adds a --no-trunc flag to podman kube play supporting YAML files with
annotations that were not truncated to the Kubernetes maximum length of
63 characters.

Does this PR introduce a user-facing change?

Adds a `--no-trunc` flag to `podman kube generate` that does not truncate > 63 char annotations. Adds a `--no-trunc` flag to `podman kube play` that suppors YAML files with annotations that are longer than 63 characters. Warning: if an annotation is longer than 63 chars, then the generated yaml file is not Kubernetes compatible. 

@jakecorrenti
Copy link
Member Author

@mheon @TomSweeneyRedHat ptal

@mheon
Copy link
Member

mheon commented Jul 3, 2023

Code LGTM

@mheon
Copy link
Member

mheon commented Jul 3, 2023

@edsantiago Mind looking at the tests?

@jakecorrenti jakecorrenti force-pushed the kube-generate-print-annotations branch from d33de48 to 214f84c Compare July 3, 2023 18:44
Copy link
Member

@edsantiago edsantiago left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests need work. If they pass, it's only by sheer coincidence. Please use something other than bind-mound-options. I'm sorry, I don't know enough about this to be able to offer a suggestion.

And, this is super super super important, never create files or directories under pwd.

test/e2e/generate_kube_test.go Outdated Show resolved Hide resolved
@mheon
Copy link
Member

mheon commented Jul 3, 2023

You can add an annotation with --annotation to completely control length

@jakecorrenti
Copy link
Member Author

jakecorrenti commented Jul 3, 2023

On my end if I add an annotation with --annotation that is 64 characters it still gets truncated, if I'm understanding you correctly

@mheon
Copy link
Member

mheon commented Jul 3, 2023

That sounds like it could be a bug, I'd expect user-added annotations to survive untruncated

@edsantiago
Copy link
Member

I haven't tested user annotations, but here's one possible way to do it with a tmpdir:

diff --git a/test/e2e/generate_kube_test.go b/test/e2e/generate_kube_test.go
index a938043e2..8732ecc48 100644
--- a/test/e2e/generate_kube_test.go
+++ b/test/e2e/generate_kube_test.go
@@ -1564,13 +1564,11 @@ USER test1`
 	})
 
 	It("podman generate kube --no-trunc on container", func() {
-		dirName := "demoDir"
 		ctrName := "demo"
-		err := os.Mkdir(dirName, os.ModePerm)
+		vol1 := filepath.Join(podmanTest.TempDir, RandomString(99))
+		err := os.MkdirAll(vol1, 0755)
 		Expect(err).ToNot(HaveOccurred())
-		defer os.RemoveAll(dirName)
-
-		session := podmanTest.Podman([]string{"run", "-v", "./" + dirName + ":/" + dirName + ":Z", "--name", ctrName, ALPINE})
+		session := podmanTest.Podman([]string{"run", "-v", vol1 + ":/tmp/foo:Z", "--name", ctrName, ALPINE})
 		session.WaitWithDefaultTimeout()
 		Expect(session).Should(Exit(0))
 
@@ -1582,7 +1580,7 @@ USER test1`
 		err = yaml.Unmarshal(kube.Out.Contents(), pod)
 		Expect(err).ToNot(HaveOccurred())
 
-		Expect(len(pod.Annotations["bind-mount-options"])).To(BeNumerically(">", 63))
+		Expect(pod.Annotations["bind-mount-options"]).To(Equal(vol1 + ":Z"))
 	})
 
 	It("podman generate kube --no-trunc on pod", func() {

@jakecorrenti jakecorrenti force-pushed the kube-generate-print-annotations branch from 214f84c to ba0baf2 Compare July 3, 2023 20:28
@jakecorrenti
Copy link
Member Author

I haven't tested user annotations, but here's one possible way to do it with a tmpdir:

diff --git a/test/e2e/generate_kube_test.go b/test/e2e/generate_kube_test.go
index a938043e2..8732ecc48 100644
--- a/test/e2e/generate_kube_test.go
+++ b/test/e2e/generate_kube_test.go
@@ -1564,13 +1564,11 @@ USER test1`
 	})
 
 	It("podman generate kube --no-trunc on container", func() {
-		dirName := "demoDir"
 		ctrName := "demo"
-		err := os.Mkdir(dirName, os.ModePerm)
+		vol1 := filepath.Join(podmanTest.TempDir, RandomString(99))
+		err := os.MkdirAll(vol1, 0755)
 		Expect(err).ToNot(HaveOccurred())
-		defer os.RemoveAll(dirName)
-
-		session := podmanTest.Podman([]string{"run", "-v", "./" + dirName + ":/" + dirName + ":Z", "--name", ctrName, ALPINE})
+		session := podmanTest.Podman([]string{"run", "-v", vol1 + ":/tmp/foo:Z", "--name", ctrName, ALPINE})
 		session.WaitWithDefaultTimeout()
 		Expect(session).Should(Exit(0))
 
@@ -1582,7 +1580,7 @@ USER test1`
 		err = yaml.Unmarshal(kube.Out.Contents(), pod)
 		Expect(err).ToNot(HaveOccurred())
 
-		Expect(len(pod.Annotations["bind-mount-options"])).To(BeNumerically(">", 63))
+		Expect(pod.Annotations["bind-mount-options"]).To(Equal(vol1 + ":Z"))
 	})
 
 	It("podman generate kube --no-trunc on pod", func() {

Thank you! This is miles better than anything I would have come up with

@jakecorrenti jakecorrenti force-pushed the kube-generate-print-annotations branch 3 times, most recently from 04680bf to 36c2dcf Compare July 5, 2023 17:50
@TomSweeneyRedHat
Copy link
Member

LGTM
once the tests and @edsantiago are hip

@jakecorrenti jakecorrenti force-pushed the kube-generate-print-annotations branch from 36c2dcf to a2b436d Compare July 5, 2023 18:19
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jul 5, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: edsantiago, jakecorrenti

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 5, 2023
@edsantiago
Copy link
Member

System tests are failing.

Also, before you repush, could you rebase? It's a really good habit to get into, your colleagues will silently thank you for it.

@edsantiago
Copy link
Member

This gets the tests passing (and fixes a typo), but brings up the question of this being a behavior change. Possibly a desired one, as discussed above, but it merits a comment in the release note.

diff --git a/pkg/domain/infra/abi/play.go b/pkg/domain/infra/abi/play.go
index fefaa8987..3aee0e6cf 100644
--- a/pkg/domain/infra/abi/play.go
+++ b/pkg/domain/infra/abi/play.go
@@ -230,7 +230,7 @@ func (ic *ContainerEngine) PlayKube(ctx context.Context, body io.Reader, options
 			podTemplateSpec.Spec = podYAML.Spec
 			for name, val := range podYAML.Annotations {
 				if len(val) > define.MaxKubeAnnotation {
-					logrus.Warnf("annotation %q=%q value length exceeds Kubernetetes max %d", name, val, define.MaxKubeAnnotation)
+					logrus.Warnf("annotation %q=%q value length exceeds Kubernetes max %d", name, val, define.MaxKubeAnnotation)
 				}
 			}
 			for name, val := range options.Annotations {
diff --git a/test/system/700-play.bats b/test/system/700-play.bats
index 62bfe2478..51ece0818 100644
--- a/test/system/700-play.bats
+++ b/test/system/700-play.bats
@@ -427,16 +427,16 @@ _EOF
     RANDOMSTRING=$(random_string 65)
     mkdir -p $TESTDIR
     echo "$testYaml" | sed "s|TESTDIR|${TESTDIR}|g" > $PODMAN_TMPDIR/test.yaml
-    run_podman 125 play kube --annotation "name=$RANDOMSTRING" $PODMAN_TMPDIR/test.yaml
-    assert "$output" =~ "annotation exceeds maximum size, 63, of kubernetes annotation:" "Expected to fail with Length greater than 63"
+    run_podman play kube --annotation "name=$RANDOMSTRING" $PODMAN_TMPDIR/test.yaml
+    assert "$output" =~ "annotation exceeds maximum size, 63, of kubernetes annotation:" "Expected to warn with Length greater than 63"
 }
 
 @test "podman play Yaml with annotation > Max" {
    RANDOMSTRING=$(random_string 65)
 
    _write_test_yaml "annotations=test: ${RANDOMSTRING}" command=id
-    run_podman 125 play kube - < $PODMAN_TMPDIR/test.yaml
-    assert "$output" =~ "invalid annotation \"test\"=\"$RANDOMSTRING\"" "Expected to fail with annotation length greater than 63"
+    run_podman play kube - < $PODMAN_TMPDIR/test.yaml
+    assert "$output" =~ "warning.*annotation .*test.*=.*$RANDOMSTRING.* value length exceeds Kubernetes max 63" "Expected to warn with annotation length greater than 63"
 }
 
 @test "podman kube play - default log driver" {

@jakecorrenti jakecorrenti force-pushed the kube-generate-print-annotations branch from a2b436d to 46c4c74 Compare July 5, 2023 20:02
@jakecorrenti jakecorrenti force-pushed the kube-generate-print-annotations branch 8 times, most recently from 044729a to 28bad68 Compare July 7, 2023 18:07
Copy link
Member

@vrothberg vrothberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, @jakecorrenti !

Final polishing and we're good to go.

@edsantiago PTAL

pkg/api/server/register_kube.go Outdated Show resolved Hide resolved
test/e2e/generate_kube_test.go Outdated Show resolved Hide resolved
test/e2e/generate_kube_test.go Show resolved Hide resolved
@jakecorrenti jakecorrenti force-pushed the kube-generate-print-annotations branch 2 times, most recently from 677f435 to f327f9e Compare July 10, 2023 13:19
@@ -214,6 +214,10 @@ When no network option is specified and *host* network mode is not configured in

This option conflicts with host added in the Kubernetes YAML.

#### **--no-trunc**

Use annotations that are not truncated to the Kubernetes maxmum length of 63 characters
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: s/maxmum/maximum

@umohnani8
Copy link
Member

one typo otherwise LGTM

@TomSweeneyRedHat
Copy link
Member

Ditto Urvashi's comment. Fix the typo, then LGTM

Adds a `--no-trunc` flag to `podman kube generate` preventing the
annotations from being trimmed at 63 characters. However, due to
the fact the annotations will not be trimmed, any annotation that is
longer than 63 characters means this YAML will no longer be Kubernetes
compatible. However, these YAML files can still be used with `podman
kube play` due to the addition of the new flag below.

Adds a `--no-trunc` flag to `podman kube play` supporting YAML files with
annotations that were not truncated to the Kubernetes maximum length of
63 characters.

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
@jakecorrenti jakecorrenti force-pushed the kube-generate-print-annotations branch from f327f9e to 7b54fd8 Compare July 10, 2023 22:03
@jakecorrenti
Copy link
Member Author

fixed and rebased

Copy link
Member

@TomSweeneyRedHat TomSweeneyRedHat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@umohnani8
Copy link
Member

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jul 11, 2023
@openshift-merge-robot openshift-merge-robot merged commit 77b36ca into containers:main Jul 11, 2023
jakecorrenti added a commit to jakecorrenti/podman that referenced this pull request Jul 14, 2023
Adds an `--podman-only` flag to `podman generate kube` to allow for
reserved annotations to be included in the generated YAML file.

Associated with: containers#19102

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Oct 10, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 10, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/api-change Change to remote API; merits scrutiny lgtm Indicates that a PR is ready to be merged. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. release-note
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants