seed-job-agent provisioning broken since Jenkins 2.348

[steigr]

Describe the bug
Since 2.348 the operator fails to parse the slave-agent.jnlp xml document.

To Reproduce
Create a operated jenkins in version 2.348.

Additional information

Kubernetes version: v1.23.6+k3s1
Jenkins Operator version: 0.7.0

Add error logs about the problem here (operator logs and Kubernetes events).

The Operator finishes to inject CasC instructions. Then it fails to parse the JNLP body:

jenkins-operator-7f956b89bc-5xcpj jenkins-operator 2022-05-17T15:48:24.279Z	WARN	controller-jenkins	Reconcile loop failed 10 times with the same errors, giving up: Node secret cannot be parsed
jenkins-operator-7f956b89bc-5xcpj jenkins-operator github.com/jenkinsci/kubernetes-operator/pkg/client.(*jenkins).GetNodeSecret
jenkins-operator-7f956b89bc-5xcpj jenkins-operator 	/workspace/pkg/client/jenkins.go:209
jenkins-operator-7f956b89bc-5xcpj jenkins-operator github.com/jenkinsci/kubernetes-operator/pkg/configuration/user/seedjobs.(*seedJobs).createAgent

The JNLP format looks like the following:

<jnlp>
	<application-desc>
		<argument>0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef</argument>
		<argument>seed-job-agent</argument>
		<argument>-workDir</argument>
		<argument>/home/jenkins</argument>
		<argument>-internalDir</argument>
		<argument>remoting</argument>
		<argument>-url</argument>
		<argument>https://jenkins.example.com/</argument>
	</application-desc>
</jnlp>

before the structure was more like this:

<jnlp codebase="https://jenkins.example.com/computer/seed-job-agent/" spec="1.0+">
<information>
	<title>Agent for seed-job-agent</title>
	<vendor>Jenkins project</vendor>
	<homepage href="https://jenkins-ci.org/"></homepage>
</information>
<security>
	<all-permissions></all-permissions>
</security>
<resources>
	<j2se version="1.8+"></j2se>
	<jar href="https://jenkins.example.com/jnlpJars/remoting.jar"></jar>
</resources>
<application-desc main-class="hudson.remoting.jnlp.Main">
	<argument>0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef</argument>
	<argument>seed-job-agent</argument>
	<argument>-workDir</argument>
	<argument>/home/jenkins</argument>
	<argument>-internalDir</argument>
	<argument>remoting</argument>
	<argument>-url</argument>
	<argument>https://jenkins.example.com/</argument>
</application-desc>
</jnlp>

(XML has been re-formatted for readability. it's all in one line).

[Pikabanga]

We are having the same problem after upgrading from 2.343-jdk11 to 2.348-jdk11

[Pikabanga]

I can confirm downgrading back to our previous version (2.343-jdk11 in our case), removed this error.

[rkrzewski]

I'm seeing this too. Downgrading Jenkins to 2.347 fixes the problem - for now.

[kodil]

any news regarding this issue ?

[sli720]

What's the status on that issue? I'm getting the same error when I want to update to the latest LTS version and have to stay on the older version for now.

[sli720]

Is it somehow possible to workaround this bug?

[cbernard-psee]

Hi, sharing the pain here, this issue prevents us to use the latest LTS release ... Any update to share on a fix or workaround will be more than welcome :-)

[sli720]

Is this project still active or looking for maintainers?

[toabi]

Same error here, tried to upgrade because some indirect dependency plugins really start breaking with current other ones. :(

If somebody else has a suddenly failing jenkins when using groovy pipelines: I had to pin script-security=1183.v774b_0b_0a_a_451

[pniederlag]

regex in jenkins seems to be relevant (?)

kubernetes-operator/pkg/client/jenkins.go

Line 16 in 3a143a0

regex = regexp.MustCompile("(<application-desc main-class=\"hudson.remoting.jnlp.Main\"><argument>)(?P<secret>[a-z0-9]*)")

[pniederlag]

docker pull carintech/jenkins:operator-81f4742b10 is a docker-image where this problem is fixed and which is capable to run with current lts image from jenkins, source ist based on this commit

Requirements:

• Update the CRD from master so you can also set seedJobAgentImage
• in deployment of operator set image to carintech/jenkins:operator-81f4742b10
• in CR for jenkins set seedJobAgentImage: jenkins/inbound-agent:3071.v7e9b_0dc08466-1
• use jenkins/jenkins:2.361-3-lts

[brokenpip3]

@pniederlag thanks but why not a PR vs upstream here?

[rkrzewski]

@brokenpip3 PR would be good, but this project appears to be dead. No one around to merge the PR and cut a release...

[brokenpip3]

I asked today in virtuslab slack about the status of this project.
In case they do not have the time to maintain the project I think that with just a couple of new maintainers this project could be revitalized, if you see the last major issues were fixed by the community PRs directly.

Let's see what they reply

[toabi]

Let's see what they reply

@brokenpip3 Did they reply something? It's really a pity that this project is falling behind so much behind.

[brokenpip3]

nope, let's try to ping @Sig00rd here

[brokenpip3]

@pniederlag any chance to create the PR? otherwise I can do it, so at least since we are trying to get attention they can just merge the community PR

[rkrzewski]

@brokenpip3 if you feel motivated to push this through, you could create your own fork, cherry-pick @pniederlag's commit and post the PR yourself.
But since the maintainers appear to be AWOL, maybe we should try to get Jenkins maintainers involved somehow? Kubernetes is a de-facto industry standard these days and this project provides a crucial integration piece. I believe it has it's role in long term viability of Jenkins as a whole.

[brokenpip3]

maybe we should try to get Jenkins maintainers involved somehow? Kubernetes is a de-facto industry standard these days and this project provides a crucial integration piece. I believe it has it's role in long term viability of Jenkins as a whole.

I totally agree with this ^, we should open a separate issue and use that to gain attention from the current maintainers and the jenkinsci ones.

My request of PR was only to have the right PR to be reviewed and merged

[brokenpip3]

Everybody: I finally create the PR to fix this issue (based on @pniederlag work) and all the others issues: #784

Please take a look and test and let me know.