Update Jenkins version for Windows Image

[Jonny-vb]

What feature do you want to see added?

The jenkins base version seems to lag quite a lot in the windows container releases. This means I often get stuck with warnings like this whilst waiting for updates:

This is for an issue that was reported over a month ago now (and not the first time I've been stuck with one of these warnings).

It looks like the linux containers have been bumped to 2.373 now, whilst the windows version is stuck on 2.356.

Can the windows version be updated please? Is there a reason that the windows version often lags so much? If so it might be nice to add some documentation explaining this somewhere.

Upstream changes

No response

[Nandini99-git]

you an use docker official image like busybox to update

[Jonny-vb]

you an use docker official image like busybox to update

I'm really not too sure what you mean by this?

[Jonny-vb]

@timja sorry to tag you here, but you were very helpful on the last issue I raised!
Could you provide any insight on how the windows base images are updated for the docker images? Is there anything I can do to get a more up to date Jenkins version into the Windows images?

[timja]

@slide possible for you to check this?

[dduportal]

@slide possible for you to check this?

Currently checking the "normal" job on trusted.ci.jenkins.io . The Windows images should be using the same Jenkins version as Linux.

@Jonny-vb can you share with us the image name and digest of the "latest"' image you have ?

@timja can you check if there are remaining SSH deploy key on this repo (like with the docker-*-agent repositories)?

[dduportal]

Right now, it seems that the 2.356 is still the version used: https://hub.docker.com/layers/jenkins/jenkins/windowsservercore-2019/images/sha256-f31ea7e5f4582b665826b094aa3f78a50ccfcbb81dfca4025979aac26a935bf0?context=explore

It is the same "version" has the one used by default: the build argument is probably not set up correctly so it does not override the image.

[dduportal]

Damn: there is nothing about build arg in https://github.com/jenkinsci/docker/blob/master/make.ps1#L6 🤦

[timja]

can you check if there are remaining SSH deploy key on this repo (like with the docker-*-agent repositories)?

I've upgraded your permissions, there was nothing interesting there just some old Jira webhooks that were broken

[dduportal]

Thanks tim! I confirm that I can reproduce the issue on my Windows machine. This is a problem in the pipeline + make.ps1 script. I'll see what I can do on short term.

[dduportal]

Hi @Jonny-vb , we've cleared any risks of an "old system" building and overriding images behind our back.

Since we reproduce the error, It means that there is an easy that can be solved in this repository.
If you are interested in helping, that would be wonderful!

I can give you guidance, pointers and help if you ping me in this issue or in the Gitter channel jenkins/docker.

The root cause here seems to be the build argument JENKINS_VERSION not being passed to Docker when https://github.com/jenkinsci/docker/blob/master/make.ps1#L6 is executed (check the linux build to compare).

[Jonny-vb]

So I assume make.ps1 is called from a pipeline somewhere controlled by yourselves?
Just so I'm clear, are you saying this is correctly being called and the correct version string is being passed into this script, but when the script runs the docker make, the argument isn't making it into that? So it instead builds the default 2.356 version in the dockerfile?

It looks like the JENKINS_VERSION param has to be the version number preceded by a hyphen based on the default (i.e. "-2.356" vs "2.356"). Is whatever is calling the make.ps1 script passing the version number in this correct form?

[dduportal]

So I assume make.ps1 is called from a pipeline somewhere controlled by yourselves? Just so I'm clear, are you saying this is correctly being called and the correct version string is being passed into this script, but when the script runs the docker make, the argument isn't making it into that? So it instead builds the default 2.356 version in the dockerfile?

It looks like the JENKINS_VERSION param has to be the version number preceded by a hyphen based on the default (i.e. "-2.356" vs "2.356"). Is whatever is calling the make.ps1 script passing the version number in this correct form?

Hi @Jonny-vb !

Let me give some pointers:

• First of all, the windows images are published by a Jenkins pipeline run inside a private instance (for obvious reason of security). The stage "Publish" of the Jenkinfile of this repository is responsible for calling the make.ps1 script.
• The target publish is passed to the make.ps1 script, which is expected to call the following code block:

  docker/make.ps1

  Lines 151 to 176 in afabdce

  	if($target -eq "publish") {
  	# Only fail the run afterwards in case of any issues when publishing the docker images
  	$publishFailed = 0
  	if(![System.String]::IsNullOrWhiteSpace($Build) -and $builds.ContainsKey($Build)) {
  	foreach($tag in $Builds[$Build]['Tags']) {
  	Write-Host "Publishing $Build => tag=$tag"
  	$cmd = "docker push {0}/{1}:{2}" -f $Organization, $Repository, $tag
  	Invoke-Expression $cmd
  	if($lastExitCode -ne 0) {
  	$publishFailed = 1
  	}
  	if($PushVersions) {
  	$buildTag = "$JenkinsVersion-$tag"
  	if($tag -eq 'latest') {
  	$buildTag = "$JenkinsVersion"
  	}
  	Write-Host "Publishing $Build => tag=$buildTag"
  	$cmd = "docker push {0}/{1}:{2}" -f $Organization, $Repository, $buildTag
  	Invoke-Expression $cmd
  	if($lastExitCode -ne 0) {
  	$publishFailed = 1
  	}
  	}
  	}
  	} else {

=> the goal is to check what is this script expected to do and if it is doing it correctly. A good start is to compare with the Linux make publish command.

[Jonny-vb]

I'm not sure that section you have highlighted is where the problem lies as that is pushing the image to docker hub. From what I can see, docker hub is updating the windows tag image, but it is the image itself that is wrong.
The dockerfile used to build the image has an argument to specify the Jenkins version which defaults to 2.356, which is the version currently used in the windows image:

docker/11/windows/windowsservercore-2019/hotspot/Dockerfile

Lines 41 to 49 in e2fbfa6

	# jenkins version being bundled in this docker image
	ARG JENKINS_VERSION
	ENV JENKINS_VERSION ${JENKINS_VERSION:-2.356}
	# jenkins.war checksum, download will be validated using it
	ARG JENKINS_SHA=1163c4554dc93439c5eef02b06a8d74f98ca920bbc012c2b8a089d414cfa8075
	# Can be used to customize where jenkins.war get downloaded from
	ARG JENKINS_URL=https://repo.jenkins-ci.org/public/org/jenkins-ci/main/jenkins-war/${JENKINS_VERSION}/jenkins-war-${JENKINS_VERSION}.war

I don't fully understand how the linux builds work, but to me it looks like the version number has to be set as part of the pipeline, otherwise it will just default to the 2.356 version. So it looks like to me the pipeline needs reviewing - how does the version number to build make it there? I can't see anything in the jenkinsfile that sets this (along with the corresponding SHA for the download)

[Jonny-vb]

@dduportal @timja - I think this need a review of the non-public pipeline to really understand what is going wrong here - are either of you able to do this?

[dduportal]

The pipeline is public: https://github.com/jenkinsci/docker/blob/master/Jenkinsfile

You are correct when saying that the version has to be set as part of the pipeline.

But here, we do NOT build with "1 github tag == 1 build with release" for this repository.

The Linux build shows the expected process (that the Windows process does not follow causing the issue): it retrieves the list of "recent versions" (with a bit of filtering) and decides wether or not it should be built or not.

The Windows process should be updated in a way or the other (maybe using a first Linux step to retrieve the list of images and deciede to build or not, and then calling windows/linux build with the version set up or not)

[dduportal]

Update:

• Short term: Thanks @Jonny-vb for Windows: Update default jenkins version to 2.410 #1656 : it should solve the problem one time on short term. make sense!
• Medium term: Context changed since the last comments: chore(publish) only 'publish' when triggered by a tag to not avoid overriding existing tags #1630 make the publication process simpler by only publishing the Jenkins Version tagged. It should allow for a simpler fix for the Windows publication script: "get the tag version and set it as the build arg to override default value).
• Long term: Code simplification: removal of default build ARG values when factorized docker-agent#418 (for agent) also make sense here, to remove the huge list of "default" value.
  • Alternative is to use updatecli/Renovabot to keep these version updated

[lemeurherve]

Reopening until validation with the next Weekly and LTS releases in two days.

[lemeurherve]

Jenkins version is now set on build time, and tags containing it are now published, allowing to use and pin a specific Jenkins version, see images since today: https://hub.docker.com/r/jenkins/jenkins/tags?page=1&name=windows