[JENKINS-50282] Allow creating Pod templates from yaml #275

carlossg · 2018-01-21T08:50:28Z

To avoid having to manually add fields for all possible options current and future, just pass the yaml to be used as base.

Yaml is merged with other fields (if set) as it if were a parent Podtemplate

chancez · 2018-01-23T19:30:53Z

This is awesome. I am a huge supporter of this being a feature. 👍 This would mean I don't need to add explicit support for using taints/tolerations and other things, making many things easier!

Will the JNLP container be merged into the resulting pod template automatically with this?

carlossg · 2018-01-24T10:10:19Z

Yes, if yaml doesn't include a jnlp container it will be added

azweb76 · 2018-01-24T18:30:15Z

So we use a jnlp container. Correct me if Im wrong, if the container is a jnlp container, it will not be merged?

hoshsadiq · 2018-02-08T14:30:31Z

Big. Would also be nice to be able to load it from a file rather than as a string.

chancez · 2018-02-16T00:02:32Z

ALso, what does this mean for the declarative step? If it supported the yaml form, it would fix most of my issues with the declarative steps, assuming you could also use container() steps within.

carlossg · 2018-03-30T20:12:04Z

The rules from combination parent/yaml Pod and child Pod are set in

kubernetes-plugin/src/main/java/org/csanchez/jenkins/plugins/kubernetes/PodTemplateUtils.java

Line 163 in 07cb5e4

public static Pod combine(Pod parent, Pod template) {

Containers are also merged if present in both parent and child with the same name

To read from file instead of inline just use readFile

hoshsadiq · 2018-03-30T21:54:08Z

src/main/java/org/csanchez/jenkins/plugins/kubernetes/PodTemplateBuilder.java

+        return this;
+    }
+
+    @Deprecated


Why is a new deprecated method being added?

this one is not new, the new one is public Pod build()

bbzg · 2018-04-01T21:02:48Z

This is a great addition, but adding a yaml template for 5+ Jenkinsfiles quickly becomes a chore... It would have been so nice if it was possible to set the template from the global Jenkins configuration (since all our jobs share the same settings). Maybe it already is, but I can not seem to find it.

Maybe I am speaking out of line, if I am I apologize.

Thanks for all your great work!

carlossg · 2018-04-01T21:13:02Z

That just needs the jelly UI code, PRs welcome

siwyd · 2018-04-02T08:28:48Z

@bbzg You can use a Jenkins shared library to define a function that keeps all that logic in a single place?

bbzg · 2018-04-02T12:12:59Z

@siwyd Hmm, that's a good idea actually since we already have that for other stuff. Thanks!

bbzg · 2018-04-02T13:44:10Z

def kubernetesNode(Closure closureForPod) {
    String template = """
apiVersion: v1
kind: Pod
metadata:
  labels:
    test: something
spec:
  tolerations:
  - key: "jenkins"
    operator: "Equal"
    value: "builder"
    effect: "NoSchedule"
"""
    GString label = "default-${UUID.randomUUID().toString()}"
    podTemplate(label: label, yaml: template, inheritFrom: "default") {
        node {
            closureForPod.call()
        }
    }
}

This schedules a pod but it does not get any of the attributes from the YAML

I also tried without "inheritFrom" and used name: "default", but with the same result.

(I am using scripted pipeline btw)

Any pointers would be greatly appreciated.

Edit:
For clarification I have a "Cloud" called "kubernetes" and a "Kubernetes pod template" called "default" with a bunch of settings that I want to (re-)use.

bbzg · 2018-04-02T17:05:46Z

I finally got it working with these settings

def kubernetesNode(Closure closureForPod) {
    String template = """
apiVersion: v1
kind: Pod
metadata:
  labels:
    test: something
spec:
  tolerations:
  - key: "jenkins"
    operator: "Equal"
    value: "builder"
    effect: "NoSchedule"
"""
    GString label = "default-${UUID.randomUUID().toString()}"
    podTemplate(label: label, yaml: template) {
        node(label) {
            closureForPod.call()
        }
    }
}

And in the global configuration for the Kubernetes plugin I set Defaults Provider Template Name to default

rlees85 · 2018-04-03T15:09:41Z

I think this change has broken something. I am not using this new feature, but a plugin upgrade from 1.4.1 to 1.5 and a pod with an emptyDir and valid mountpath now just throws this error:

WARNING: Error in provisioning; agent=KubernetesSlave name: packer-slave-xz2bp, template=PodTemplate{inheritFrom='', name='packer-slave', namespace='devops-testing', instanceCap=25, idleMinutes=5, label='packer', nodeSelector='', nodeUsageMode=EXCLUSIVE, workspaceVolume=org.csanchez.jenkins.plugins.kubernetes.volumes.workspace.EmptyDirWorkspaceVolume@48b9a7ee, volumes=[org.csanchez.jenkins.plugins.kubernetes.volumes.EmptyDirVolume@14e5dc74], containers=[ContainerTemplate{name='jnlp', image='removed.dkr.ecr.eu-west-1.amazonaws.com/jenkins-slave-packer:latest', alwaysPullImage=true, workingDir='/home/jenkins', command='', args='${computer.jnlpmac} ${computer.name}', ttyEnabled=true, resourceRequestCpu='', resourceRequestMemory='512Mi', resourceLimitCpu='', resourceLimitMemory='512Mi', envVars=[KeyValueEnvVar [getValue()=-Xmx256M -Xms256M, getKey()=JAVA_OPTS], KeyValueEnvVar [getValue()=tcp://localhost:2375 , getKey()=DOCKER_HOST]], livenessProbe=org.csanchez.jenkins.plugins.kubernetes.ContainerLivenessProbe@8e44d37}, ContainerTemplate{name='docker', image='docker:stable-dind', privileged=true, alwaysPullImage=true, workingDir='', command='', args='', resourceRequestCpu='', resourceRequestMemory='512Mi', resourceLimitCpu='', resourceLimitMemory='512Mi', livenessProbe=org.csanchez.jenkins.plugins.kubernetes.ContainerLivenessProbe@53a14519}], annotations=[org.csanchez.jenkins.plugins.kubernetes.PodAnnotation@6bd82452]}
io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://api.removed/api/v1/namespaces/devops-testing/pods. Message: Pod "packer-slave-xz2bp" is invalid: spec.containers[1].volumeMounts[1].mountPath: Required value. Received status: Status(apiVersion=v1, code=422, details=StatusDetails(causes=[StatusCause(field=spec.containers[1].volumeMounts[1].mountPath, message=Required value, reason=FieldValueRequired, additionalProperties={})], group=null, kind=Pod, name=packer-slave-xz2bp, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=Pod "packer-slave-xz2bp" is invalid: spec.containers[1].volumeMounts[1].mountPath: Required value, metadata=ListMeta(resourceVersion=null, selfLink=null, additionalProperties={}), reason=Invalid, status=Failure, additionalProperties={}).
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:472)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:411)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:381)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:344)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleCreate(OperationSupport.java:227)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleCreate(BaseOperation.java:756)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:334)
	at org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.launch(KubernetesLauncher.java:105)
	at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:288)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Jenkins LTS latest

carlossg · 2018-04-03T15:26:50Z

This is tracked as https://issues.jenkins-ci.org/browse/JENKINS-50525

What would be helpful is a test with an example that fails in https://github.com/jenkinsci/kubernetes-plugin/tree/master/src/test/resources/org/csanchez/jenkins/plugins/kubernetes/pipeline to

bparees · 2018-04-10T18:52:38Z

@carlossg any thoughts about exposing this in the config.xml (so you can paste a pod template yaml into the k8s cloud plugin config ui, instead of having to define it in your pipeline)? Or is this already possible and i'm missing it?

/cc @gabemontero

carlossg · 2018-04-10T19:13:28Z

it's just a matter of adding a new yaml field to https://github.com/jenkinsci/kubernetes-plugin/blob/master/src/main/resources/org/csanchez/jenkins/plugins/kubernetes/PodTemplate/config.jelly and associated help file.
Lack of time on my side, PRs welcome

bparees · 2018-04-10T19:20:47Z

@carlossg we might be able to carve some cycles from @gabemontero to do it in the next few weeks.

shdowofdeath · 2020-01-22T07:10:16Z

i cannot use it like this :

containerTemplate(name: 'basic',
                            image: slaveImage,
                            ttyEnabled: true,
                            shell: 'sh',
                            envVars: [ valueFrom(key: 'parentHost', fieldRef:  "status.hostIP")
                            ],
                            privileged: true,
                            workingDir: '/data/',
                            alwaysPullImage: false,

?? it;s not woking for me

jasminen · 2020-02-23T14:22:52Z

@shdowofdeath It works if you use the 'yaml' attribute

carlossg changed the title ~~Allow creating Pod templates from yaml~~ [WiP] Allow creating Pod templates from yaml Jan 21, 2018

This was referenced Jan 21, 2018

Jenkins-46519: Support for new affinities abstraction for slave templates #243

Closed

Move PodTemplate -> Pod conversion to PodTemplateBuilder #276

Merged

carlossg force-pushed the yaml branch from de2fd24 to 6cc67d4 Compare January 21, 2018 09:26

carlossg force-pushed the yaml branch from 6cc67d4 to a0fbfa8 Compare January 24, 2018 09:46

carlossg mentioned this pull request Jan 24, 2018

Add support for Restart Policy #278

Closed

Allow creating Pod templates from yaml

cba7b78

carlossg force-pushed the yaml branch from a0fbfa8 to cba7b78 Compare January 29, 2018 18:04

ClementGautier mentioned this pull request Feb 7, 2018

Add support for initContainers #254

Closed

carlossg added 3 commits February 21, 2018 11:02

Merge branch 'master' into yaml

21d39be

Fix missing import

4b85294

Merge branch 'master' into yaml

8b5c9d5

carlossg changed the title ~~[WiP] Allow creating Pod templates from yaml~~ [WiP][JENKINS-50282] Allow creating Pod templates from yaml Mar 20, 2018

carlossg mentioned this pull request Mar 21, 2018

add ConfigMapKeyRefEnvVar and fieldRefEnvVar to set env from more sources #297

Closed

carlossg added 4 commits March 25, 2018 17:22

Merge branch 'master' into yaml

59354bc

Prevent NPE when container args is null

0b4b998

Merge branch 'master' into yaml

cd4b89b

Use default workingDir when not set

07cb5e4

carlossg changed the title ~~[WiP][JENKINS-50282] Allow creating Pod templates from yaml~~ [JENKINS-50282] Allow creating Pod templates from yaml Mar 30, 2018

Improve the readme for yaml option

5d2f4c7

hoshsadiq reviewed Mar 30, 2018

View reviewed changes

carlossg merged commit bbf9a20 into master Apr 1, 2018

carlossg deleted the yaml branch April 1, 2018 18:47

oliverlockwood mentioned this pull request Apr 16, 2018

Add yaml field to PodTemplate's toString() method #309

Merged

gabemontero mentioned this pull request Apr 16, 2018

Expose pod cfg via yaml to UI and merge tolerations #311

Merged

onelapahead mentioned this pull request Aug 30, 2018

RFE: YAML PodTemplate support for Agent ConfigMap's openshift/jenkins-sync-plugin#257

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[JENKINS-50282] Allow creating Pod templates from yaml #275

[JENKINS-50282] Allow creating Pod templates from yaml #275

carlossg commented Jan 21, 2018

chancez commented Jan 23, 2018 •

edited

Loading

carlossg commented Jan 24, 2018

azweb76 commented Jan 24, 2018

hoshsadiq commented Feb 8, 2018

chancez commented Feb 16, 2018

carlossg commented Mar 30, 2018

hoshsadiq Mar 30, 2018

carlossg Mar 31, 2018

bbzg commented Apr 1, 2018

carlossg commented Apr 1, 2018

siwyd commented Apr 2, 2018

bbzg commented Apr 2, 2018

bbzg commented Apr 2, 2018 •

edited

Loading

bbzg commented Apr 2, 2018

rlees85 commented Apr 3, 2018

carlossg commented Apr 3, 2018

bparees commented Apr 10, 2018

carlossg commented Apr 10, 2018

bparees commented Apr 10, 2018

shdowofdeath commented Jan 22, 2020

jasminen commented Feb 23, 2020

[JENKINS-50282] Allow creating Pod templates from yaml #275

[JENKINS-50282] Allow creating Pod templates from yaml #275

Conversation

carlossg commented Jan 21, 2018

chancez commented Jan 23, 2018 • edited Loading

carlossg commented Jan 24, 2018

azweb76 commented Jan 24, 2018

hoshsadiq commented Feb 8, 2018

chancez commented Feb 16, 2018

carlossg commented Mar 30, 2018

hoshsadiq Mar 30, 2018

Choose a reason for hiding this comment

carlossg Mar 31, 2018

Choose a reason for hiding this comment

bbzg commented Apr 1, 2018

carlossg commented Apr 1, 2018

siwyd commented Apr 2, 2018

bbzg commented Apr 2, 2018

bbzg commented Apr 2, 2018 • edited Loading

bbzg commented Apr 2, 2018

rlees85 commented Apr 3, 2018

carlossg commented Apr 3, 2018

bparees commented Apr 10, 2018

carlossg commented Apr 10, 2018

bparees commented Apr 10, 2018

shdowofdeath commented Jan 22, 2020

jasminen commented Feb 23, 2020

chancez commented Jan 23, 2018 •

edited

Loading

bbzg commented Apr 2, 2018 •

edited

Loading