This repository contains the resources necessary to provision a Jenkins master (Based on the official Red Hat OpenShift Docker image) and run jobs on a dynamic set of slaves using the Jenkins Swarm plugin or to dynamically provision slaves using the Kubernetes plugin within OpenShift.
- Jenkins Master
- Extension of the Red Hat OpenShift Docker image to include plugins and configurations to support auto discovery and dynamic provisioning of Jenkins slaves
- Jenkins Slave
- Container running either the Jenkins Swarm client (Based on the work from csanchez/jenkins-slave/ when auto discovering slaves) or the jnlp agent when dynamically provisioning slave instances
- Templates
- The jenkins-cluster-ephemeral and jenkins-cluster-persistent OpenShift templates are available for rapidly building and provisioning a Jenkins master and slave pods.
The following steps describe how to create a project within OpenShift and add the templates using the OpenShift client tool oc which will be used to provision a Jenkins master and slave instances.
- Clone the repository locally
- Create a new project or use an existing project that will contain the Jenkins master and slaves .
oc new-project jenkins
- Add the templates from the support folder to the OpenShift project that provide .
oc create -f support/jenkins-cluster-persistent-template.json,support/jenkins-cluster-ephemeral-template.json
The templates added to OpenShift in the previous section provide the OpenShift components necessary to build and deploy Docker images containing the Jenkins master and slaves. Both the jenkins-cluster-persistent-template.json and jenkins-cluster-ephemeral-template.json templates provision both the master and slave Jenkins instances. The jenkins-cluster-persistent-template.json allows for a PersistentVolume to be used to save the state of the Jenkins to disk. The using the jenkins-cluster-ephemeral-template.json template, once the pod dies, all configuration will be lost.
Each template contains several parameters that can be modified to modify and tailor the instances to the particular environment. The table below details the parameters available in the templates
| Name | Description | Default Value |
|---|---|---|
| APPLICATION_NAME | Name of the application. The slave will be suffixed with -slave | jenkins |
| APPLICATION_HOSTNAME | Hostname to access Jenkins | <application-name>.<project>.<default-domain-suffix> |
| password | Password securing the admin account of Jenkins | password |
| JENKINS_SERVICE_ACCOUNT | OpenShift service account injected into the Jenkins master | default |
| GIT_URI | Url of this Git repository | https://github.com/sabre1041/ose-jenkins-cluster.git |
| GIT_REF | Git Branch | master |
| EXECUTORS | Number of executors for each Jenkins Swarm slave | 1 |
| SLAVE_RECCURENCE_PERIOD | Interval of time to check whether to provision additional slave nodes | 500 |
| VOLUME_CAPACITY | Available in the jenkins-cluster-persistent-template.json template. The amount space to allocate for data | 512MB |
One of the methods running Jenkins jobs in this project is to dynamically provision slave instances in OpenShift using the Kubernetes plugin. Jenkins communicates with OpenShift using a secured value stored as Credentials. By default, the Jenkins instance is configured to leverage the API token from a service account that is injected into pods by default. The default service account for a pod does not have the adequate permissions necessary to fully provision a slave instance.
Add the edit role to the default service account
oc policy add-role-to-user edit system:serviceaccount:jenkins:default
Alternatively, you can choose to utilize a separate service account to run the Jenkins master. The support folder contains a service account called jenkins that can be added to the project. To instantiate the service account, execute the following command from within the support folder:
oc create -f jenkins.json
The jenkins service account will be created. Now add the edit role to this account as shown previously for the default service account:
oc policy add-role-to-user edit system:serviceaccount:jenkins:jenkins
Subsequent sections will illustrate how to leverage this account
Using either the OpenShift web console or the oc tool, instantiate one of the provided templates. An array of parameters can be specified using the --param= option
oc new-app --template=jenkins-cluster-ephemeral
By default, this will create a new application containing a master, swarm based slave and expose a route for the Jenkins UI at jenkins.<project>.<default-domain-suffix>. A new build of both Docker images will begin and once complete, containers will be deployed. This can be confirmed in the Jenkins web console be not
By default, the following credentials can be used to log into the Jenkins UI
Username: admin
Password: password
The slaves which are connected can be seen under the Build Executor Status section on the left side of the page. Additional slaves can be created by scaling the slave (2 replicas for example)
oc scale dc jenkins-slave --replicas=2
The Jenkins master and slave docker images contain the majority of the functionality preconfigured. When leveraging dynamic slave provisioning using the Kubernetes plugin, there are several values that must either be manually configured or confirmed.
Settings for the Kubernetes plugin can be configured in the Jenkins global system configuration by logging into the Jenkins master and selecting Manage Jenkins on the left side and then selecting Configure System.
Under the Cloud section is a section for Kubernetes. A base configuration has been provided with the majority of the configuration necessary to dynamically provision slaves. Addressing to OpenShift components leverages the built in SkyDNS functionality. The Kubernetes plugin communicates with the OpenShift api at https://openshift.default.svc.cluster.local using the service account injected into the pod. When using this credential, the additional prerequisite steps described above must have been completed to give the service account the requisite permissions. Alternatively, a user name and password based credential can be used instead of a service account.
Next, verify the name the Kubernetes namespace (OpenShift project) is correct. jenkins is used by default. Then validate and modify as necessary the Jenkins URL and Jenkins tunnel addresses. These addresses map to the two services that have been defined for the project (jenkins pointing at port 8080 for API and 50000 for slave connections) These URL's take the following form:
<app_name>.<namespace>.svc.cluster.local:<port>
When the Kubernetes plugin attempts to allocate a new slave dynamically, it will leverage a docker image that was built when the template was instantiated. Since the image is stored in the OpenShift integrated repository, the image name is specific to each OpenShift cluster which is configured in the Docker image textbook underneath the Kubernetes Pod Template section. The value from the Jenkins slave [ImageStream] can be used and can be found by navigating to the ImageStream's after highlighting the Browse button:
Or by executing the following command using the OpenShit cli
oc get is jenkins-slave --no-headers | awk '{ print $2 }'
Hit Save to apply the changes