Skip to content

Home

Jump to bottom
Fred G edited this page Oct 14, 2024 · 8 revisions

Jenkins logo

Jenkins is a continuous integration (CI) server. It is in use on Eclipse servers for Eclipse projects as part of the Common Build Infrastructure (CBI). This page is about the hosted service at Eclipse.org. For more information on the project itself, or to download Jenkins, please see the Jenkins project page.

Table of Contents

General Information

Jenkins instances are maintained by the Eclipse Webmasters/Release Engineers.

Asking for Help

Requesting a JIPP instance for CBI

Please file a HelpDesk ticket to request your project's own instance. Please ensure your project lead can +1 the request.

FAQ

How can I run my build in a container with root privileges?

Unfortunately, for security reasons, you cannot do that. We run an infrastructure open to the internet, which potentially runs stuff from non-trusted code (e.g., PR) so we need to follow a strict policy to protect the common good.

More specifically, we run containers using an arbitrarily assigned user ID (e.g. 1000100000) in our OpenShift cluster. The group ID is always root (0) though. The security context constraints we use for running projects' containers is "restricted". You cannot change this level from your podTemplate.

Unfortunately, most of images you can find on DockerHub (including official images) do not support running as an arbitrary user. Actually, most of them expect to run as root, which is definitely a bad practice.

OpenShift publishes guidelines with best practices about how to create Docker images. More specifically, see the section about how to support running with arbitrary user ID.

To test if an image is ready to be run with an arbitrarily assigned user ID, you can try to start it with the following command line: 

   $ docker run -it --rm -u $((1000100000 + RANDOM % 100000)):0 image/name:tag

I want to build a custom Docker image (with docker build), but it does not work. What should I do?

You can use and integrate the eclipse foundation jenkins shared library named: jenkins-pipeline-library. This library propose a containerBuild function for building docker images in the eclipse foundation infrastructure. 

@Library('releng-pipeline') _

pipeline {
    agent any
    environment {
        HOME = "${env.WORKSPACE}"
    }
    stages {
        stage('build') {
            agent {
                kubernetes {
                    yaml loadOverridableResource(
                        libraryResource: 'org/eclipsefdn/container/agent.yml'
                    )
                }
            }
            steps {
                container('containertools') {
                    containerBuild(
                        credentialsId: '<jenkins-credential-id>',
                        name: 'docker.io/<namespace-name>/<container-name>',
                        version: 'latest'
                    )
                }
            }
        }
    }
}

How do I use /opt/tools in a custom container?

You need to specify the tools persistence volume. 

pipeline {
  agent {
    kubernetes {
      label 'my-agent-pod'
      yaml """
apiVersion: v1
kind: Pod
spec:
  containers:
  - name: custom-name
    image: my-custom-image:latest
    tty: true
    command:
    - cat
    volumeMounts:
    - name: tools
      mountPath: /opt/tools
  volumes:
  - name: tools
    persistentVolumeClaim:
      claimName: tools-claim-jiro-<project_shortname>
"""
    }
  }
  stages {
    stage('Run maven') {
      steps {
        container('custom-name') {
          sh '/opt/tools/apache-maven/latest/bin/mvn -version'
        }
      }
    }
  }
}
    • Important: Do not forget to replace <project_shortname> in the claimName with your project name (e.g. tools-claim-jiro-cbi for the CBI project).**

In a custom container the build can't write to /home/jenkins, what do I need to do?

Due to recent changes in the Jenkins Kubernetes plugin, you need to specify an empty dir volume for /home/jenkins, if your build uses a directory like /home/jenkins/.ivy2 or /home/jenkins/.npm. 

pipeline {
  agent {
    kubernetes {
      label 'my-agent-pod'
      yaml """
apiVersion: v1
kind: Pod
spec:
  containers:
  - name: custom-name
    image: my-custom-image:latest
    tty: true
    command:
    - cat
    volumeMounts:
    - mountPath: "/home/jenkins"
      name: "jenkins-home"
      readOnly: false
  volumes:
  - name: "jenkins-home"
    emptyDir: {}
"""
    }
  }
  stages {
    stage('Run maven') {
      steps {
        container('custom-name') {
          sh 'mkdir -p /home/jenkins/foobar'
        }
      }
    }
  }
}
    • Note: We are not satisfied with this workaround and are actively looking for a more convenient way to let projects use custom containers without specifying a bunch of volume mounts.**

How do I deploy artifacts to download.eclipse.org?

You cannot just cp stuff to a folder. You need to do that with ssh. Therefore SSH credentials need to be set up on the project's Jenkins instance. This is already set up by default for all instances on our infrastructure.

This service provide access to the Eclipse Foundation file servers storage:

  • /home/data/httpd/download.eclipse.org
  • /home/data/httpd/archive.eclipse.org
  • /home/data/httpd/download.polarsys.org
  • /home/data/httpd/download.locationtech.org
Depending on how you run your build, the way you will use them are different. See the different cases below.

Freestyle job

You need to activate the SSH Agent plugin in your job configuration and select the proper credentials genie.''projectname'' (ssh://projects-storage.eclipse.org).

800px

Then you can use ssh, scp, rsync and sftp commands to deploy artifacts to the server, e.g., 

scp -o BatchMode=yes target/my_artifact.jar genie.<projectname>@projects-storage.eclipse.org:/home/data/httpd/download.eclipse.org/<projectname>/
ssh -o BatchMode=yes genie.<projectname>@projects-storage.eclipse.org ls -al /home/data/httpd/download.eclipse.org/<projectname>/
rsync -a -e ssh <local_dir> genie.<projectname>@projects-storage.eclipse.org:/home/data/httpd/download.eclipse.org/<projectname>/

Deployment with Maven

It is possible to deploy build output from within Maven, using Maven Wagon and wagon-ssh-external. As the build environment uses an SSH agent, the Maven Wagon plugins must use the external SSH commands so that the agent is used.

If the build outputs are executables or p2 update site and not Maven artifacts then the standard maven deploy needs to be disabled. E.g. with this in the appropriate profile in the parent/pom.xml 

<plugin>
	<artifactId>maven-deploy-plugin</artifactid>
	<configuration>
		<skip>true</skip>
	</configuration>
</plugin>

Define some properties for the destination in parent/pom.xml: 

<download-publish-path>/home/data/httpd/download.eclipse.org/[projectname]/snapshots/update-site</download-publish-path>
<download-remote-publish-path>genie.[projectname]@projects-storage.eclipse.org:/home/data/httpd/download.eclipse.org/[projectname]/snapshots/update-site</download-remote-publish-path>

Define the Wagon transport in parent/pom.xml: 

<build>
	<plugins>
		<plugin>
                ...
		</plugin>
	</plugins>
	<extensions>
		<extension>
			<groupId>org.apache.maven.wagon</groupid>
			<artifactId>wagon-ssh-external</artifactid>
			<version>3.0.0</version>
		</extension>
	</extensions>
</build>

Do the actual upload during the deploy phase (be sure to add that to the Maven invocation). 

<plugin>
	<groupId>org.codehaus.mojo</groupid>
	<artifactId>wagon-maven-plugin</artifactid>
	<version>2.0.0</version>
	<executions>
		<execution>
			<id>prepare-publish</id>
			<phase>deploy</phase>
			<goals>
				<goal>sshexec</goal>
			</goals>
			<configuration>
				<url>scpexe://${download-remote-publish-path}</url>
				<commands>
					<command>rm -rf ${download-publish-path}/*</command>
				</commands>
			</configuration>
		</execution>
		<execution>
			<id>publish</id>
			<phase>deploy</phase>
			<goals>
				<goal>upload</goal>
			</goals>
			<configuration>
				<fromDir>target/repository</fromdir>
				<includes>*/**</includes>
				<url>scpexe://${download-remote-publish-path}</url>
				<toDir></todir>
			</configuration>
		</execution>
	</executions>
</plugin>
This uses the sshexec goal to delete old files and upload to copy new files. Note */** for all directories. <toDir></todir> appears to be relative to the path given in the URL.

Be careful with paths and properties to ensure you upload to the correct place and do not delete the wrong thing.

Eclipse Memory Analyzer uses the above with Maven Wagon to deploy the snapshot nightly builds.

Pipeline job without custom pod template 

pipeline {
  agent any

  stages {
    stage('stage 1') {
      ...
    }
    stage('Deploy') {
      steps {
        sshagent ( ['projects-storage.eclipse.org-bot-ssh']) {
          sh 
            ssh -o BatchMode=yes genie.projectname@projects-storage.eclipse.org rm -rf /home/data/httpd/download.eclipse.org/projectname/snapshots
            ssh -o BatchMode=yes genie.projectname@projects-storage.eclipse.org mkdir -p /home/data/httpd/download.eclipse.org/projectname/snapshots
            scp -o BatchMode=yes -r repository/target/repository/* genie.projectname@projects-storage.eclipse.org:/home/data/httpd/download.eclipse.org/projectname/snapshots
          
        }
      }
    }
  }
}

Pipeline job with custom pod template

    • Important: A 'jnlp' container is automatically added, when a custom pod template is used to ensure connectivity between the Jenkins master and the pod. If you want to deploy files to download.eclipse.org, you only need to specify the known-hosts volume for the JNLP container (as seen below) to avoid "host verification failed" errors.**
pipeline {
  agent {
    kubernetes {
      label 'my-pod'
      yaml 
apiVersion: v1
kind: Pod
spec:
  containers:
  - name: maven
    image: maven:alpine
    command:
    - cat
    tty: true
  - name: jnlp
    volumeMounts:
    - name: volume-known-hosts
      mountPath: /home/jenkins/.ssh
  volumes:
  - name: volume-known-hosts
    configMap:
      name: known-hosts

    }
  }
  stages {
    stage('Build') {
      steps {
        container('maven') {
            sh 'mvn clean verify'
        }
      }
    }
    stage('Deploy') {
      steps {
        container('jnlp') {
          sshagent ( ['projects-storage.eclipse.org-bot-ssh']) {
            sh 
              ssh -o BatchMode=yes genie.projectname@projects-storage.eclipse.org rm -rf /home/data/httpd/download.eclipse.org/projectname/snapshots
              ssh -o BatchMode=yes genie.projectname@projects-storage.eclipse.org mkdir -p /home/data/httpd/download.eclipse.org/projectname/snapshots
              scp -o BatchMode=yes -r repository/target/repository/* genie.projectname@projects-storage.eclipse.org:/home/data/httpd/download.eclipse.org/projectname/snapshots
            
          }
        }
      }
    }
  }
}

How do I use the local Nexus server as proxy for Maven Central (artifact caching)?

Every JIPP has a Maven settings file set up that specifies our local Nexus instance as cache for Maven Central.

    • Important: In Jiro this works out of the box for the default pod templates (labels: basic, centos-7). No additional configuration is required for Freestyle and Pipeline jobs. For custom containers, see below: Custom container on Jiro**

Custom container on Jiro

You need to add the settings-xml volume, like shown below. Please note, the m2-repo volume is required as well, otherwise /home/jenkins/.m2/repository is not writable.

    • Note: In custom containers the user.home environment variable needs to be set to /home/jenkins via MAVEN_OPTS, otherwise settings.xml and settings-security.xml can not be found.**
pipeline {
  agent {
    kubernetes {
      label 'my-agent-pod'
      yaml """
apiVersion: v1
kind: Pod
spec:
  containers:
  - name: maven
    image: maven:alpine
    tty: true
    command:
    - cat
    env:
    - name: "MAVEN_OPTS"
      value: "-Duser.home=/home/jenkins"
    volumeMounts:
    - name: settings-xml
      mountPath: /home/jenkins/.m2/settings.xml
      subPath: settings.xml
      readOnly: true
    - name: m2-repo
      mountPath: /home/jenkins/.m2/repository
  volumes:
  - name: settings-xml
    secret:
      secretName: m2-secret-dir
      items:
      - key: settings.xml
        path: settings.xml
  - name: m2-repo
    emptyDir: {}
"""
    }
  }
  stages {
    stage('Run maven') {
      steps {
        container('maven') {
          sh 'mvn -version'
        }
      }
    }
  }
}

How can artifacts be deployed to Nexus OSS (repo.eclipse.org)?

If your project does not have its own repo on Nexus yet, then simply file a bug and specify what project you'd like a Nexus repo for.

If your project does have its own repo on Nexus already, then you can use Maven (or Gradle) to deploy your artifacts. This is also described here: https://wiki.eclipse.org/Services/Nexus#Deploying_artifacts_to_repo.eclipse.org.

    • Note: On our cluster-based infra (Jiro), a separate Maven settings file for deployment to Nexus is not required. All information is contained in the default Maven settings file located at /home/jenkins/.m2/settings.xml, which does not need to be specified explicitly in your job configuration.**

Custom container on Jiro

You need to add the settings-xml volume, like shown below. Please note, the m2-repo volume is required as well, otherwise /home/jenkins/.m2/repository is not writable.

    • Note: In custom containers the user.home environment variable needs to be set to /home/jenkins via MAVEN_OPTS, otherwise settings.xml and settings-security.xml can not be found.**
pipeline {
  agent {
    kubernetes {
      label 'my-agent-pod'
      yaml """
apiVersion: v1
kind: Pod
spec:
  containers:
  - name: maven
    image: maven:alpine
    tty: true
    command:
    - cat
    env:
    - name: "MAVEN_OPTS"
      value: "-Duser.home=/home/jenkins"
    volumeMounts:
    - name: settings-xml
      mountPath: /home/jenkins/.m2/settings.xml
      subPath: settings.xml
      readOnly: true
    - name: settings-security-xml
      mountPath: /home/jenkins/.m2/settings-security.xml
      subPath: settings-security.xml
      readOnly: true
    - name: m2-repo
      mountPath: /home/jenkins/.m2/repository
  volumes:
  - name: settings-xml
    secret:
      secretName: m2-secret-dir
      items:
      - key: settings.xml
        path: settings.xml
  - name: settings-security-xml
    secret:
      secretName: m2-secret-dir
      items:
      - key: settings-security.xml
        path: settings-security.xml
  - name: m2-repo
    emptyDir: {}
"""
    }
  }
  stages {
    stage('Run maven') {
      steps {
        container('maven') {
          sh 'mvn clean deploy'
        }
      }
    }
  }
}

How can artifacts be deployed to OSSRH / Maven Central?

Deploying artifacts to OSSRH (OSS Repository Hosting provided by Sonatype) requires an account at OSSRH. It is also required to sign all artifacts with GPG. The Eclipse IT team will set this up for the project.
Please file a bug for this first.

    • Note: On our cluster-based infra (Jiro), a separate Maven settings file for deployment to OSSRH is not necessary. All information is contained in the default Maven settings file located at /home/jenkins/.m2/settings.xml, which does not need to be specified explicitly in your job configuration.
      If you are using a custom container, please see https://wiki.eclipse.org/Jenkins#Custom_container_on_Jiro_3**

Required steps for a freestyle job

1. Insert secret-subkeys.asc as secret file in job File:InjectSecretFile2.png
2. Import GPG keyring with --batch and trust the keys non-interactively in a shell build step (before the Maven call) gpg --batch --import "${KEYRING}" for fpr in $(gpg --list-keys --with-colons | awk -F: '/fpr:/ {print $10}' | sort -u); do echo -e "5\ny\n" | gpg --batch --command-fd 0 --expert --edit-key $fpr trust; done 700px
3. If a newer GPG version (> 2.1+) is used, --pinentry-mode loopback needs to be added as gpg argument in the pom.xml. This does not need to be added when using the "centos-7"/"migration" pod template (which uses GPG 2.0.22)! 
 <plugin>
   &lt;groupid&gt;org.apache.maven.plugins&lt;/groupid&gt;
   &lt;artifactid&gt;maven&#45;gpg&#45;plugin&lt;/artifactid&gt;
   &lt;version&gt;1.6&lt;/version&gt;
   &lt;executions&gt;
     &amp;lt;execution&amp;gt;
       &amp;amp;lt;id&amp;amp;gt;sign&amp;amp;#45;artifacts&amp;amp;lt;/id&amp;amp;gt;
         &amp;amp;lt;phase&amp;amp;gt;verify&amp;amp;lt;/phase&amp;amp;gt;
         &amp;amp;lt;goals&amp;amp;gt;
           &amp;amp;amp;lt;goal&amp;amp;amp;gt;sign&amp;amp;amp;lt;/goal&amp;amp;amp;gt;
         &amp;amp;lt;/goals&amp;amp;gt;
         &amp;amp;lt;configuration&amp;amp;gt;
           &amp;amp;amp;lt;gpgarguments&amp;amp;amp;gt;
             &amp;amp;amp;amp;lt;arg&amp;amp;amp;amp;gt;&amp;amp;amp;amp;#45;&amp;amp;amp;amp;#45;pinentry&amp;amp;amp;amp;#45;mode&amp;amp;amp;amp;lt;/arg&amp;amp;amp;amp;gt;
             &amp;amp;amp;amp;lt;arg&amp;amp;amp;amp;gt;loopback&amp;amp;amp;amp;lt;/arg&amp;amp;amp;amp;gt;
           &amp;amp;amp;lt;/gpgarguments&amp;amp;amp;gt;
         &amp;amp;lt;/configuration&amp;amp;gt;
     &amp;lt;/execution&amp;gt;
   &lt;/executions&gt;
 </plugin>

Required steps for a pipeline job

This is a simple pipeline job, that allows to test the GPG signing. 

pipeline {
    agent any
    tools {
        maven 'apache-maven-latest'
        jdk 'adoptopenjdk-hotspot-jdk8-latest'
    }
    stages {
        stage('Build') {
            steps {
                sh "mvn -B -U archetype:generate -DgroupId=com.mycompany.app -DartifactId=my-app -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false"
                sh cat >my-app/pom.xml <<EOL
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelversion>
  <groupId>com.mycompany.app</groupid>
  <artifactId>my-app</artifactid>
  <packaging>jar</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>my-app</name>
  <url>http://maven.apache.org</url>
  <dependencies>
    <dependency>
      <groupId>junit</groupid>
      <artifactId>junit</artifactid>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupid>
        <artifactId>maven-gpg-plugin</artifactid>
        <version>1.6</version>
        <executions>
          <execution>
            <id>sign-artifacts</id>
            <phase>verify</phase>
            <goals>
              <goal>sign</goal>
            </goals>
            <configuration>
              <gpgArguments>
                <arg>--pinentry-mode</arg>
                <arg>loopback</arg>
              </gpgarguments>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>
EOL
                withCredentials([file(credentialsId: 'secret-subkeys.asc', variable: 'KEYRING')]) {
                    sh 'gpg --batch --import "${KEYRING}"'
                    sh 'for fpr in $(gpg --list-keys --with-colons  | awk -F: \'/fpr:/ {print $10}\' | sort -u); do echo -e "5\ny\n" |  gpg --batch --command-fd 0 --expert --edit-key ${fpr} trust; done'
                }
                sh "mvn -B -f my-app/pom.xml clean verify"
                sh 'gpg --verify my-app/target/my-app-1.0-SNAPSHOT.jar.asc'
            }
        }
    }
}

Custom container on Jiro

When you are using a custom container on Jiro, you will need to add the settings-xml and settings-security-xml volume, like shown below.

Please note: 

pipeline {
  agent {
    kubernetes {
      label 'my-agent-pod'
      yaml """
apiVersion: v1
kind: Pod
spec:
  containers:
  - name: maven
    image: maven:alpine
    tty: true
    command:
    - cat
    volumeMounts:
    - name: settings-xml
      mountPath: /home/jenkins/.m2/settings.xml
      subPath: settings.xml
      readOnly: true
    - name: toolchains-xml
      mountPath: /home/jenkins/.m2/toolchains.xml
      subPath: toolchains.xml
      readOnly: true
    - name: settings-security-xml
      mountPath: /home/jenkins/.m2/settings-security.xml
      subPath: settings-security.xml
      readOnly: true
    - name: m2-repo
      mountPath: /home/jenkins/.m2/repository
  volumes:
  - name: settings-xml
    secret:
      secretName: m2-secret-dir
      items:
      - key: settings.xml
        path: settings.xml
  - name: toolchains-xml
    configMap:
      name: m2-dir
      items:
      - key: toolchains.xml
        path: toolchains.xml
  - name: settings-security-xml
    secret:
      secretName: m2-secret-dir
      items:
      - key: settings-security.xml
        path: settings-security.xml
  - name: m2-repo
    emptyDir: {}
"""
    }
  }
  stages {
    stage('Run maven') {
      steps {
        container('maven') {
          sh 'mvn -version'
        }
      }
    }
  }
}

Common issues with GPG

Error message Solution
gpg: signing failed: Not a tty GPG version > 2.1 is used and --pinentry-mode loopback needs to be added to the maven-gpg-plugin config in the pom.xml (see above).
gpg: invalid option "--pinentry-mode" GPG version < 2.1 is used and --pinentry-mode loopback needs to be removed from the maven-gpg-plugin config in the pom.xml
gpg: no default secret key: No secret key gpg: signing failed: No secret key GPG keyring needs to be imported (see above)

How can SonarCloud be integrated into a Jiro JIPP?

Integrating SonarCloud into a project's CI builds is quite easy. Please open a Bugzilla issue for this and the releng/webmaster team will help you set this up.

We're setting this up with the webmaster's SonarCloud.io account, so there is no need to provide a SonarCloud token. To avoid leaking the token in the console log, we store it as secret text (Jenkins credentials).

We will configure the SonarQube Jenkins plugin to use SonarCloud to achieve a slightly better integration with Jenkins. For example, a link to SonarCloud will show up in the left menu of a job page.

Freestyle job

For a freestyle job configuration two things need to be done:

  1. Under "Build environment" enable "Use secret text(s) or file(s)", add "Secret text", name the variable "SONARCLOUD_TOKEN" and select the right credential (e.g. "Sonarcloud token").
  2. either a shell build step or a Maven build step can be used to run the sonar goal with the right parameters: 
    mvn clean verify sonar:sonar -Dsonar.projectKey=<project-name> -Dsonar.organization=<organization> -Dsonar.host.url=${SONAR_HOST_URL} -Dsonar.login=${SONARCLOUD_TOKEN}

Pipeline job

For a pipeline job the following needs to be added: 

withCredentials([string(credentialsId: 'sonarcloud-token', variable: 'SONARCLOUD_TOKEN')]) {
    withSonarQubeEnv('SonarCloud.io') {
        mvn clean verify sonar:sonar -Dsonar.projectKey=<project-name> -Dsonar.organization=<organization> -Dsonar.host.url=${SONAR_HOST_URL} -Dsonar.login=${SONARCLOUD_TOKEN}
    }
}

Please note: <project-name></project-name> and <organization></organization> should be replaced with the corresponding project name and organization.

Can projects get admin access on their Jiro JIPP?

In general, we want to avoid handing out admin rights in the future. At the same time, - in the spirit of configuration as code - we are planning to allow projects to submit pull requests to our Jiro repo and change the configuration of their CI instance. E.g. adding plugins, etc. This will allow better tracking of configuration changes and rollback in case of issues.

We understand that some projects heavily rely on their admin permissions. We will make sure to find an amicable solution in those cases.

How can a new plugin be added to a Jiro JIPP?

The preferred way is to open a pull request in the Jiro GitHub repo. For example, to add a new plugin to the CBI instance, one would need to edit https://github.com/eclipse-cbi/jiro/blob/master/instances/technology.cbi/config.jsonnet and add the ID of the plugin to the plugins+ section. If the jenkins+/plugins+ section does not exist yet, it needs to be added as well.

Example: 

&#123;
  project+&#58; &#123;
    fullName&#58; &quot;technology.cbi&quot;,
    displayName&#58; &quot;Eclipse CBI&quot;,
  &#125;,
  jenkins+&#58; &#123;
    plugins+&#58; &#91;
      &quot;jacoco&quot;,
    &#93;,
  &#125;
&#125;

Only additional plugins that are not listed in https://wiki.eclipse.org/Jenkins#Default_plugins_-_Jiro_.28ci-staging.eclipse.org.2Fxxx_and_ci.eclipse.org.2Fxxx.29 need to be added to the config.jsonnet file.

The ID of a Jenkins plugin can be found here: https://plugins.jenkins.io/

If this sounds to complicated, you can also open a bug on Bugzilla.

How to build my project's website with Jenkins?

The preferred static website generator for build Eclipse project websites is Hugo. You should first put your Hugo sources in a dedicated Git repository, either at GitHub if your source code is already hosted at GitHub or at git.eclipse.org. If you don't have such a repository already, feel free to open a request, the Eclipse IT team will create one for you.

Note that each and every Eclipse project automatically gets a Git repository with git.eclipse.org/www.eclipse.org/&amp;lt&#59;project_name&amp;gt&#59; (see this repository index for complete list). This is not where you want to push your Hugo sources. This repository contains the webpages that are automatically and regularly pulled and published on the www.eclipse.org HTTP server. All the content from the master branch will eventually be available at the URL https://www.eclipse.org/<project_name/>.

Once your Hugo sources are in the proper repository, create a file named Jenkinsfile at the root of the repository with the following content (don't forget to specify the proper value for PROJECT_NAME and PROJECT_BOT_NAME environment variable): 

  environment &#123;
    PROJECT_NAME &#61; &quot;&amp;lt&#59;project_name&amp;gt&#59;&quot; // must be all lowercase.
    PROJECT_BOT_NAME &#61; &quot;&amp;lt&#59;Project_name&amp;gt&#59; Bot&quot; // Capitalize the name
  &#125;
 
  triggers &#123; pollSCM(&#39;H/10 &#42; &#42; &#42; &#42;&#39;) 

 &#125;

  options &#123;
    buildDiscarder(logRotator(numToKeepStr&#58; &#39;5&#39;))
    checkoutToSubdirectory(&#39;hugo&#39;)
  &#125;
 
  stages &#123;
    stage(&#39;Checkout www repo&#39;) &#123;
      steps &#123;
        dir(&#39;www&#39;) &#123;
            sshagent(&#91;&#39;git.eclipse.org&#45;bot&#45;ssh&#39;&#93;) &#123;
                sh &#39;&#39;&#39;
                    git clone ssh&#58;//genie.$&#123;PROJECT_NAME&#125;@git.eclipse.org&#58;29418/www.eclipse.org/$&#123;PROJECT_NAME&#125;.git .
                    git checkout $&#123;BRANCH_NAME&#125;
                &#39;&#39;&#39;
            &#125;
        &#125;
      &#125;
    &#125;
    stage(&#39;Build website (master) with Hugo&#39;) &#123;
      when &#123;
        branch &#39;master&#39;
      &#125;
      steps &#123;
        container(&#39;hugo&#39;) &#123;
            dir(&#39;hugo&#39;) &#123;
                sh &#39;hugo &#45;b https&#58;//www.eclipse.org/$&#123;PROJECT_NAME&#125;/&#39;
            &#125;
        &#125;
      &#125;
    &#125;
    stage(&#39;Build website (staging) with Hugo&#39;) &#123;
      when &#123;
        branch &#39;staging&#39;
      &#125;
      steps &#123;
        container(&#39;hugo&#39;) &#123;
            dir(&#39;hugo&#39;) &#123;
                sh &#39;hugo &#45;b https&#58;//staging.eclipse.org/$&#123;PROJECT_NAME&#125;/&#39;
            &#125;
        &#125;
      &#125;
    &#125;
    stage(&#39;Push to $env.BRANCH_NAME branch&#39;) &#123;
      when &#123;
        anyOf &#123;
          branch &quot;master&quot;
          branch &quot;staging&quot;
        &#125;
      &#125;
      steps &#123;
        sh &#39;rm &#45;rf www/&#42; &amp;&amp; cp &#45;Rvf hugo/public/&#42; www/&#39;
        dir(&#39;www&#39;) &#123;
            sshagent(&#91;&#39;git.eclipse.org&#45;bot&#45;ssh&#39;&#93;) &#123;
                sh &#39;&#39;&#39;
                git add &#45;A
                if &#33; git diff &#45;&#45;cached &#45;&#45;exit&#45;code&#59; then
                  echo &quot;Changes have been detected, publishing to repo &#39;www.eclipse.org/$&#123;PROJECT_NAME&#125;&#39;&quot;
                  git config user.email &quot;$&#123;PROJECT_NAME&#125;&#45;bot@eclipse.org&quot;
                  git config user.name &quot;$&#123;PROJECT_BOT_NAME&#125;&quot;
                  git commit &#45;m &quot;Website build $&#123;JOB_NAME&#125;&#45;$&#123;BUILD_NUMBER&#125;&quot;
                  git log &#45;&#45;graph &#45;&#45;abbrev&#45;commit &#45;&#45;date&#61;relative &#45;n 5
                  git push origin HEAD&#58;$&#123;BRANCH_NAME&#125;
                else
                  echo &quot;No changes have been detected since last build, nothing to publish&quot;
                fi
                &#39;&#39;&#39;
            &#125;
        &#125;
      &#125;
    &#125;
  &#125;
&#125;

Finally, you can create a multibranch pipeline job on your project's Jenkins instance. It will automatically be triggered on every new push to your Hugo source repository, build the website and push it to the git.eclipse.org/www.eclipse.org/&amp;lt&#59;project_name&amp;gt&#59;.git repository. As mentioned above, the Eclipse Foundation website's infrastructure will eventually pull the content of the latter and your website will be published and available on http://www.eclipse.org/<project_name>.

If you don't have a Jenkins instance already, ask for one. If you need assistance with the process, open a ticket.

How to accept fork contribution?

By default, Jenkins project configurations using the GitLab Branch Source plugin are set up with 'Trusted Members' as the default option.

Definition of 'trusted members': [Recommended] Discover merge requests from forked projects whose authors have Developer/Maintainer/Owner access levels in the origin project.

To accept contributions from contributors with fork project, the project should:

  • Configure the CI project by changing 'Discover merge requests from forks' to 'Members'.
  • The project lead should add the user to the list of collaborators in PMI.
  • The contributor should change the forked project's visibility to public.

GitLab integration

The Jenkins GitLab Plugin and GitLab Branch Source Plugin will be installed by default on every Jiro instance (yours might need to be redeployed). The GitLab integration should work similar to the integration with GitHub and the respective plugins, yet some naming/functionality might be different or missing.

Please let us know if you want to use GitLab with Jenkins and open a Bugzilla issue (Product: Community, Component: CI-Jenkins). We will help you set up the integration on your Jenkins instance.

Here is what we will do:

  • create a GitLab bot user
  • create a GitLab API token
  • set up a webhook that can trigger builds on pushes or merge requests

Gerrit integration

Gerrit Trigger Plugin

You may use the Jenkins Gerrit Trigger Plugin in order to run a Jenkins job to verify each new patchset uploaded to Gerrit for code review. Jenkins (named "CI Bot") will then also vote on these changes using the "Verify" voting category.

450px

450px

Below, the configuration sections for the Git plugin and the Gerrit trigger plugin of the verification job used by the EGit project may serve as an example.

General configuration settings

  1. Check This project is parameterized. Click the Add button and select String Parameter. Set the parameter Name to GERRIT_REFSPEC and Default Value to refs/heads/master.
600px

Configuration of Source Code Management

  1. Under Source Code Management select Git.
  2. Under Repositories, click on Advanced and change the Refspec to ${GERRIT_REFSPEC}.
  3. Under Additional Behaviours, add Strategy for choosing what to build and select Gerrit Trigger as a strategy.
600px

Note that the section Branches to build won't be used and may be deleted.

Configuration of Build Triggers

  1. Under Build Triggers, select Gerrit event.
600px
  1. Under Trigger on, click on Add and select at least Patchset Created. This will configure the job to run on each new patchset. You can also add additional trigger, like Comment Added Contains Regular Expression. In the example below, a build will be triggered for the latest patch set if the comment is exactly CI Bot, run a build please.
600px
  1. Finally, configure at least one Gerrit Project. The pattern is the name of project (i.e. if your repository is git.eclipse.org/&amp;lt&#59;xx&amp;gt&#59;/&amp;lt&#59;yy&amp;gt&#59;.git, then fill the pattern xx/yy). The Branches section is the list of branch to listen for events as configured above. Generally, you want one, named master to build patches submitted for the master branche, or &amp;&#35;42&#59;&amp;&#35;42&#59; to build patches submitted to each and every branches. Set the type to Path.
600px

Configuration of the build action

Under "Build" click the "Add a build step" button, and select the appropriate action. The actual action depends on what you want Hudson to do. A typical example, for projects build with Maven is to select "Invoke Maven 3" and set "Maven 3" to "apache-maven-latest" and "Goals" to "clean verify".

Cluster migration

See CBI/Jenkins_Migration_FAQ

Clone this wiki locally