Incrementals

See JEP-305 for context.

Usage in plugin POMs

Since most Jenkins repositories host plugins, this use case will be documented first. You must be using parent POM version 3.10 or later.

Enabling incrementals (the easy way)

Just run

mvn io.jenkins.tools.incrementals:incrementals-maven-plugin:incrementalify -DgenerateBackupPoms=false

or if your POM is already new enough (3.10+)

mvn incrementals:incrementalify

Check the usual build

mvn clean package

and if all is well,

git add .mvn pom.xml
git checkout -b incrementals
git commit -m Incrementalified.

and file as a pull request.

Enabling incrementals (the hard way)

Enabling consumption of incrementals

If your plugin has (or may have) dependencies on incremental versions, run:

mkdir -p .mvn
echo -Pconsume-incrementals >> .mvn/maven.config
git add .mvn

(See this guide for details on the .mvn directory.)

This profile merely activates access to the Incrementals repository.

Enabling production of incrementals

To produce incremental artifacts from your plugin, first edit your pom.xml. If your plugin declares

<version>1.23-SNAPSHOT</version>

then replace that with

<version>${revision}${changelist}</version>

and then in the <properties> section add

<revision>1.23</revision>
<changelist>-SNAPSHOT</changelist>

If you have a multimodule reactor build, the new properties need be defined only in the root POM, but every child POM should use the edited version to refer to its parent. (It should not override the version.) Intermodule dependencyes may use ${project.version} to refer to the version of the sibling.

Also change

<scm>
  <!-- … -->
  <tag>HEAD</tag>
</scm>

to

<scm>
  <!-- … -->
  <tag>${scmTag}</tag>
</scm>

Also replace jenkinsci/your-repo occurrences in the scm section with a POM property named gitHubRepo. This will ensure that deployments from forked pull requests will give correct checkout information, used for example by plugin-compat-tester.

Now run

mkdir -p .mvn
echo -Pmight-produce-incrementals >> .mvn/maven.config

Finally, configure git-changelist-maven-extension in .mvn/extensions.xml. (Update the version to the latest version for this tool.):

<extensions xmlns="http://maven.apache.org/EXTENSIONS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/EXTENSIONS/1.0.0 http://maven.apache.org/xsd/core-extensions-1.0.0.xsd">
  <extension>
    <groupId>io.jenkins.tools.incrementals</groupId>
    <artifactId>git-changelist-maven-extension</artifactId>
    <version>1.4</version>
  </extension>
</extensions>

You may now

git add .mvn pom.xml

and commit and push your edits.

Production and consumption

A single plugin may both consume Incrementals releases, and produce its own. Just make both kinds of edits. (.mvn/maven.config may have multiple lines.)

Producing incrementals

Assumes you have set up the might-produce-incrementals as above, either by hand or using the incrementalify goal.

If you file a pull request built on ci.jenkins.io, and the pull request is up to date with its target branch, and the build is stable, the artifact will be automatically deployed to the Incrementals repository. Your plugin will need to have a github field in repository-permissions-updater if it does not already.

To check from a shell if your deployment is ready, get the gh CLI and try:

$ gh pr checks | fgrep incrementals | awk '{print $4}'
https://repo.jenkins-ci.org/incrementals/org/jenkins-ci/plugins/ldap/1.27-rc474.9577fe6b8ea6/

To produce equivalent artifacts in your local repository while working offline:

mvn -Dset.changelist -DskipTests clean install

If you do not select the -Dset.changelist option, you will create a regular *-SNAPSHOT artifact. (And that is what you must do if you have any local modifications or untracked files.)

Updating dependencies

Once you have some dependencies on incremental versions in your POM, you can

mvn incrementals:update

to get a newer version of some dependencies, if merged; or

mvn incrementals:update -Dbranch=yourghacct:experiments-JENKINS-12345

to get the most recent versions from some set of unmerged PRs. Then commit and push the resulting pom.xml edits.

You will need GitHub credentials for this: instructions

Updating versions for Jenkins Docker images

Official Jenkins Docker images offer plugins.txt which supports Incrementals. See this page for more information. Incrementals maven plugin can be used to update plugin versions there. Currently, only Incrementals version update is supported.

mvn incrementals:updatePluginsTxt -DpluginsFile=plugins.txt

When plugins.txt format is used, it is also possible to pass the branch name to the incrementals definition so that the Incrementals version is updated from a particular branch.

Example of the file with incrementals:

scm-api:latest
script-security:latest
workflow-aggregator:2.5
# https://github.com/jenkinsci/workflow-api-plugin/pull/17
workflow-api:incrementals;org.jenkins-ci.plugins.workflow;2.30-rc-802-fa-5-alpha-94-c-8-alpha-65;jglick;logs-JENKINS-38381
workflow-basic-steps:latest
# https://github.com/jenkinsci/workflow-durable-task-step-plugin/pull/21
workflow-durable-task-step:incrementals;org.jenkins-ci.plugins.workflow;jglick;2.20-rc333.74dc7c303e6d
# https://github.com/jenkinsci/workflow-job-plugin/pull/27
workflow-job:incrementals;org.jenkins-ci.plugins.workflow;2.25-rc-824.49-c-91-cd-14666;jglick;logs-JENKINS-38381
workflow-multibranch:latest
# https://github.com/jenkinsci/workflow-support-plugin/pull/15
workflow-support:incrementals;org.jenkins-ci.plugins.workflow;2.21-rc-617.27-alpha-34-dc-2-c-64-c;jglick;logs-JENKINS-38381
# Incrementals from the master branch
artifact-manager-s3:incrementals;io.jenkins.plugins;1.2-rc82.a1e113b09b19

Running Maven releases

You may still use the Maven release plugin (MRP) when might-produce-incrementals is activated:

mvn -B release:{prepare,perform}

The released artifacts should have sensible metadata. (You may notice that they deploy a “flattened” POM file, but this should not break anything.)

Sufficiently recent parent POMs (3.18+) also include a incrementals:reincrementalify mojo run as part of completion goals, so you will notice that the [maven-release-plugin] prepare for next development iteration commit brings your source tree back to a state where the plugin is ready to produce Incrementals. To verify that this is working, after running a release try running

git diff HEAD^^

You should see something like

--- a/pom.xml
+++ b/pom.xml
   <version>${revision}${changelist}</version>
   <properties>
-    <revision>1.1</revision>
+    <revision>1.2</revision>
     <changelist>-SNAPSHOT</changelist>
   </properties>

indicating that the net effect of the [maven-release-plugin] prepare release something-1.1 commit and the commit after it is to change the plugin from 1.1-SNAPSHOT to 1.2-SNAPSHOT. If this failed and your <version> was still a number, you can manually run

mvn incrementals:reincrementalify

to fix it up.

Superseding Maven releases

See Setting up automated plugin release

Usage in other POMs

From repositories with POMs not inheriting from org.jenkins-ci.plugins:plugin you can follow similar steps to use Incrementals. If you inherit from org.jenkins-ci:jenkins, the same profiles are available; otherwise you will need to copy the definitions of the consume-incrementals, might-produce-incrementals, and produce-incrementals profiles from org.jenkins-ci:jenkins, as well as the incrementals.url and scmTag properties, into your parent POM or directly into your repository POM. Some adjustment of maven-enforcer-plugin configuration may also be necessary.

Publishing

Once Incrementals is enabled in a plugin repository, the stock buildPlugin method takes care of publishing artifacts from stable builds up to date with the base branch. For libraries or other components with custom Jenkinsfiles, you will need to set this up manually:

node('maven') {
  checkout scm
  sh 'mvn -Dset.changelist install'
  infra.prepareToPublishIncrementals()
}
infra.maybePublishIncrementals()

Offline testing

If you wish to test usage offline, run

docker run --rm --name nexus -p 8081:8081 -v nexus-data:/nexus-data sonatype/nexus3

Log in to http://localhost:8081/ and pick an admin password as per instructions, then edit your ~/.m2/settings.xml to:

<settings>
  <servers>
    <server>
      <id>incrementals</id>
      <username>admin</username>
      <password>admin123</password>
    </server>
  </servers>
  <mirrors>
    <mirror>
      <id>incrementals-mirror</id>
      <url>http://localhost:8081/repository/maven-releases/</url>
      <mirrorOf>incrementals</mirrorOf>
    </mirror>
  </mirrors>
</settings>

and then add to the command line when producing incremental versions with maven-deploy-plugin:

-DaltDeploymentRepository=incrementals::http://localhost:8081/repository/maven-releases/

Changelog

1.1 and later

See GitHub releases.

1.0-beta-7

2018 Sep 04

• mvn incrementals:update mishandled property expressions.

1.0-beta-6

2018 Aug 30

• New mvn incrementals:updatePluginsTxt goal.

1.0-beta-5

2018 Jul 24

• Support mvn incrementals:incrementalify on projects using the org.jenkins-ci:jenkins parent.

1.0-beta-4

2018 Jul 19

• JENKINS-51869: no longer using --first-parent in revision count.
• Match indentation in mvn incrementals:incrementalify.
• Make mvn incrementals:reincrementalify fail comprehensibly on a non-incrementalified repository.

1.0-beta-3 and earlier

Not recorded.