Traces of Jobs and Pipeline Builds

Traces, Spans and Span Attributes

The Jenkins OpenTelemetry integration collects comprehensive contextual attributes of the jobs and pipelines builds to:

Provide build executions details in order to be an alternative to the Jenkins GUI if desired
Enable troubleshooting CI problems slicing and dicing in any dimension
Offer traceability and audit on CI/CD as part of the Software Lifecycle process integrity (see Introducing SLSA, an End-to-End Framework for Supply Chain Integrity...)

The attributes of the traces and spans are normalized in order

Enriching pipeline build traces with custom span attributes

Attributes can be added to the spans associated with pipeline steps using one of the following pipeline steps:

withSpanAttribute(key, value[, type][, target])
setSpanAttributes([spanAttribute(key, value[, type][, target])])
withSpanAttributes([spanAttribute(key, value[, type][, target])])

Example:

withSpanAttribute(key: "pipeline_type", value: "release_pipeline", target: "PIPELINE_ROOT_SPAN")
...
stage ("build") {
    withSpanAttribute(key: "build.tool", value: "maven")
    setSpanAttributes([spanAttribute(key: "build.tool", value: "maven")])
    withSpanAttributes([spanAttribute(key: "test.tool", value: "junit")]) {
        sh "./run-tests.sh"
    }
}

Supported attribute types are STRING, BOOLEAN, LONG, DOUBLE, STRING_ARRAY, BOOLEAN_ARRAY, LONG_ARRAY, DOUBLE_ARRAY. If not specified, the type of the attribute is inferred from the passed value. It is recommended to not specify the type and to rely on the inference from the value.
Supported targets are:
- CURRENT_SPAN (default value): the attribute is set on the span of the current pipeline phase (current node, stage, step...)
- PIPELINE_ROOT_SPAN: the attribute is set on the root span of the pipeline (ie the "BUILD ..." span)

Note that none of the steps setSpanAttributes, withSpanAttribute, withSpanAttributes create a span in the pipeline build trace.

withSpanAttribute and setSpanAttributes set the given attribute(s) on the target.

withSpanAttributes sets the given attribute(s) on child spans.

It's also possible to use withSpanAttributes in declarative pipelines:

pipeline {
    options {
        withSpanAttributes([spanAttribute(key: 'pipeline.type', value: 'release', target: 'PIPELINE_ROOT_SPAN')])
    }
    stages {
        stage('build') {
            ...
        }
    }
}

Custom spans

Custom spans can be defined using the withNewSpan step, which accepts the following parameters

label
- the label of the span
- the value is a string

attributes

a list of attributes, defined the same way as in the withSpanAttributes step

attributes: ([
    spanAttribute(key: 'modules', value: '2'),
    spanAttribute(key: 'command', value: 'mvn clean install')
])

setAttributesOnlyOnParent
- flag used to define whether to inherit the provided attributes to the children spans or not
- true by default, all user-defined attributes for a span are passed on to children spans
- the value is a boolean, true or false

Example definitions:

All parameters provided

stage('build') {
    withNewSpan(label: 'custom-build-span', attributes: ([
        spanAttribute(key: 'modules', value: '2'),
        spanAttribute(key: 'command', value: 'mvn clean install')
    ]), setAttributesOnlyOnParent: true) {
        sh './build-module1.sh'
        sh './build-module2.sh'
    }
}

Only the label parameter is required, all others are optional.

stage('build') {
    withNewSpan(label: 'custom-build-span') {
        sh './build-module1.sh'
        sh './build-module2.sh'
    }
}

Pipeline, freestyle, and matrix project build spans

Attributes reported on the root span of Jenkins job and pipeline builds:

Attribute	Description	Type
ci.pipeline.id	Job name	String
ci.pipeline.name	Job name (user friendly)	String
ci.pipeline.type	Job type	Enum (`freestyle`, `matrix`, `maven`, `workflow`, `multibranch`, `unknown`)
ci.pipeline.template.id	Job DSL seed job full name. Added when the executed job has been generated by the Job DSL plugin	String
ci.pipeline.template.url	Job DSL seed job URL. Added when the executed job has been generated by the Job DSL plugin	String
ci.pipeline.multibranch.type	Multibranch type	Enum (`branch`, `tag`, `change_request`)
ci.pipeline.axis.names	When using matrix projects, names of the axis of the job	String[]
ci.pipeline.axis.values	When using matrix projects, values of the axis of the job	String[]
ci.pipeline.agent.id	Name of the agent	String
ci.pipeline.run.completed	Is this a complete build?	Boolean
ci.pipeline.run.durationMillis	Build duration	Long
ci.pipeline.run.description	Build description	String
ci.pipeline.run.number	Build number	Long
ci.pipeline.run.result	Build result	Enum (`aborted`, `success`, `failure`, `not_build` and `unstable`)
ci.pipeline.run.url	Build URL	String
ci.pipeline.run.user	Who triggered the build	String
ci.pipeline.run.cause	List of machine-readable build causes like `UserIdCause:anonymous` or `BranchIndexingCause`. Pattern : `${cause.class.simpleName}[:details]`	String[]
ci.pipeline.run.committers	List of users that caused the build.	String[]
ci.pipeline.parameter.sensitive	Whether the information contained in this parameter is sensitive or security related.	Boolean[]
ci.pipeline.parameter.name	Name of the parameters	String[]
ci.pipeline.parameter.value	Value of the parameters. "Sensitive" values are redacted	String[]

Pipeline step spans

Attributes reported on the span of pipeline steps:

Status Code	Status Description	Description
OK		for step and build success
UNSET	Machine readable status like `FlowInterruptedException:FailFastCause:Failed in branch failingBranch`	For interrupted steps of type fail fast parallel pipeline interruption, pipeline build superseded by a newer build, or pipeline build cancelled by user, the span status is set to `UNSET` rather than `ERROR` for readability
ERROR	Machine readable status like `FlowInterruptedException:ExceededTimeout:Timeout has been exceeded`	For other causes of step failure

Attribute	Description	Type
jenkins.pipeline.step.name	Step name (user friendly)	String
jenkins.pipeline.step.type	Step name	String
jenkins.pipeline.step.id	Step id	String
jenkins.pipeline.step.result	Step result	Enum (`ABORTED`, `FAILURE`, `NOT_EXECUTED`, `PAUSED_PENDING_INPUT`, `QUEUED`, `SUCCESS`, `UNSTABLE`; see GenericStatus)
jenkins.pipeline.step.plugin.name	Jenkins plugin for that particular step	String
jenkins.pipeline.step.plugin.version	Jenkins plugin version	String
jenkins.pipeline.step.agent.label	Labels attached to the agent	String
jenkins.pipeline.step.interruption.causes	List of machine readable causes of the interruption of the step like `FailFastCause:Failed in branch failingBranch`. Common causes of interruption: `CanceledCause: Superseded by my-pipeline#123`, `ExceededTimeout: Timeout has been exceeded`, `FailFastCause:Failed in branch the-failing-branch`, `UserInterruption: Aborted by a-user`	String[]
git.branch	Git branch name	String
git.repository	Git repository	String
git.username	Git user	String
git.clone.shallow	Git shallow clone	Boolean
git.clone.depth	Git shallow clone depth	Long
git.username	Git user	String
jenkins.url	Jenkins URL	String
jenkins.computer.name	Name of the agent	String

Environment variables for trace context propagation and integrations

The trace context is exposed as environment variables in shell/bat/powershell steps to ease integration with third party tools.

TRACEPARENT: the W3C Trace Context header traceparent
TRACESTATE: the W3C Trace Context header tracestate
TRACE_ID: the trace id of the job / pipeline build
SPAN_ID: the id of the pipeline shell/bat/powershell step span

When the configuration options "Export OpenTelemetry configuration as environment variables", the following OpenTelemetry environment variables will be exported according to the settings of the plugin to enable configuring downstream OpenTelemetry aware CI/CD tools:

OTEL_EXPORTER_OTLP_ENDPOINT: Target to which the exporter is going to send spans or metrics.
OTEL_EXPORTER_OTLP_INSECURE: Whether to enable client transport security for the exporter's gRPC connection.
OTEL_EXPORTER_OTLP_HEADERS: Key-value pairs to be used as headers associated with gRPC or HTTP requests. Usually used to pass credentials.
OTEL_EXPORTER_OTLP_TIMEOUT: Maximum time the OTLP exporter will wait for each batch export.
OTEL_EXPORTER_OTLP_CERTIFICATE: The trusted certificate to use when verifying a server's TLS credentials.

In addition, if the backends were configured then there will be an environment variable for each of them pointing to the URL with the span/transactions:

OTEL_CUSTOM_URL
OTEL_ELASTIC_URL
OTEL_JAEGER_URL
OTEL_ZIPKIN_URL

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

job-traces.md

job-traces.md

Traces of Jobs and Pipeline Builds

Traces, Spans and Span Attributes

Enriching pipeline build traces with custom span attributes

Custom spans

Pipeline, freestyle, and matrix project build spans

Pipeline step spans

Environment variables for trace context propagation and integrations

Files

job-traces.md

Latest commit

History

job-traces.md

File metadata and controls

Traces of Jobs and Pipeline Builds

Traces, Spans and Span Attributes

Enriching pipeline build traces with custom span attributes

Custom spans

Pipeline, freestyle, and matrix project build spans

Pipeline step spans

Environment variables for trace context propagation and integrations