Igor is a service that provides a single point of integration with Continuous Integration (CI) and Source Control Management (SCM) services for Spinnaker.

Common Polling Architecture

Igor runs a number of pollers that all share the same common architecture. At a high level, they all:

• periodically get a list of items from an external resource (e.g. builds on a Jenkins master)
• compare that list against their own persisted cache of items (the difference is called delta size)
• send an echo event for each new item
• cache the new list of items

Features:

• health: igor has a HealthIndicator that reports Down if no pollers are running or if they have not had a successful polling cycle in a long time
• locking: pollers can optionally acquire a distributed lock in the storage system before attempting to complete a polling cycle. This makes it possible to run igor in a high-availability configuration and scale it horizontally.
• safeguards: abnormally large delta sizes can indicate a problem (e.g. lost or corrupt cache data) and cause downstream issues. If a polling cycle results in a delta size above the threshold, the new items will not be cached and events will not be submitted to echo to prevent a trigger storm. Manual action will be needed to resolve this, such as using the fast-forward admin endpoint: /admin/pollers/fastforward/{monitorName}[?partition={partition}]. Fast-forwarding means that all pending cache state will be polled and saved, but will not send echo notifications.

Relevant properties:

Property	Default value	Description
spinnaker.build.pollingEnabled	true	Defines whether or not the build system polling mechanism is enabled. Disabling this will effectively disable any integration with a build system that depends on Igor polling it.
spinnaker.build.pollInterval	60	Interval in seconds between polling cycles
spinnaker.pollingSafeguard.itemUpperThreshold	1000	Defines the upper threshold for number of new items before a cache update cycle will be rejected
locking.enabled	false	Enables distributed locking so that igor can run on multiple nodes without interference

Relevant metrics:

Metric	Type	Description
pollingMonitor.newItems	gauge	represents the number of new items cached by a given monitor during a polling cycle
pollingMonitor.itemsOverThreshold	gauge	0 if deltaSize < threshold, deltaSize otherwise
pollingMonitor.pollTiming	timer	published for every polling cycle with the duration it took to complete
pollingMonitor.failed	counter	an error counter indicating a failed polling cycle

All these metrics can be grouped by a monitor tag (e.g. DockerMonitor, JenkinsMonitor...) to track down issues.

Storage

The following storage backends are supported:

• Redis

Relevant properties:

redis:
  enabled: true
  connection: redis://host:port

Integration with SCM services

The following SCM services are supported:

• Bitbucket
• Github
• Gitlab
• Stash

Commit controller classes expose APIs to retrieve lists of commits, such as /github/{{projectKey}}/{{repositorySlug}}/compareCommits?from={{fromHash}}&to={{toHash}}

At the moment, igor only exposes read APIs, there are no pollers and no triggers involving SCM services directly.

Relevant properties:

github:
  baseUrl: "https://api.github.com"
  accessToken: '<your github token>'
  commitDisplayLength: 8

stash:
  baseUrl: "<stash url>"
  username: '<stash username>'
  password: '<stash password>'

bitbucket:
  baseUrl: "https://api.bitbucket.org"
  username: '<bitbucket username>'
  password: '<bitbucket password>'
  commitDisplayLength: 7

gitlab:
  baseUrl: "https://gitlab.com"
  privateToken: '<your gitlab token>'
  commitDisplayLength: 8

Integration with CI services

The following CI services are supported:

• Artifactory
• Nexus
• Concourse
• Gitlab CI
• Google Cloud Build (GCB)
• Jenkins
• Travis
• Wercker

For each of these services, a poller can be enabled (e.g. with jenkins.enabled) that will start monitoring new builds/pipelines/artifacts, caching them and submitting events to echo, thus supporting pipeline triggers. GCB is a bit different in that it doesn't poll and requires setting up pubsub subscriptions.

The BuildController class also exposes APIs for services that support them such as:

• getting build status
• listing builds/jobs on a master
• listing queued builds
• starting and stopping builds/jobs

These APIs are used to provide artifact information for bake stages.

Configuring Jenkins Masters

In your configuration block (either in igor.yml, igor-local.yml, spinnaker.yml or spinnaker-local.yml), you can define multiple masters blocks by using the list format.

You can obtain a Jenkins API token by navigating to http://your.jenkins.server/me/configure (where me is your username).

jenkins:
  enabled: true
  masters:
    -
      address: "https://spinnaker.cloudbees.com/"
      name: cloudbees
      password: f5e182594586b86687319aa5780ebcc5
      username: spinnakeruser
    -
      address: "http://hostedjenkins.amazon.com"
      name: bluespar
      password: de4f277c81fb2b7033065509ddf31cd3
      username: spindoctor

Configuring Travis Masters

In your configuration block (either in igor.yml, igor-local.yml, spinnaker.yml or spinnaker-local.yml), you can define multiple masters blocks by using the list format.

To authenticate with Travis you use a "Personal access token" on a git user with permissions read:org, repo, user. This is added in settings -> Personal access tokens on github/github-enterprise.

travis:
  enabled: true
  # Travis names are prefixed with travis- inside igor.
  masters:
  - name: ci # This will show as travis-ci inside spinnaker.
    baseUrl: https://travis-ci.com
    address: https://api.travis-ci.com
    githubToken: 6a7729bdba8c4f9abc58b175213d83f072d1d832
  regexes:
  - /Upload https?:\/\/.+\/(.+\.(deb|rpm))/

When parsing artifact information from Travis builds, igor uses a default regex that will match on output from the jfrog rt/art CLI tool. Different regexes than the default may be configured using the regexes list.

Configuring Gitlab CI Masters

In your configuration block (either in igor.yml, igor-local.yml, spinnaker.yml or spinnaker-local.yml), you can define multiple masters blocks by using the list format.

To authenticate with Gitlab CI use a Personal Access Token with permissions read_api.

gitlab-ci:
  enabled: true
  itemUpperThreshold: 1000 # Optional, default 1000.  Determines max new pipeline count before a cache cycle is rejected
  masters:
    - address: "https://git.mycompany.com"
      name: mygitlab
      privateToken: kjsdf023ofku209823
      # Optional:
      defaultHttpPageLength: 100 # defaults 100, page length when querying paginated Gitlab API endpoints (100 is max per Gitlab docs)
      limitByOwnership: false # defaults false, limits API results to projects/groups owned by the token creator
      limitByMembership: true # defaults true, limits API results to projects/groups the token creator is a member in
      httpRetryMaxAttempts: 5 # defaults 5, # default max number of retries when hitting Gitlab APIs and errors occur
      httpRetryWaitSeconds: 2 # defaults 2, # of seconds to wait between retries
      httpRetryExponentialBackoff: false # deafults false, if true retries to Gitlab will increase exponentially using the httpRetryWaitSeconds option's value

Build properties are automatically read from successful Gitlab CI Pipelines using the pattern SPINNAKER_PROPERTY_*=value. For example a log containing a line SPINNAKER_PROPERTY_HELLO=world will create a build property item hello=world. Gitlab CI artifacts are not yet supported.

Integration with Docker Registry

Clouddriver can be configured to poll your registries. When that is the case, igor can then create a poller that will list the registries indexed by clouddriver, check each one for new images and submit events to echo (hence allowing Docker triggers)

Relevant properties:

• dockerRegistry.enabled
• requires services.clouddriver.baseUrl to be configured

Running igor

Igor requires redis server to be up and running.

Start igor via ./gradlew bootRun. Or by following the instructions using the Spinnaker installation scripts.

Debugging

To start the JVM in debug mode, set the Java system property DEBUG=true:

./gradlew -DDEBUG=true

The JVM will then listen for a debugger to be attached on port 8188. The JVM will not wait for the debugger to be attached before starting igor; the relevant JVM arguments can be seen and modified as needed in build.gradle.