Code for running GitLab CI/CD jobs on cloud.gov or another CloudFoundry based PaaS.
- Differences from other GitLab Runner executor types
- How it works
- Deploying
- Troubleshooting
- Design Decisions
The goal of this runner executor is to support most CI/CD use cases without
needing to modify your .gitlab-ci.yml
. Please note the current limitiations
and differences in comparison to the Docker executor:
- No shared filesystem or direct exec capability - Some executors can share filesystems between containers and directly execute processes in containers. CloudFoundry does not support sharing filesystems and only supports use of SSH to execute into a running container. This runner attempts to transparently work around these limitations where possible. Your CI job may require significant modification if it relies on either of these features.
- Use CI_SERVICE_alias for service DNS names - Ephemeral networks like the Docker networks
used for the Docker executor are not available in CloudFoundry. This means
that each service you create lives in a common internal DNS namespace with
other services in the same CloudFoundry space. The cloud.gov runner populates
a
CI_SERCVCE_<alias>
variable for each service defined in a job. Here is an example snippet of GitLab CI Yaml showing the definition of a service and how the job steps can then connect to that service:# Start a HTTP "echo" service and then send a request to it echo-test-job: stage: test image: ubuntu:24.04 services: - name: http-https-echo:latest # Note "echo" is the alias name, so the CI_SERVICE_alias variable key name # for this service is CI_SERVICE_echo alias: echo script: # Using the CI_SERVICE_alias to provide the FQDN of the service - curl http://${CI_SERVICE_echo}:8080
This is a custom executor borrowing ideas from https://docs.gitlab.com/runner/executors/custom.html.
It runs a "manager" GitLab Runner instance responsible for listening for new jobs. Each job is run on a new application instance in CloudFoundry using the specified OCI image.
The relevant components of the solution are shown above with two running stages.
- "GitLab Runner - Manager" - This is the persistent application defined in this very repository. It registers with the GitLab instance and waits for further instructions.
- "GitLab Runner - Worker" - The manager starts worker application instances
using the container image you specify and runs job steps via CloudFoundry
cf ssh
calls into the running containers. - "GitLab Runner - Service" - Optional service instances can be started for use by the Worker instances. These run as separate apps using the container image you specify.
The above sequence diagram illustrates the series of events in a given job run.
Optional service lifecycles are not shown, but run as part of the prepare
phase.
The runner manager registration and other flow details are shown in Runner Execution Flow.
-
GitLab access and ownership of one or more projects/repos - The runners need a GitLab service to get their instructions from. This is a good time to familiarize yourself with registering runners and generating runner tokens.
-
A cloud.gov organization - You need a cloud.gov organization to deploy your runner(s) into. See Get access to cloud.gov for details.
-
A cloud.gov space for use by runners - Runners should only be deployed into their own dedicated space(s). Runners can execute arbitrary non-production code pushed to GitLab by anyone with write access to your code repositories.
-
Network access from your runner space to required Internet destinations - Runners need to reach your GitLab server via HTTPS. They also need to be able to reach package repositories and other resources used by your CI/CD jobs. The specific destinations vary.
- For a production deployment we recommend tightly controlled name based egress filtering using a proxy such as the cg-egress-proxy.
- For testing/non-production deployments you may consider opening up full
Internet access from the runners.
(
⚠️ WARNING⚠️ - Open egress Internet access may result in data exfiltration, participation in a botnet, or worse if you allow anonymous or untrusted contributors to submit code that is run by CI/CD.) To allow open internet access bind thepublic_networks_egress
security group to your runner space:cf bind-security-group public_networks_egress ORG_NAME --space RUNNER_SPACE_NAME
-
Log in to cloud.gov
cf login -a api.fr.cloud.gov --sso
-
Target the org and space for deployment
cf target -o ORG_NAME -s RUNNER_SPACE_NAME
For example:
cf target -o sandbox-gsa -s yer.name
-
Create a cloud.gov service account, tagged with
gitlab-service-account
cf create-service cloud-gov-service-account space-deployer SERVICE_ACCOUNT_INSTANCE -t "gitlab-service-account"
-
Create a cloud.gov brokered S3 bucket -
basic-sandbox
is suggested. Note thatOBJECT_STORE_INSTANCE
only needs to be unique within the specific space as it will have a prefix prepended to create the S3 bucket name.cf create-service s3 basic-sandbox OBJECT_STORE_INSTANCE
-
Copy
vars.yml-template
tovars.yml
cp vars.yml-template vars.yml
-
Edit
vars.yml
and modify the values there as needed. In particular, you must- supply the
ci_server_token
provided when you configure the runner at the target GitLab URL - supply the
service_account_instance
name that you used when you created the service instance in step 3 - supply the
object_store_instance
name that you used when you created the brokered S3 bucket in step 4
- supply the
-
Deploy the GitLab runner
cf push --vars-file vars.yml
-
Check to see that the runner has registered itself in GitLab under your project repository under Settings -> CI/CD -> Runners (Expand)
At this point the runner should be available to run jobs. See Use GitLab - Use CI/CD to build your application - Getting started for much more on GitLab CI/CD and runners.
Problems with runner registration often requiring viewing it's logs.
cf logs --recent RUNNER-NAME
"Request error: Get https://API-URL/v2/info: dial tcp X.X.X.X:443: connect: connection refused"
The GitLab Runner manager needs to contact the CloudFoundry API to schedule runner applications. This error indicates your CloudFoundry space security group may be too restrictive or not set. See prerequisites.
- Make it possible to run DAST/a11y/other tests requiring a running copy of the application and a separate scanning process.
- Set custom executor timeouts based on measurements.
- Add linting and tests.
- Improve documentation.
- Add support for RedHat based images in
prepare.sh
. - Add version pinning and support for other architectures to
gitlab-runner-helper
installation inprepare.sh
.
Recent versions of gitlab-runner
expose almost all initial configuration
variables for the register
subcommand as environment variables. This allows
us to do almost all configuration in manifest.yml
and skip modifying
command line options in runner/.profile
or having a .toml add on.
Security controls and other compliance documents for the DevTools SaaS system live at https://github.com/GSA-TTS/devtools-compliance and also as a git submodule under /doc/compliance
From this repo, run bin/trestle
to start a trestle CLI with helper scripts for maintaining and updating the documents.
See the docker-trestle README for help with the individual scripts.
See git's submodule documentation for more information on tracking changes to the compliance files.
git clone --recurse-submodules <<REPO_ADDRESS>>
git pull --recurse-submodules
git push --recurse-submodules=check
then git push --recurse-submodules=on-demand
git config diff.submodule log
git config status.submodulesummary 1
git config push.recurseSubmodules check
git config submodule.recurse true