code-gov-update

Docker Image

This project contains code for updating the DHS code.gov inventory published here.

How it works

The LLNL/scraper project is used to scrape a handful of GitHub organizations that belong to DHS and produce an updated JSON file per the code.gov specification. If that file differs from the previously-generated one, it is emailed to the appropriate address so that it can be used to update the content hosted here.

Running

Running with Docker

To run the cisagov/code-gov-update image via Docker:

docker run cisagov/code-gov-update:0.2.0

Running with Docker Compose

1. Create a docker-compose.yml file similar to the one below to use Docker Compose.

   ---
   version: '3.7'
   
   services:
     update:
       image: 'cisagov/code-gov-update:0.2.0'
       init: true
       environment:
         - AWS_CONFIG_FILE=path/to/aws_config
         - AWS_PROFILE=default

2. Start the container and detach:

   docker compose up --detach

Using secrets with your container

This container also supports passing sensitive values via Docker secrets. Passing sensitive values like your credentials can be more secure using secrets than using environment variables. See the secrets section below for a table of all supported secret files.

1. To use secrets, create aws_config and scraper.json files containing the values you want set:

   [default]
   credential_source = Ec2InstanceMetadata
   region = us-east-2
   role_arn = arn:aws:iam::123456789012:role/AssumeSesSendEmail-CodeGovUpdate

   Please see the documentation for creating your own scraper.json configuration file.

2. Then add the secrets to your docker-compose.yml file:

   ---
   version: '3.7'
   
   secrets:
     aws_config:
       file: ./src/secrets/aws_config
     scraper_config:
       file: ./src/secrets/scraper.json
   
   services:
     update:
       image: 'cisagov/code-gov-update:0.2.0'
       init: true
       secrets:
         - source: aws_config
           target: aws_config
         - source: scraper_config
           target: scraper_config.json
       environment:
         - AWS_CONFIG_FILE=/run/secrets/aws_config
         - AWS_PROFILE=default

Updating your container

Docker Compose

1. Pull the new image from Docker Hub:

   docker compose pull

2. Recreate the running container by following the previous instructions:

   docker compose up --detach

Docker

1. Stop the running container:

   docker stop <container_id>

2. Pull the new image:

   docker pull cisagov/code-gov-update:0.2.0

3. Recreate and run the container by following the previous instructions.

Image tags

The images of this container are tagged with semantic versions. It is recommended that most users use a version tag (e.g. :0.2.0).

Image:tag	Description
cisagov/code-gov-update:0.2.0	An exact release version.
cisagov/code-gov-update:0.2	The most recent release matching the major and minor version numbers.
cisagov/code-gov-update:0	The most recent release matching the major version number.
cisagov/code-gov-update:edge	The most recent image built from a merge into the develop branch of this repository.
cisagov/code-gov-update:nightly	A nightly build of the develop branch of this repository.
cisagov/code-gov-update:latest	The most recent release image pushed to a container registry. Pulling an image using the :latest tag should be avoided.

See the tags tab on Docker Hub for a list of all the supported tags.

Volumes

There are no volumes.

Ports

No ports are exposed by this container.

Environment variables

Required

There are no required environment variables.

Optional

There are no optional environment variables.

Secrets

Filename	Purpose
aws_config	Provides the necessary AWS authentication to send email using SES.
scraper.json	Provides the configuration to use for LLNL/scraper.

Building from source

Build the image locally using this git repository as the build context:

docker build \
  --tag cisagov/code-gov-update:0.2.0 \
  https://github.com/cisagov/code-gov-update.git#develop

Cross-platform builds

To create images that are compatible with other platforms, you can use the buildx feature of Docker:

1. Copy the project to your machine using the Code button above or the command line:

   git clone https://github.com/cisagov/code-gov-update.git
   cd code-gov-update

2. Create the Dockerfile-x file with buildx platform support:

   ./buildx-dockerfile.sh

3. Build the image using buildx:

   docker buildx build \
     --file Dockerfile-x \
     --platform linux/amd64 \
     --output type=docker \
     --tag cisagov/code-gov-update:0.2.0 .

Contributing

We welcome contributions! Please see CONTRIBUTING.md for details.

License

This project is in the worldwide public domain.

This project is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the CC0 1.0 Universal public domain dedication.

All contributions to this project will be released under the CC0 dedication. By submitting a pull request, you are agreeing to comply with this waiver of copyright interest.