patchman-engine

System Patch Manager is one of the applications for console.redhat.com. This application allows users to display and manage available patches for their registered systems. This code repo stores sources for the backend part of the application which provides the REST API to the frontend.

Table of content

• Architecture
• Database
• Development environment
  • Local running
  • Local app requests
  • Tests running
  • OpenAPI docs
• Control by private API
• VMaaS
• Monitoring
• [Profiling] (#profiling)

Development environment

Running locally

Uses podman-compose to deploy the individual project components and supporting containers, which simulate the CMSfR platform and database:

podman-compose up --build # Build images if needed and start containers
podman-compose down       # Stop and remove containers

Use --profile monitoring to run local prometheus and grafana, for example

podman-compose --profile monitoring up

Grafana is accessible at http://localhost:3000, Prometheus at http://localhost:9090

Local app requests

When podman compose is running, you can test the app using dev shell scripts:

cd dev/scripts
./systems_list.sh         # show systems
./advisories_list.sh      # show advisories
./platform_sync.sh        # trigger vmaas_sync to sync (using vmaas mock)
./platform_upload.sh      # simulate archive upload to trigger listener and evaluator_upload

Running in host OS

Run single component in host OS, rest in podman-compose:

podman-compose stop evaluator_upload # stop single component running using podman-compose
export $(xargs < conf/local.env)
./scripts/entrypoint.sh evaluator # (or listener, or manager) run component in host OS

Running tests

We cover a large part of the application functionality with tests; this requires also running a test database and mocked services. This is all encapsulated into the configuration runable using podman-compose command. It also includes static code analysis, database migration tests and dockerfiles checking. It's also used when checking pull requests for the repo.

podman-compose -f docker-compose.test.yml up --build --abort-on-container-exit

Run single test

After running all test suit, testing platform components are still running (kafka, platform, db). This is especially useful when fixing some test or adding a new one. You need to have golang installed.

podman-compose -f docker-compose.test.yml up --build --no-start # build images
podman-compose -f docker-compose.test.yml start db platform kafka # start containers
. scripts/export_local_env.sh # setup needed env variables for tests
go test -count=1 -v ./evaluator -run TestEvaluate # run "TestEvaluate" test from "evaluator" component

Run tests in VS Code

Prerequisite is to have Go Extension installed.

To set it up copy the example settings from .vscode/settings.example.json:

cp .vscode/settings.example.json .vscode/settings.json

Access to dev/test database

When a podman compose (either dev or test one) is running, then the database can be access directly by executing

podman exec -it db psql -d patchman -U admin

or locally using psql with:

export $(cat conf/local.env conf/database_admin.env | xargs ) 2>/dev/null; ./dev/scripts/psql.sh

OpenAPI docs

Our REST API is documented using OpenAPI v3. On a local instance it can be accessed on http://localhost:8080/openapi/index.html.

To update/regenerate OpenAPI sources run:

go get -u github.com/swaggo/swag/cmd/swag # download binary to generate, do it first time only
./scripts/generate_docs.sh

Control by private API

There is a private API accessible only from inside of vmaas_sync container. It allows running component routines manually. In local environment it can be tested like this:

podman exec -it patchman-engine_vmaas_sync_1 ./sync.sh    # trigger advisories syncing event.
podman exec -it patchman-engine_vmaas_sync_1 ./re-calc.sh # trigger systems recalculation event.
podman exec -it patchman-engine_vmaas_sync_1 ./caches-check.sh # trigger account caches checking.

VMaaS

This project uses VMaaS for retrieving information about advisories, and resolving which advisories can be applied to which systems. For local development this repo contains VMaaS service mock as a part of platform mock allowing independent running of the service using podman-compose.

Monitoring

Each application component (except for the database) exposes metrics for Prometheus on /metrics endpoint (see docker-compose.yml for ports). Runtime logs can be sent to Amazon CloudWatch if configuration environment variables are set (see awscloudwatch.go).

Kafka control

Your can control and inspect def Kafka instance using:

docker-compose exec kafka bash # enter kafka component and run inside:
/usr/bin/kafka-topics --list --bootstrap-server=kafka:9092 # show created topics

# list all messages send to a topic
/usr/bin/kafka-console-consumer --bootstrap-server=kafka:9092 --topic platform.inventory.events --from-beginning

# send debugging message to a topic
echo '{"id":"00000000-0000-0000-0000-000000000002"}' | /usr/bin/kafka-console-producer --broker-list kafka:9092 --topic patchman.evaluator.upload

Run SonarQube code analysis

export SONAR_HOST_URL=https://sonar-server
export SONAR_LOGIN=paste-your-generated-token
export SONAR_CERT_URL=https://secret-url-to/ca.crt # optional
podman-compose -f dev/sonar/docker-compose.yml up --build

Update Grafana config map

Copy Grafana board json config to the temporary file, e.g. grafana.json and run:

./scripts/grafana-json-to-yaml.sh grafana.json > ./dashboards/app-sre/grafana-dashboard-insights-patchman-engine-general.configmap.yaml

Profiling

App can be profiled using /net/http/pprof. Profiler is exposed on app's private port.

Local development

• set ENABLE_PROFILE=true in the cont/common.env
• docker-compose up --build
• go tool pprof http://localhost:{port}/debug/pprof/{heap|profile|block|mutex} available ports:
• 9000 - manager
• 9002 - listener
• 9003 - evaluator-upload
• 9004 - evaluator-recalc

Admin API

• set ENABLE_PROFILE_{container_name}=true in the ClowdApp
• download the profile file using internal api /api/patch/admin/pprof/{manager|listener|evaluator_upload|evaluator_recalc}/{heap|profile|block|mutex|trace}
• go tool pprof <saved.file>

Deps backup

patchman-engine-deps