Dependencies

👉 Decision document about how we manage dependencies

Python packages

Each Python app should have:

• requirements.txt - pinned versions of all production dependencies
• requirements.in - main / top-level dependencies, used to generate requirements.txt
• requirements_for_test.txt - additional dependencies for dev / test

requirements.txt is generated from the requirements.in in order to pin versions of all nested dependencies. If requirements.in has been changed, run make freeze-requirements to regenerate it.

Note: for Dockerised apps, you'll need to do ./scripts/run_with_docker.sh make freeze-requirements instead.

General updates

Currently we do this as-and-when. We should prioritise dependencies with security vulnerabilities.

NPM packages

We use specific version numbers, not semantic version ranges so, when installing new versions of packages, use the --save-exact flag.

Security updates

We have a weekly job that audits Production packages for security vulnerabilities:

• Check the output in Concourse to see which packages have vulnerabilities.
• Make a PR to update affected packages to their patched versions.

If the packages are listed in package.json under dependencies, update the versions there and then run npm install to update the lockfile. For sub-dependencies, there's unfortunately no easy way to tell NPM to "update just this package and anything that conflicts with it":

• Try updating just the affected packages in package-lock.json and then run npm ci to see if there are any conflicts.
• If you see conflicts with other packages, it's usually safe to just run a dumb npm install, which will update everything.

General updates

Currently we do this as-and-when. We should prioritise dependencies with security vulnerabilities.

Docker images

Security updates

We have a weekly job that scans Production Docker images for vulnerable packages, in this repo

These images are stored in the notify-deploy account.

The resources were created to allow scan of images on the notify-deploy account.

An event bridge is scheduled to run once per week triggering the lambda function.

The lambda function scans the ECR repository to list the repositories and it also finds the very latest image version pushed by date.

After that if starts the scan on the images and if any CRITICAL vulnerability is found, it aggregates the results into a findings.txt file that will be sent to a SNS group at the end in the form of an email with the subject: ECR SCAN-Critical Vulnerability Report