Skip to content

Dependencies

Jump to bottom
Ben Thorner edited this page May 10, 2022 · 12 revisions

👉 Decision document about how we manage dependencies

Python packages

Each Python app should have:

  • requirements.txt - pinned versions of all production dependencies
  • requirements.in - main / top-level dependencies, used to generate requirements.txt
  • requirements_for_test.txt - additional dependencies for dev / test

requirements.txt is generated from the requirements.in in order to pin versions of all nested dependencies. If requirements.in has been changed, run make freeze-requirements to regenerate it.

Note: for Dockerised apps, you'll need to do ./scripts/run_with_docker.sh make freeze-requirements instead.

General updates

Currently we do this as-and-when. We should prioritise dependencies with security vulnerabilities.

NPM packages

Security updates

We have a weekly job that audits Production packages for security vulnerabilities:

  • Check the output in Concourse to see which packages have vulnerabilities.
  • Make a PR to update affected packages to their patched versions.

If the packages are listed in package.json, update the versions there and then run npm install to update the lockfile. For sub-dependencies, there's unfortunately no easy way to tell NPM to "update just this package and anything that conflicts with it":

  • Try updating just the affected packages in package-lock.json and then run npm ci to see if there are any conflicts.
  • If you see conflicts with other packages, it's usually safe to just run a dumb npm install, which will update everything.

General updates

Currently we do this as-and-when. We should prioritise dependencies with security vulnerabilities.

Docker images

Security updates

We have a weekly job that audits Production Docker images for vulnerable packages:

  • Check the output in Concourse to see which packages have vulnerabilities.
  • Make a PR to pin the packages to their patched versions in the Dockerfile.

Sometimes the package won't be listed in the Dockerfile, which means it's a sub-dependency:

  • Try running the master build job in Concourse to install the latest packages.
    • This is unlikely to help for Antivirus, which gets rebuilt every day automatically.
  • If this doesn't work, you will need to investigate how the package was installed.
    • e.g. you may need to bump a pinned package to get the latest sub-dependencies.
Clone this wiki locally