Torchbox.com on Wagtail

This is the main Torchbox.com website.

Technical documentation

Please do have a read through of the documentation before starting any work on the project, as there is lots of detail in there.

This project contains technical documentation written in Markdown in the /docs folder. This covers, among others:

• our general apporach and principles for the build
• lots of front-end details specific to this build
• continuous integration
• deployment
• git branching
• project conventions

You can view it using mkdocs by running:

mkdocs serve

The documentation will be available at: http://localhost:8001/

It is also available via github pages at https://torchbox.github.io/torchbox.com/

Setting up a local build

This repository includes docker-compose configuration for running the project in local Docker containers, and a fabfile for provisioning and managing this.

There are a number of other commands to help with development using the fabric script. To see them all, run:

fab -l

Dependencies

The following are required to run the local environment. The minimum versions specified are confirmed to be working: if you have older versions already installed they may work, but are not guaranteed to do so.

• Docker, version 19.0.0 or up
  • Docker Desktop for Mac installer OR Orbstack
  • Docker Engine for Linux installers
• Docker Compose, version 1.24.0 or up
  • Install instructions (Linux-only: Compose is already installed for Mac users as part of Docker Desktop or Orbstack.)
• Fabric, version 2.4.0 or up
  • Install instructions
• Python, version 3.6.9 or up

Note that on Mac OS, if you have an older version of fabric installed, you may need to uninstall the old one and then install the new version with pip3:

pip uninstall fabric
pip3 install fabric

You can manage different python versions by setting up pyenv: https://realpython.com/intro-to-pyenv/

Additionally, for interacting with production / staging environments, you'll need:

• Heroku CLI
• AWS CLI

Running the local build for the first time

If you are using Docker Desktop, ensure the Resources:File Sharing settings allow the cloned directory to be mounted in the web container (avoiding mounting OCI runtime failures at the end of the build step).

Starting a local build can be done by running:

git clone git@github.com:torchbox/torchbox.com.git
cd torchbox.com
fab build
fab migrate
fab start

This will start the containers in the background, but not Django. To do this, connect to the web container with fab sh and run honcho start to start both Django and the Webpack dev server in the foreground.

Then, connect to the running container again (fab sh) and:

dj createcachetable
dj createsuperuser

The site should be available on the host machine at: http://127.0.0.1:8000/

If you only wish to run the frontend or backend tooling, the commands honcho runs are in docker/Procfile.

Upon first starting the container, the static files may not exist, or may be out of date. To resolve this, simply run npm run build.

Frontend tooling

Here are the common commands:

# Install front-end dependencies.
npm install
# Start the Webpack build in watch mode, without live-reload.
npm run start
# Start the Webpack server build on port 3000 only with live-reload.
npm run build
# Do a one-off Webpack production build.
npm run build:prod

There are two ways to run the frontend tooling:

• In Docker. This is the default, most portable and secure, but much slower on macOS.
• Or run npm commands from a terminal on your local machine. Create a .env file in the project root (see .env.example) with FRONTEND=local. fab start will no longer start a frontend container. Now, when running fab start, Docker won't attempt to bind to the ports needed for the frontend dev server, meaning they can be run locally. All the tooling still remains available in the container.

Installing python packages

Python packages can be installed using poetry in the web container:

fab sh
poetry add wagtail-guide

To reset installed dependencies back to how they are in the poetry.lock file:

fab sh
poetry install --no-root

Deployments

To deploy, merge your feature branch to main or staging branch. Once CI pipelines have passed, it will be deployed to the respective Heroku site automatically.

This is done via Heroku Github integration.

Database snapshots during development

Database snapshots are a valuable asset in development, especially when dealing with Django migrations.

The following set of fabric commands are available for local execution:

Create a database snapshot

fab dellar-snapshot <filename>

Restore a database snapshot

fab dellar-restore <filename>

List available database snapshots

fab dellar-list

Remove a database snapshot

fab dellar-remove <filename>

These snapshots are created and restored behind the scenes using pg_dump and psql within the db container, as detailed in the official Postgres documentation.

Running fab dellar-snapshot <filename> creates a <filename>.psql file in the root directory / of the db container:

❯ fab dellar-snapshot test01
Database snapshot created
❯ fab dellar-snapshot test02
Database snapshot created
❯ fab dellar-list
Database snapshots:
 - test01
 - test02
Restore with `dellar-restore <snapshot>`
❯ docker-compose exec db bash
root@ad84f5154816:/# ls -lah | grep sql
-rw-r--r--   1 root root  545 Jan 10 12:03 test01.psql
-rw-r--r--   1 root root  545 Jan 10 12:03 test02.psql
root@ad84f5154816:/#

What's with the dellar name?

These commands draw inspiration from [Stellar](https://pypi.org/project/stellar/) — a database snapshot and restore tool that is no longer maintained.