Skip to content

Commit

Permalink
Add README to models and migrations folders (#605)
Browse files Browse the repository at this point in the history
Having just completed a refactoring of our models it is no longer as clear as it used to be where they are sourced from. We're also now starting to mix up models based on tables (ours) with those based on views (legacy).

So, we thought it a prudent idea to add a bit of documentation to provide a bit of context when it comes to the models and the migrations.

This change adds a README for each folder that will hopefully be useful for a new starter (and a reminder for those who have been around too long!)
  • Loading branch information
Cruikshanks authored and Demwunz committed Dec 20, 2023
1 parent 281384f commit 80f2182
Show file tree
Hide file tree
Showing 2 changed files with 98 additions and 0 deletions.
21 changes: 21 additions & 0 deletions app/models/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Models

Our models need a little explaining because unlike other services there is some context new contributors need to be aware of.

## The context

This repo (at time of writing) is overtime replacing features in the legacy code. The legacy service built by the previous team is a series of apps each talking to their own schema. It uses a microservice architecture. If one app needs returns data, for example, it needs to request it from the returns app.

Our new 'system' app ignores the microservice architecture. It queries the database for all the data it needs. But should it wish, for example, to join licence data with returns data it can't because they are in different schemas.

There are also issues in the existing tables. Some common fields are not consistently named across tables. They also didn't follow standard naming conventions, for example, primary keys being named `id`. This makes working with queried data 'messy'.

## The solution

By connecting to the legacy tables via [Updateable views](https://www.postgresql.org/docs/current/sql-createview.html) we have solved these problems.

For each legacy table we create a view in the `public` schema. In the view we can correct field name inconsistencies and patterns we don't wish to use. With them being in one schema we can also join tables that underneath are held in different schemas.

## Views and tables

Where new tables are needed we have created them in the `public` schema as well. So, be aware that models can be based on views or tables.
77 changes: 77 additions & 0 deletions db/migrations/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Migrations

The database migrations are run whenever **water-abstraction-system** is deployed to an AWS environment. They are built using [Knex](https://knexjs.org/guide/migrations.html).

## Legacy VS public

Any migrations in [/db/migrations/legacy](/db/migrations/legacy) are used to support unit testing. These recreate legacy tables in their various legacy schemas.

Migrations in [/db/migrations/public](/db/migrations/public) are run both in `production` and `test` environments against the **public** schema. They create and alter the views and tables we rely on.

## Creating a migration

> We assume you are running the [wal-dev-environment](https://gitlab-dev.aws-int.defra.cloud/water-abstraction/wal-dev-environment)
We use npm scripts in `package.json` to trigger **Knex** to create a 'blank' migration file. We then update the file to implement whatever the change is; create a new table or view, or alter an existing one.

Use the **💻 Open** helper task and then navigate to `/home/repos/water-abstraction-system`. If creating a _legacy migration_ run

```bash
npm run migrate:make:test [migration-file-name]
```

Else run

```bash
npm run migrate:make [migration-file-name]
```

Replace `[migration-file-name]` with a filename that matches the migration you are creating. For example

- `create-return-logs-view`
- `create-review-results`
- `alter-bill-runs-view`
- `alter-review-return-results`

## Applying a migration

After creating the migration and updating it to make the changes you need you can apply it to the environment. If the migration is a `/public` one you'll need to apply it to both the main DB and the test one.

```bash
npm run migrate
npm run migrate:test
```

If the migration is just to recreate a legacy table you only need to apply it to the test DB.

```bash
npm run migrate:test
```

## Rolling back a migration

The most common scenario is having created a new migration and applied it to your local DB, you realise a change is needed or a mistake needs correcting. You can rollback the last migrations applied using

```bash
npm run rollback
```

If the migrations were applied against the test DB you can use

```bash
npm run rollback:test
```

## Rolling back all migrations

Should you ever need to completely reset your migrations you can do so. Use the following to reset the main DB (`public schema` only)

```bash
npm run rollback:all
```

The same goes for the test DB.

```bash
npm run rollback:all:test
```

0 comments on commit 80f2182

Please sign in to comment.