Backup Database and put in s3

[asteel-gsa]

Deployment Note:
A tag is created ->
deploy-production.yml is run with the on: function, looking for a tag v1.* ->
inside deploy-application.yml which is called from deploy-production.yml the stage to backup the database runs only on a tag matching v1.. It is not wildcarded, hence the startsWith(github.ref, 'refs/tags/v1.'). I tested this, in a different way by doing the following:

• A workflow that is run on tag v1.*

  • a stage in that workflow that has if: startsWith(github.ref, 'refs/tags/v2.') and the job was skipped
  • change that stage to if: startsWith(github.ref, 'refs/tags/v1.') and the step was run

• List any special steps reviewers have to follow to test the PR. For example, adding a local environment variable, creating a local test file, etc.

# Backup
> cf login -a api.fr.cloud.gov --sso
> cf t -s preview
> cf ssh gsa-fac
> /tmp/lifecycle/shell
> source .profile
> set +e
> export PATH=/home/vcap/deps/0/apt/usr/lib/postgresql/15/bin:$PATH
> python manage.py dbbackup

# Restore
> cf login -a api.fr.cloud.gov --sso
> cf t -s preview
> cf ssh gsa-fac
> /tmp/lifecycle/shell
> source .profile
> set +e
> export PATH=/home/vcap/deps/0/apt/usr/lib/postgresql/15/bin:$PATH
> python manage.py dbrestore --noinput
# > python manage.py dbrestore -i <backup_file_name> --noinput

That will net the following result:

vcap@de52d882-ffca-418e-7975-f5e9:~$ export PATH=/home/vcap/deps/0/apt/usr/lib/postgresql/15/bin:$PATH
vcap@de52d882-ffca-418e-7975-f5e9:~$ python manage.py dbbackup
{"message": "Backing Up Database: cgawsbrokerprodn0xoxhttnue8x2n"}
{"message": "Writing file to default-de52d882-ffca-418e-7975-f5e9-2023-09-13-180815.psql.bin"}

Full output, of backup and restore:

vcap@e3e5818e-1276-4f72-4558-eeb5:~$ export PATH=/home/vcap/deps/0/apt/usr/lib/postgresql/15/bin:$PATH
vcap@e3e5818e-1276-4f72-4558-eeb5:~$ set +e
vcap@e3e5818e-1276-4f72-4558-eeb5:~$ date=$(date '+%Y-%m-%d-%H%M')
vcap@e3e5818e-1276-4f72-4558-eeb5:~$ python manage.py dbbackup -o "test-db-backup-$date.dump"
{"message": "Backing Up Database: cgawsbrokerprodn0xoxhttnue8x2n"}
{"message": "Writing file to test-db-backup-2023-09-14-1328.dump"}
vcap@e3e5818e-1276-4f72-4558-eeb5:~$ python manage.py dbrestore -i test-db-backup-2023-09-14-1328.dump --noinput
{"message": "Restoring backup for database 'default' and server 'None'"}
{"message": "Restoring: test-db-backup-2023-09-14-1328.dump"}
{"message": "Restore tempfile created: 340.1 MiB"}
vcap@e3e5818e-1276-4f72-4558-eeb5:~$ 

• Make sure you’ve merged main into your branch shortly before creating the PR. (You should also be merging main into your branch regularly during development.)

PR checklist: reviewers

• SSH into preview instance and run the above commands to generate a backup of the database.
• Verify that it is present in fac-private-s3 in prod

Because this modifies settings.py, adding Tadhg and Tim as reviewers.

What this PR does:

• adds a new bash script to backup the database. We cannot directly invoke it, because of the same -i issue with load data. While we set the psql alias in .profile, the task runner needs have it set as $PATH in order to run. Since we have to do this for load_data.sh we may as well keep it standard for backup_database.sh
• Removes postgresql-client from apt.yml, our 0 buildpack, and replaces it with postgresql-client-15
• Modifies settings.py to accept django-dbbackup, and point it to the private s3
  • requires django-storages[boto3] on top of django-dbbackup

[github-actions]

Terraform plan for meta

Plan: 2 to add, 0 to change, 0 to destroy.

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # null_resource.share-backup-to-spaces will be created
  + resource "null_resource" "share-backup-to-spaces" {
      + id = (known after apply)
    }

  # module.s3-backups.cloudfoundry_service_instance.bucket will be created
  + resource "cloudfoundry_service_instance" "bucket" {
      + id                             = (known after apply)
      + name                           = "backups"
      + replace_on_params_change       = false
      + replace_on_service_plan_change = false
      + service_plan                   = "021bb2a3-7e11-4fc2-b06b-d9f5938cd806"
      + space                          = "5593dba8-7023-49a5-bdbe-e809fe23edf9"
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Warning: Argument is deprecated

  with module.s3-backups.cloudfoundry_service_instance.bucket,
  on /tmp/terraform-data-dir/modules/s3-backups/s3/main.tf line 14, in resource "cloudfoundry_service_instance" "bucket":
  14:   recursive_delete = var.recursive_delete

Since CF API v3, recursive delete is always done on the cloudcontroller side.
This will be removed in future releases

❌ Error applying plan in Deploy to Development and Management Environment #210

[github-actions]

Terraform plan for dev

No changes. Your infrastructure matches the configuration.

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration
and found no differences, so no changes are needed.

Warning: Argument is deprecated

  with module.dev.module.database.cloudfoundry_service_instance.rds,
  on /tmp/terraform-data-dir/modules/dev.database/database/main.tf line 14, in resource "cloudfoundry_service_instance" "rds":
  14:   recursive_delete = var.recursive_delete

Since CF API v3, recursive delete is always done on the cloudcontroller side.
This will be removed in future releases

(and 2 more similar warnings elsewhere)

✅ Plan applied in Deploy to Development and Management Environment #210

[github-actions]

File	Coverage		Missing
All files	89%	❌
api/serializers.py	90%	✅	125-126 131 136
api/test_views.py	96%	✅	105
api/uei.py	96%	✅	17-18 107-108
api/views.py	97%	✅	196-197 204-205 226 362-363
audit/excel.py	87%	❌	402 418 424 429 434 456-457 469 670-671 680-686 696 716
audit/forms.py	56%	❌	22-29
audit/intake_to_dissemination.py	93%	✅	70-71 200-206
audit/models.py	87%	❌	185 218 389 407-408 416 438 514-515 519 527 536 546 549-555
audit/test_commands.py	87%	❌
audit/test_mixins.py	90%	✅	112-113 117-119 184-185 189-191
audit/test_validators.py	95%	✅	434 438 606-607 846 853 860 867
audit/utils.py	76%	❌	13 21 38
audit/validators.py	96%	✅	269-270 298-299 314-315 325 516-525
audit/views.py	42%	❌	86-107 130-131 205-206 251-252 263-264 266-270 317-330 333-347 352-365 382-388 393-413 440-444 449-478 521-525 530-550 577-581 586-615 658-662 667-679 682-692 697-709 732-733 738-787 790-830 833-850
audit/cross_validation/additional_ueis.py	93%	✅	33
audit/cross_validation/check_award_ref_declaration.py	90%	✅
audit/cross_validation/check_award_reference_uniqueness.py	93%	✅
audit/cross_validation/check_certifying_contacts.py	87%	❌
audit/cross_validation/check_findings_count_consistency.py	91%	✅
audit/cross_validation/check_ref_number_in_cap.py	90%	✅
audit/cross_validation/check_ref_number_in_findings_text.py	90%	✅
audit/cross_validation/errors.py	78%	❌	30 65
audit/cross_validation/naming.py	68%	❌	178-182
audit/cross_validation/submission_progress_check.py	85%	❌	64 79-82
audit/cross_validation/tribal_data_sharing_consent.py	86%	❌	20
audit/cross_validation/validate_general_information.py	93%	✅	28-29
audit/fixtures/single_audit_checklist.py	79%	❌	156 232-241
audit/management/commands/load_fixtures.py	46%	❌	39-45
audit/viewlib/submission_progress_view.py	96%	✅	158-159
audit/viewlib/upload_report_view.py	30%	❌	32-35 44 91-115 118-186
cms/views.py	57%	❌	11-16 29-30
config/urls.py	71%	❌	87
djangooidc/backends.py	78%	❌	32 57-63
djangooidc/exceptions.py	66%	❌	19 21 23 28
djangooidc/oidc.py	16%	❌	32-35 45-51 64-70 92-149 153-199 203-226 230-275 280-281 286
djangooidc/views.py	80%	❌	22 43 114
djangooidc/tests/common.py	96%	✅
report_submission/forms.py	92%	✅	35
report_submission/views.py	76%	❌	83 215-216 218 240-241 260-261 287-394 397-407
report_submission/templatetags/get_attr.py	76%	❌	8 11-14 18
support/admin.py	49%	❌	9-10 13 34 37 74 77 82 89-95 98-100 103-104 107-108
support/cog_over.py	91%	✅	30-33 93 140
support/signals.py	66%	❌	23-24 29-30
support/test_cog_over.py	98%	✅	134-135 224
tools/update_program_data.py	89%	❌	96
users/auth.py	95%	✅	39-40
users/fixtures/user_fixtures.py	91%	✅

Minimum allowed coverage is 90%

Generated by 🐒 cobertura-action against 72ac199