-
Notifications
You must be signed in to change notification settings - Fork 53
Database anonymization
alismx edited this page Dec 23, 2022
·
2 revisions
- Check for sensitive fields using the detect script from PostgreSQL Anonymizer (Our script is working, but PostgreSQL Anonymizer is not, I've reported a bug (here)[https://gitlab.com/dalibo/postgresql_anonymizer/-/issues/300])
- Generate fake data (if needed)
- Create a db_dump from your source database using the steps described below
- Restore the anonymized database to a new database
- Sync/Create users in Okta and Non-production testing environment
To ensure access to any database created from an anonymized dump, please make sure you have an account in the source database. In the future, we can use the Okta API to grant proper permissions based on need. https://github.com/CDCgov/prime-simplereport/issues/3962
Create an anonymized local database
- start your database
- restore the snapshot
- Docker DB:
yarn anon:dump
- Local DB:
yarn anon:dump:localdb
- Docker DB:
Restore an anonymized local PostgreSQL dump
- start your database
- restore the snapshot
yarn anon:restore
- Restart your apps
Automated report of potentially sensitive columns (only to be used as an aid and not a source of truth)
- start your database
- restore the snapshot
yarn anon:detect
Completely removes anon from a database
- start your database
- restore the snapshot
yarn anon:remove
At this point, it's a pile of scripts and some local docker changes, nothing that gets used in our automated processes or remote environments yet.
- Getting Started
- [Setup] Docker and docker compose development
- [Setup] IntelliJ run configurations
- [Setup] Running DB outside of Docker (optional)
- [Setup] Running nginx locally (optional)
- [Setup] Running outside of docker
- Accessing and testing weird parts of the app on local dev
- Accessing patient experience in local dev
- API Testing with Insomnia
- Cypress
- How to run e2e locally for development
- E2E tests
- Database maintenance
- MailHog
- Running tests
- SendGrid
- Setting up okta
- Sonar
- Storybook and Chromatic
- Twilio
- User roles
- Wiremock
- CSV Uploader
- Log local DB queries
- Code review and PR conventions
- SimpleReport Style Guide
- How to Review and Test Pull Requests for Dependabot
- How to Review and Test Pull Requests with Terraform Changes
- SimpleReport Deployment Process
- Adding a Developer
- Removing a developer
- Non-deterministic test tracker
- Alert Response - When You Know What is Wrong
- What to Do When You Have No Idea What is Wrong
- Main Branch Status
- Maintenance Mode
- Swapping Slots
- Monitoring
- Container Debugging
- Debugging the ReportStream Uploader
- Renew Azure Service Principal Credentials
- Releasing Changelog Locks
- Muting Alerts
- Architectural Decision Records
- Backend Stack Overview
- Frontend Overview
- Cloud Architecture
- Cloud Environments
- Database ERD
- External IDs
- GraphQL Flow
- Hibernate Lazy fetching and nested models
- Identity Verification (Experian)
- Spring Profile Management
- SR Result bulk uploader device validation logic
- Test Metadata and how we store it
- TestOrder vs TestEvent
- ReportStream Integration
- Feature Flag Setup
- FHIR Resources
- FHIR Conversions
- Okta E2E Integration
- Deploy Application Action
- Slack notifications for support escalations
- Creating a New Environment Within a Resource Group
- How to Add and Use Environment Variables in Azure
- Web Application Firewall (WAF) Troubleshooting and Maintenance
- How to Review and Test Pull Requests with Terraform Changes