API-26689: VBADocuments Data Migration to Remove doc_type from UploadSubmission Records

[kristen-brown]

Summary

The doc_type stored in UploadSubmission records is a consumer-provided string. We previously stopped storing this key-value pair due to PII/PHI concerns, since we can't control what the consumer sends us in this field. To build upon that work, this PR scrubs the doc_type key and value from the uploaded_pdf jsonb column of all UploadSubmission records.

Related issue(s)

API-26689

Testing done

The rake task was run locally against my database containing a variety of UploadSubmission records. The contents of the rake task were also run on the development server and confirmed to work as expected to remove the doc_type from the uploaded_pdf jsonb column and leave the rest of the column's contents intact.

The task will be run in the following order on the vets-api environments:

1. Development (6,415 records) ✅
2. Staging (21,715 records)
3. Sandbox (166,619 records)
4. Production (2,577,864 records)

Screenshots

None

What areas of the site does it impact?

This PR impacts the data stored in UploadSubmission records (VBADocuments module). It also renames a couple of existing VBADocuments rake task files to move the date stamp to the front of the file name (for file sorting) and updates the namespace of the data migration tasks so that they're not under a generic temp namespace.

Acceptance criteria

• I fixed|updated|added unit tests and integration tests for each feature (if applicable). – N/A
• No error nor warning in the console.
• Events are being sent to the appropriate logging solution – N/A
• Documentation has been updated (link to documentation) – N/A
• No sensitive information (i.e. PII/credentials/internal URLs/etc.) is captured in logging, hardcoded, or specs
• Feature/bug has a monitor built into Datadog or Grafana (if applicable) – N/A
• If app impacted requires authentication, did you login to a local build and verify all authenticated routes work as expected – N/A
• I added a screenshot of the developed feature – N/A

Requested Feedback

Given the large difference in UploadSubmission record count between the lower environments and Production, I request that this PR be reviewed through the lens of performance, and I hope that the reviewer will be able to help identify if there are any data loss risks (beyond the intended data loss) to running the migration as written on the Production server. From my research, it appeared to be a performant query (and certainly more performant than updating the records via ActiveRecord), but a second set of eyes/expertise would be appreciated. Thanks!

[mathisto]

I think this is a great approach. The jsonb - operator is perfect. The only thing I am worried about is the number of records on Sandbox and Prod. 2+ million records will surely give the VM something to think about, especially since it is digging into the complex data structure for each row. What I have done in the past is to batch over the ID's in groups of a thousand or so.