Split OptionalServiceCode table into 2 tables to save space #99
Milestone
Comments
|
Did a little more digging here:
Splitting optional services gets us down to ~2.4GB + index, and shirking some additional columns that don't need to be text gets us to 1.98GB + index. For the filing and filing component tables, shrinking the other columns reduces disk usage by 181MB and 116MB respectively. Will need to take some dumps and example tables, this is a much bigger migration than previous DB schema upgrades. It shouldn't be critical right now, but it will be very soon. |
Merged
BryceStevenWilley
added a commit
that referenced
this issue
Jun 14, 2023
Shrinks the code tables, particularly the optional services one, which required some refactoring to not assume that most operations take only a single transaction. It shrinks the disk usage by a few GB, can't find my notes that I have for how much. It wasn't a ton though, and would need to go to enums if we wanted to actually decrease usage. For now though, addresses #99. * Update 6to7, smaller database than otherwise More progress; still fails test from nothing Had to pivot again, and start putting a lot of specific DELETE, INSERT, and CREATE operations into the optional service class, since that table is now two tables, which means two statements, which breaks all of the abstractions we had that adding to a single codes table is adding to a single SQL table. Just special cased a lot of stuff for Optionalservices. Passed all tests, still need to local test * Removed optionalservices from table list, not used anymore * DataFieldRow neews booleans now, not strings * Fix tests for optsrv, add test for deleting * Return after deleting from optionalservices Instead of continuing to non-optional service stuff.
|
This was helpful, but not as much as it should have been. Still, the issue is done for now. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The optional service codes are by far the largest tables, and now that we are running 3 (soon to be 6) different domains (jurisidiction and stage / test), it's starting to take up a lot of space. The issue is that for each individual optional service, there are 1000's of filing codes, and each service / filing code combination takes up another row and repeats the whole name / all the other data points. We need to split up the table into two to reduce the extra disk usage (we're at 7.3GB for 2.5 domains), and should also improve performance too.
The key will be to figure out a way to decouple the exact codes zip we get from Tyler into the two different tables, without changing how we handle all the tables too much.
The text was updated successfully, but these errors were encountered: