Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partition tables by week and setup maintenance code #1254

Open
rorymckinley opened this issue Nov 1, 2023 · 3 comments
Open

Partition tables by week and setup maintenance code #1254

rorymckinley opened this issue Nov 1, 2023 · 3 comments
Assignees

Comments

@rorymckinley
Copy link
Collaborator

Partition the following tables by week:

  • work_orders
  • attempts
  • runs
  • log_lines
@rorymckinley rorymckinley self-assigned this Nov 1, 2023
@jyeshe
Copy link
Member

jyeshe commented Nov 30, 2023

Hey @rorymckinley talked to @stuartc this morning and he accepted the approach we've talked about, to start with log_lines and measure the performance to decide about partitioning other tables.

taylordowns2000 added a commit that referenced this issue Nov 30, 2023
@rorymckinley
Copy link
Collaborator Author

Hey @rorymckinley talked to @stuartc this morning and he accepted the approach we've talked about, to start with log_lines and measure the performance to decide about partitioning other tables.

@jyeshe Ok. I am going to look to preserve the work done on this so far, so it does not have to be repeated should we need to pick it up again (e.g. improved test coverage for the management of date-based partitions) and I will also restore the FKs removed by #1347.

@rorymckinley
Copy link
Collaborator Author

This is a 'mothballing' comment of the state of work on this issue:

#1392 is functional but needs to be tested against a DB. It contains code changes that will be useful if date-based partitioning is implemented on other tables.
#1453 requires code changes (see comments).
#1522 is WIP as it depends on #1453 being merged in (see the note about the shim migration). Tests are failing on this branch because I had to duplicate the Db changes of #1453 but I do not have the additional code form #1453. Once #1453 is merged in, the shim migration will be superfluous.

Once the above have been merged in, if the partitioning were to continue as originally envisaged, this is what remains:

  • Setup partitions on attempts and runs
  • Update the code that maintains the partitions so that any :all scopes also cover attempts and runs.

@taylordowns2000 taylordowns2000 moved this to Icebox in v2 Feb 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Icebox
Development

No branches or pull requests

2 participants