Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add synthetic UNIQUE constraint to retention policy table #28

Merged
merged 1 commit into from
Jul 10, 2023

Conversation

amotl
Copy link
Member

@amotl amotl commented Jul 8, 2023

@amotl amotl requested a review from hammerhead July 8, 2023 09:48
@amotl amotl marked this pull request as ready for review July 8, 2023 09:54
@amotl amotl linked an issue Jul 9, 2023 that may be closed by this pull request
selectable = sa.select(table).where(
table.c.table_schema == policy.table_schema,
table.c.table_name == policy.table_name,
)
Copy link
Member

@hammerhead hammerhead Jul 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can make sense to have multiple policies for one table. Especially to combine the reallocate strategy with others.
Example: For table x, I want to reallocate partitions after 30 days from a hot to a warm node, and after 60 days I want to permanently delete it (or snapshot with deletion).

But indeed it doesn't make sense to have a combination of the snapshot + delete strategies on the same table, as both will erase the data.

A reallocate strategy with a shorter retention period than a delete/snapshot strategy makes sense, though.

Copy link
Member Author

@amotl amotl Jul 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Niklas.

I was having roughly about the same thoughts, so I am happy you are referring to a specific example.

Maybe we can track the new insights at GH-26, in order to come up with a better plan how to handle the constraints more specifically? It looks like we need to come up with specific rules here instead of implementing uniqueness constraints in a generic manner.

In this way, we can proceed with integrating all the stacked PRs, and iterate on this topic later once more.

WIth kind regards,
Andreas.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, let's save it for another iteration 👍

Base automatically changed from amo/naming-things/hot-warm to main July 10, 2023 06:12
@amotl amotl requested a review from hammerhead July 10, 2023 06:24
@amotl amotl merged commit 3a4df6b into main Jul 10, 2023
3 checks passed
@amotl amotl deleted the amo/datamodel-uniqueness branch July 10, 2023 07:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants