-
-
Notifications
You must be signed in to change notification settings - Fork 450
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Archiving Event Streams #1612
Comments
Per conversation w/ @oskardudycz & @babuannamalai -->
|
@jeremydmiller I think that still, we'll need to provide an option to mark events as deleted. Besides removing the whole stream, someone may want to just archive events older than specific events. It's a common practice to send a summary "end of booking day" event that checkpoints the current state and allows to move the events to the "cold storage". Such events should be available for moving to some other storage (e.g. for compliance reasons). I think that it'd be good to have such metadata both on the events and stream information. Using partitioning, it's tempting to simplify operational behaviour, as it allows out of the box detaching partition. However, I need to verify at first how this will behave for already existing data. Depending on how we approach that, this might not be a breaking change (if we add it as an opt-in). |
Assumptions, Questions, & Given Tasks
Do this through partitioning or by a separate table?If by partitioning...
If by a separate table
EvaluationI've gone back and forth quite a bit. At this point, I think I'm back to leaning partitioning, but that's dependent upon us believing that the one time data migration to enable the partitioning is okay |
We're going w/ the partition strategy. So, tasking:
|
More soon to come...
The text was updated successfully, but these errors were encountered: