Flexible Event Backdating Threshold

[Kobby-Bawuah]

Currently, our system enforces a hard limit on how far back users can date an event, which is set at 30 days. This limitation can pose challenges for users who require flexibility within their organization's retention policies. For example, a user on a business plan with a 90-day retention period has expressed the need to have the ability to backdate events within the 90-day retention window.

The existing behaviour adjusts the timestamp of events older than 30 days to the ingest time, indicating that the original timestamp should be taken with caution. While this behaviour has its reasons, it may not align with users' needs and can lead to data loss for older events.

For context, if they go longer than 30, they get

Invalid timestamp (too old) (1)
Adjusted timestamps by a month

Request:

We would like to propose a feature that allows users to set the event backdating threshold based on their organization's retention policy. This would provide greater flexibility and better align our system with our users' needs.

[jjbayer]

@Kobby-Bawuah thanks, two questions:

1. Do you know where the need to backdate that far into the past comes from? If events are that old, where were they buffered before sending them to sentry?
2. Do they have this requirement only for error events, or also for performance and release health data (that is, transactions and sessions)? Asking because for transactions and sessions, we have an even stricter limit of 5 days.

[Kobby-Bawuah]

@jjbayer
So this was generated from a discussion I was having with a user. Please forgive the lack of knowledge on the users specific use case.

• The need to backdate events to as far as 90 days arises from a migration process. Felipe’s company is migrating an existing error reporting system into Sentry and has 90 days' worth of data that needs to be imported into Sentry.

• It’s implied that the events were stored in the previous error reporting system that Felipe’s company was using before migrating to Sentry. The discussion does not explicitly mention where the events were buffered, but they likely resided in that system.

• The discussion does not explicitly specify if the backdating requirement applies to performance and release health data (transactions and sessions). It is focused on error events. The stricter 5-day limit for transactions and sessions is mentioned only in your query, not in the discussion itself. Therefore, it's unclear whether Felipe’s company has similar requirements for those types of data.

[iambriccardo]

We started to work on this ticket and our proposed solution would be to use the event retention time as the backdating threshold. This means that for most customers this will equal to 90 days.