Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ATProto firehose events from the past have current timestamp #1015

Closed
mackuba opened this issue May 6, 2024 · 2 comments
Closed

ATProto firehose events from the past have current timestamp #1015

mackuba opened this issue May 6, 2024 · 2 comments
Labels

Comments

@mackuba
Copy link

mackuba commented May 6, 2024

When you connect to a PDS's firehose with a cursor in the past, it sends events from the past - but the events have a timestamp, and those are normally the timestamps of when the event was originally created with that seq.

See e.g. here from Amanita:

Screen Shot 2024-05-06 at 02 04 36

I connected with a cursor from 6 hours ago, it shows events creating posts with a date from 6 hours ago, and the commit events have ~ more or less the same timestamp, modulo the time difference of the createRecord request (since the post createdAt is set on the client before the request and the commit time is set on the PDS once it receives it).

And here's Bridgy (just a moment later):

Screen Shot 2024-05-06 at 02 04 48

I connected with an older cursor, and it shows me commit events of posts from 3rd May, 2+ days ago (commits were created a bit later than the posts since the sync is not instant, but should be just maybe up to 1 hour later). But the commit timestamps are current time, 6th May.

I've dug a bit into the code and I think this is the line in question? https://github.com/snarfed/arroba/blob/main/arroba/xrpc_sync.py#L123

Also I've looked at where this data used there is coming from, and it looks like you might not have the commit timestamp saved at all (the Block structure)? I think it would be good to start saving it whenever a commit is created, basically same place where the seq is kept and written down same time when seq is incremented.

I'm honestly not sure how much this matters in terms of protocol - it might not matter at all. I don't know what the Relay takes into account. But it's not impossible that it's also looking at commit timestamps and might get confused. In any case, any third party devs working with the Bridgy PDS somehow would also be :)

@snarfed
Copy link
Owner

snarfed commented May 6, 2024

Good call, and good sleuthing! Agreed, I don't exactly know what would use or depend on this either, but it can't hurt to use the real timestamp. And you're also right that Block doesn't have it, but the underlying AtpBlock does, in AtpBlock.created, so I'd just need to expose it up from there.

This could be a pretty straightforward one to tackle if you're interested and want to dust off your Python!

@snarfed
Copy link
Owner

snarfed commented May 17, 2024

Fixed! Haven't deployed yet, I will soon ish.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants