allocating sequence numbers should be transactional with writing blocks

[snarfed]

From snarfed/bridgy-fed#1130:

I suspect this is because we're not allocating sequence numbers and writing blocks in the same transaction. If we allocates a seq for the new repo's initial commit, and then another commit allocates the next seq and writes its commit, and our firehose reads it, before we write ours, our firehose will skip our commit entirely. And I'm more likely to trigger that when I do this kind of manual change locally, outside GCP, because of the network distance.

Ugh.

[snarfed]

Another layer of defense here would be, if our firehose sees a missed sequence number, wait for a while, but not indefinitely, to see if it shows up before continuing on.

[snarfed]

Started on this in the allocate-seq-transactional branch, above, but it's not pretty, and not functional yet. I may try the query idea in #34 (comment) instead.

[snarfed]

Another option here: drop the storage API abstraction and fully tie this library to the GCP Datastore. Not ideal, but it would make this refactoring much easier.

[snarfed]

Yet another option: actually refactor the Repo and Storage APIs to let storage implementations finalize commits - notably, generate their CIDs and signatures - all at once, so they can put it all inside a single transaction.

Wouldn't be backward compatible, but we're not 1.0 yet, that's still reasonable.

[snarfed]

Another layer of defense here would be, if our firehose sees a missed sequence number, wait for a while, but not indefinitely, to see if it shows up before continuing on.

I did this, commit above. Seems ok before, and we're definitely hitting it, averaging once every few min or so. 🤞🤞🤞

[snarfed]

Leaving this open to track combining allocating seqs and writing blocks into the same tx, but that seems lower priority if the query fix makes sure we still emit every seq.

[derspyy]

sorry for suddenly appearing here, but does that workaround allow me to maybe have my account's metadata filled in?

if you recall, i had an issue where my avatar, banner, username weren't going through:
https://bsky.app/profile/did:plc:iykvw5xeefafbsai4ygi6dk3

[Tamschi]

I used the refresh button on https://fed.brid.gy/ap/@piuvas@capivarinha.club earlier, but the profile still didn't update.
There might be something else still stuck in this case.

[snarfed]

Sorry @derspyy! The Bluesky team is working on handle the huge growth they've seen over the last week or two. They're making some upgrades and changes to their relay, and it's currently behind on ingesting commits from federated PDSes like BF. Hopefully that will clear up soon. bluesky-social/atproto#3036