indexer formal restore 1/N: restore packages and move objects

[gegaowp]

Description

first of the stack PR to restore indexer from sui archives & formal snapshot

Test plan

test locally with GCS buckets, local snapshot dir and local PG server

██████████████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 75 out of 555 move object files restored (Restored 569395 live move objects and 0 wrapped or deleted objects from epoch_500/1_53.obj)2024-09-10T14:21:39.854508Z  INFO sui_indexer::restorer::formal_snapshot: Finished downloading move object file Path { raw: "epoch_500/1_125.obj" }
2024-09-10T14:21:44.111960Z  INFO sui_indexer::restorer::formal_snapshot: Start persisting 565556 move objects from epoch_500/1_125.obj
2024-09-10T14:22:12.142760Z  INFO sui_indexer::store::pg_indexer_store: Persisted 563126 objects snapshot elapsed=7058.20103075
2024-09-10T14:22:12.142830Z  INFO sui_indexer::restorer::formal_snapshot: Finished persisting 0 wrapped or deleted objects from epoch_500/1_86.obj
[10:46:20]

need to benchmark in the production env

---

Release notes

Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.

For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.

• Protocol:
• Nodes (Validators and Full nodes):
• Indexer:
• JSON-RPC:
• GraphQL:
• CLI:
• Rust SDK:
• REST API:

[vercel]

@gegaowp is attempting to deploy a commit to the Mysten Labs Team on Vercel.

A member of the Team first needs to authorize it.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
sui-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Sep 19, 2024 3:42pm

3 Skipped Deployments

Name	Status	Preview	Comments	Updated (UTC)
multisig-toolkit	⬜️ Ignored (Inspect)	Visit Preview		Sep 19, 2024 3:42pm
sui-kiosk	⬜️ Ignored (Inspect)	Visit Preview		Sep 19, 2024 3:42pm
sui-typescript-docs	⬜️ Ignored (Inspect)	Visit Preview		Sep 19, 2024 3:42pm

[gegaowp]

@bmwill @emmazzz I changed the formal snapshot read from GCS to S3 and updated it with the test plan, can I have a review? next step is to benchmark it in GKE with CloudSQL and I am working on it.

[bmwill]

@williampsmith would you be able to take a look at this as well?

[amnn]

It would be good to get some doc comments in here before we land it!

[williampsmith]

Overall this LGTM. Curious to see how this performs, as it looks like you have a better parallelism construction in the object download portion. We should do the same on the sui side.

[gegaowp]

@williampsmith I benchmarked the restorer, the whole restorer to PG DB took about 3 hours while seems PG DB was the bottleneck.