Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTP retrievals only + more templates #12

Closed
Tracked by #15
bajtos opened this issue Aug 28, 2023 · 2 comments
Closed
Tracked by #15

HTTP retrievals only + more templates #12

bajtos opened this issue Aug 28, 2023 · 2 comments
Assignees

Comments

@bajtos
Copy link
Member

bajtos commented Aug 28, 2023

ETA: 2023-11-30

  • Rework our job templates to use only (CID, SP) pairs supporting retrievals using the HTTP protocol.
  • Enlarge our list of job templates to the top 1000 CIDs (or more). We need this for the next milestone, where we want to choose ~100 tasks per round. Selecting 100 random tasks from 200 templates wouldn’t work that well.
  • Rework the SPARK module and hard-code retrievals to use the HTTP protocol.

Earlier discussion about IPNI protocols: https://filecoinproject.slack.com/archives/C03PQG6UT2B/p1690560331187479

  • HTTP Gateway transport has code 2336 (0x0920)

From https://github.com/multiformats/multicodec/blob/master/table.csv

name tag code status description
transport-ipfs-gateway-http transport 0x0920 draft HTTP IPFS Gateway trustless datatransfer
@bajtos bajtos self-assigned this Aug 28, 2023
@bajtos bajtos moved this to 🏗 in progress in Space Meridian Aug 28, 2023
@bajtos bajtos moved this from 🏗 in progress to 📥 todo in Space Meridian Aug 28, 2023
@bajtos
Copy link
Member Author

bajtos commented Aug 29, 2023

We can get top 1000 CIDs from Saturn here: https://orchestrator.strn.pl/top-cids?limit=1000

@bajtos
Copy link
Member Author

bajtos commented Nov 6, 2023

In Saturn's TOP 1000 CIDs, only web3.storage advertises HTTP protocol to IPNI. If we disabled protocols other than HTTP, then SPARK would become a checker for web3.storage only, which does not make sense to me. Let's keep testing all three protocols (bitswap, graph sync, http) for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: ✅ done
Development

No branches or pull requests

1 participant