NIP-35: Torrents

[v0l]

Making a NIP for torrents with a few updates

read: https://github.com/nostr-protocol/nips/blob/feat/nip35/35.md

[v0l]

@hzrd149

[hzrd149]

This looks great. however It could be worth mentioning k:2004 events that dtan and noStrudel use for comments on torrents since I think they could be useful for other clients wanting to implement this nip

The comment events are similar to kind 1 and follow NIP-10

[utxo-one]

Great NIP!

[hzrd149]

looks good, I like the addition of the comments 👍

[flox1an]

FYI this PR uses i tags for external references: #1185

Could make sense for imdb, tmdb, .. and the like.

[jb55]

what about v2?

[jb55]

v2 is a lot more interesting that v1: merkle torrents so you can incrementally update a torrent file over time while sharing parts, etc. the same file can share seeders across many different torrents. I having been looking at v2 for notedeck file sharing, but it looks like this is hardcoded to v1.

[fiatjaf]

Is anyone using v2?

[jb55]

they should be, but it is not widely adopted yet no.

[jb55]

it would be useful if the torrent info-dictionary was embedded in the note itself, bencoded or as json. That way you could express v1 or v2 torrents, and you could see all of the files, their file-level merkle root hashes (in the case of v2), and the corresponding trackers (EDIT: looks like this is outside info-dictionary... so maybe that should be in their own tags)

Then nostr clients could open the note as a torrent, allowing you to download individual files from bittorrent or even blossom nodes by looking at file hashes. v2 aligns nicely with blossom because it switches to SHA256

https://www.bittorrent.org/beps/bep_0052.html

[jb55]

Hmm started diving into the spec and the pieces root may not be the same has the file hash, so this may foil my plan to have it compatible with blossom. darn...

[fiatjaf]

Regardless of that I like the idea of having the metainfo inside the event.

[jb55]

although one interesting observation is by breaking a file into 16KiB blocks you can always reconstruct the corresponding bittorrent v2 file root hash for a given file:

so a smart blossom seeder that wanted to help seed v2 torrents would:

• split entire file into N 16 KiB blocks, hash these blocks individually, and then hash these all together into the bittorrent-v2 sha2 file hash (root hash)
• store this root hash alongside the standard sha2 hash of the file
• A hybrid torrent/blossom client could check to see if a blossom server has a file from some torrent by making a HEAD request for the root hash

Notice because of a branching factor of 2, the fact that piece sizes must be powers of 2, and fixed 16KiB block sizes, the root merkle hash never changes, even if the piece size changes.

Let's say the client is only missing the 2nd block from the torrent file, we can make a range request against the blossom server for the second block:

GET /5584a1cc51f777b13b9de2a6a1012ee0e7a93652143c415901b8fe005ed1124c HTTP/1.1
Host: blobstr.xyz
Range: bytes=16384-32768

Where 5584a1cc51f777b13b9de2a6a1012ee0e7a93652143c415901b8fe005ed1124c is the root hash

Then we can hash the response and match it against our torrent metadata piece layers to confirm it's correct.

Alternatively the blossom node can become a bittorrent peer. Anything uploaded to the blossom server could serve pieces to v2 swarms. bittorrent v1 has the concept of web seeds, so maybe this idea is just extending that to v2

This is getting a bit off topic, but just wanted to put this out for consideration when it comes to torrents + nostr.

[v0l]

compatible with blossom

Not a huge fan of Blossom, there is a lot of overlap with NIP96, it would have been nicer to reuse that spec and expand.

it would be useful if the torrent info-dictionary was embedded in the note itself,

Sounds good to me, i can add this to the spec if you still think its a good idea?

[arthurfranca]

Not a huge fan of Blossom, there is a lot of overlap with NIP96, it would have been nicer to reuse that spec and expand.

Finally someone said it! Never got people praising blossom. It is a NIP-96 rip-off hurting interop.

Blossom spec is exactly the same as NIP-96 but with small changes that bring incompatiblity like kind:10063 for discovery instead of kind:10096 and JSON responses that have the same fields from NIP-96 but with different structure for no reason.

Blossom only additions that could be simply added to NIP-96: a route to list uploaded items of a pubkey, a HEAD one to check for an item existence.

[jb55]

blossom, nip-96, blobstr, it's all the same, the only relevant thing to my proposal above is and endpoint for fetching data in a CAS-way. blobstr is the most minimal that is needed to support this:

• Draft for file archival and retrieval NIP #1098 <- blobstr

[v0l]

blossom, nip-96, blobstr, it's all the same, the only relevant thing to my proposal above is and endpoint for fetching data in a CAS-way. blobstr is the most minimal that is needed to support this:

• Draft for file archival and retrieval NIP #1098 <- blobstr

Fair enough, ive been building a Blossom/NIP-96 server over here: https://git.v0l.io/Kieran/void-cat-rs would be cool to trial the bittorrent compat with this.

One thing im not sure of is how you get the hash of each file from the torrent, since we're only storing the root hash, do we also need to store the hash of each file? because we have the file entry already, maybe we should include the sha256 in those tags?

And once we have a way to link these torrent files which servers would you lookup?

[jb55]

One thing im not sure of is how you get the hash of each file from the torrent, since we're only storing the root hash, do we also need to store the hash of each file? because we have the file entry already, maybe we should include the sha256 in those tags?

And once we have a way to link these torrent files which servers would you lookup?

I think the embedded torrent file should maybe be an optional thing, because some torrent files can be quite large. yeah I think the sha256 hash would be nice to have, bonus points for the v2 file hash (16KiB block sha2 merkle tree with branch factor 2)

maybe disregard my crazy idea alltogether, I realize v2 implementations are still far off so probably won't be relevant for awhile, and v1 torrent metadata format is horrible.

[arthurfranca]

Save the planetnostr, guys! Support #1233 with a 👍? =D

[hzrd149]

I have not read through the entirety of #1233 but I think there is merit in having a specific kind for torrent comments and using a simple schema like NIP-10 for marking replies and threads

It seems reasonable that a NIP would define its primary event kind (in this case 2003) and a comment / reply kind that is specifically used for it. It seems like a better solution then using k tags as a "meta" kind

[vitorpamplona]

Thread roots should be defined by the NIP. Replies can be generalized.

Otherwise, should we have reactions, zap, labels, and report kinds for each NIP as well? Replies are no different than reactions.

[v0l]

Seems reasonable, but im not sold on the new tag structure

[v0l]

Going to merge anyway, we can always change it later once your PR is merged, nobody is really using comments on torrents yet. I only see 24 comments so far.