Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NIP96 - Adding pubkeys to file urls to allow data migration between media servers. #1097

Closed
wants to merge 1 commit into from
Closed

NIP96 - Adding pubkeys to file urls to allow data migration between media servers. #1097

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions 96.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ The upload response is a json object as follows:
// Could also be any url to download the file
// (using or not using the /.well-known/nostr/nip96.json's "download_url" prefix),
// for load balancing purposes for example.
["url", "https://your-file-server.example/custom-api-path/719171db19525d9d08dd69cb716a18158a249b7b3b3ec4bbdec5698dca104b7b.png"],
["url", "https://your-file-server.example/custom-api-path/pubkey/719171db19525d9d08dd69cb716a18158a249b7b3b3ec4bbdec5698dca104b7b.png"],
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a fan of changing the URL scheme now. Also, I do not see the point of this or how it will help anything. There might be a use case for this, but since we do not (or should not) allow directory browsing, having npub in the URL does not really help.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, especially when #1236 would allow the user to list their files and easily migrate them.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The goal is to make it easy to know if a file was uploaded by a pubkey or another, it is clear that with a call to the API we can have it, but for a user it will not be easy.

I think this is in line with what blossom promotes, url's standards, in this case the original pubkey (in my opinion) is very relevant.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if you are considering that maintaining additional information comes at a cost, compute or otherwise. Adding npub to a URL will require to somehow know that this npub has this hash under it, and that some other npub might have the same hash. The less info you need to resolve what to serve, the cheaper, faster and more efficient you can do it.

I see it all the time on the nostr, most people think in single servers and single DBs, and nothing about how and when can you scale. If you run a single server then you are good. If you run a highly distributed infra across the globe and trying to shave nanoseconds of time from each request because you serve thousands a second, and trying to remove any possible call to anything when serving a file. Then you are screwed with all of this. It is nice to have things that nobody needs or cares about.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@quentintaranpino I think a better and simpler way to solve the issue you mention on this pull request is the following:

  1. Obtain a list of the servers that were used to upload media.
  2. Query those servers for all files.
  3. Upload them to the new server.
  4. Request deletion from the original server.

This does not require having the npub in the url.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The goal was to not rely on API calls to know if an image was uploaded by a pubkey. In case nobody sees the point, I give up 😂

@arthurfranca @v0l @fishcakeday @sant0s12

I am going to close this PR, if you think there is something interesting in it you can add it to the other PR's that are open.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The spirit of this PR will live on https://link.to/<image-sha256>.png#ox=<hash>&nip96u=<pubkey> kind of urls if nip96u tag is added heheh

// SHA-256 hash of the ORIGINAL file, before transformations.
// The server MUST store it even though it represents the ORIGINAL file because
// users may try to download/delete the transformed file using this value
Expand Down Expand Up @@ -226,21 +226,27 @@ However, for all file actions, such as download and deletion, the **original** f

## Download

`Servers` must make available the route `https://your-file-server.example/custom-api-path/<sha256-file-hash>(.ext)` (route taken from `https://your-file-server.example/.well-known/nostr/nip96.json` "api_url" or "download_url" field) with `GET` method for file download.
`Servers` must make available the route `https://your-file-server.example/custom-api-path/<pubkey>/<sha256-file-hash>(.ext)` (route taken from `https://your-file-server.example/.well-known/nostr/nip96.json` "api_url" or "download_url" field) with `GET` method for file download.

The primary file download url informed at the upload's response field `nip94_event.tags.*.url`
can be that or not (it can be any non-standard url the server wants).
If not, the server still MUST also respond to downloads at the standard url
mentioned on the previous paragraph, to make it possible for a client
to try downloading a file on any NIP-96 compatible server by knowing just the SHA-256 file hash.

The server **must** report the pubkey that uploaded the file in the url, thus enabling interoperability between servers, allowing uploaded data to be migrated from one server to another while respecting the structure of the url.

Clients should scan the user's configured NIP96 server list when a file is not available at the original url using the

Example: `<download_url>/375fdc8cb766664da915d638f46ca0399cd13cbd81a3f25eb37371d9dbe1bc81/719171db19525d9d08dd69cb716a18158a249b7b3b3ec4bbdec5698dca104b7b.png`

Note that the "\<sha256-file-hash\>" part is from the **original** file, **not** from the **transformed** file if the uploaded file went through any server transformation.

Supporting ".ext", meaning "file extension", is required for `servers`. It is optional, although recommended, for `clients` to append it to the path.
When present it may be used by `servers` to know which `Content-Type` header to send (e.g.: "Content-Type": "image/png" for ".png" extension).
The file extension may be absent because the hash is the only needed string to uniquely identify a file.

Example: `https://your-file-server.example/custom-api-path/719171db19525d9d08dd69cb716a18158a249b7b3b3ec4bbdec5698dca104b7b.png`
Example: `https://your-file-server.example/custom-api-path/375fdc8cb766664da915d638f46ca0399cd13cbd81a3f25eb37371d9dbe1bc81/719171db19525d9d08dd69cb716a18158a249b7b3b3ec4bbdec5698dca104b7b.png`

### Media Transformations

Expand Down