Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NIP-59 Gift Wrap #468

Closed
wants to merge 3 commits into from
Closed

NIP-59 Gift Wrap #468

wants to merge 3 commits into from

Conversation

v0l
Copy link
Member

@v0l v0l commented Apr 23, 2023

A stupid simple solution to DM metadata leakage.

Is it terrible? Yes!

Does it work? Also Yes!

https://github.com/v0l/nips/blob/59/59.md

@Giszmo
Copy link
Member

Giszmo commented Apr 24, 2023

The use of ephemeral keys was proposed before in different ways and was always the weak point. Relays will not be too happy to accept ephemeral keys, especially when the content is encrypted. So the relay you are talking to would have to learn it's you which brings us back to what fiatjaf and others favor: the relay learns the meta data but promises the sender and receiver of DMs to not let unauthorized clients request these DMs.

The advantage of using ephemeral keys is that in theory, events could be mirrored to other relays without damage but that holds DOS and spam potential.

Copy link
Collaborator

@Semisol Semisol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is the temporary key communicated?

@v0l
Copy link
Member Author

v0l commented Apr 24, 2023

How is the temporary key communicated?

Its not, it's used only wrap the actual DM event, but as it turns out this has been suggested before (forgot to check closed PR's/Issues).

And for reasons which @Giszmo has suggested it makes sense that this would be bad for relays, although as it currently stands free relays already have to deal with random keys spamming them.

@d-krause
Copy link

d-krause commented Apr 24, 2023

This is essentially what I have been working on with nip 76 (https://github.com/d-krause/nostr-nips/blob/nip76-draft-2/76.md)

It does not get any interest, but I am still pressing on with Draft 3 that addresses more concerns. I guess you could say that Nip 76 Keys are "ephmeral", but they are also deterministic, using one determined key for the author and another determined key for the encryption. We could let relay operators in on the author determined key, but still withhold the encryption determined key.

Something along these lines that masks the pubkey needs to happen and it will happen one way or another despite what relay operators like and dislike. It is high time we stop storing or data in the clear, both in terms of whether the event is readable AND who originated the event.

If we can't get relay operators that control this repo on board, I can see a Nostr fork coming where there will be operators that are on board. I don't understand their resistance anyway - they are just "NIPS" - not mandates. They can approve a NIP without agreeing to conform to it.

@Giszmo
Copy link
Member

Giszmo commented Apr 25, 2023

although as it currently stands free relays already have to deal with random keys spamming them.

Well, nip-42 is only now being rolled out. No client supported it a week ago.

@Giszmo
Copy link
Member

Giszmo commented Apr 25, 2023

I said it many times and I will repeat it here, too. I don't like any approaches that require limiting event replication across relays. A new relay should be easy to spin up and saturated with past events and other relay operators should only charge for what it costs them to stream over all that data. There should not be information privy to one relay or another. That is why I am in favor of plausible deniability in the DM meta data using a nip like my #17

With approaches that limit DMs to one relay to not leak too much meta data make parties to the conversation dependent on that relay.

@threeseries
Copy link

@v0l This NIP is not quite the same as #499. In it I suggested using a random key pair to encrypt the inner event, but the key doesn't have to be random. Instead the event could be sent using one's own private key to an intermediate pubkey which is trusted to forward the message along to someone else. That should help hide sender and receiver as long as the timing isn't too obvious. To @Giszmo's point both messages can also be broadcast everwhere so there's no concern about events being limited to specific relays.

Such a forwarding service could be a very simple bot that's always online, and could also be programmed to only forward messages once enough are in a queue, giving deniability to everyone using the service.

@starbackr-dev
Copy link
Contributor

Hello Everyone, This is Arun from the Current App team.

There are several open pull requests aimed at solving the privacy issues surrounding Direct Messages on Nostr. Our team has conducted research on this matter over the past few weeks. Many of these requests involve the exchange of keys between two users to ensure security or create secure channels. They are complex in terms of preventing leaks and ensuring privacy measures are in place.

We have also examined the use case from the perspective of end users. Almost all the proposed solutions address the scenario of "We are friends and need to communicate securely, so let's use Nostr DM." We believe this is not a valid use case for Nostr.

The use case for DMs on Nostr is quite different. It begins with a message like, "Hey, I saw your post about attending this meetup. Do you have time to meet up in person?" The content of the event is already encrypted. The only issue here is the leakage of metadata, which could be used by anyone to create a social graph or for spamming profiles that receive DMs from trending profiles.

This NIP-59 proposal, which wraps DMs as a gift into another event, works well for this use case by addressing the metadata leakage. After reviewing other solutions, we kept returning to this NIP because of its simplicity and effectiveness within Nostr.

Here are the different levels of DM privacy that I see:

Level 1 - Kind 4: Content is encrypted, but I have no concern about metadata leaks.
Level 2 - Kind 1059: I am comfortable trusting my paid relay to view my metadata, but not others.
Level 3 - Kind XX: I will use onion routing to obfuscate metadata and send it to routing bots.
Level 4 - I need NSA-level super-private secure, encrypted DMs to use with my friends.

We should begin by implementing level 2, which requires changes to the client, and then move on to implementing level 3. Level 4 should never be implemented within Nostr and direct them to Signal or SimpleX.

Of course, this does not solve everything, and a couple of issues were raised during this discussion. Let me address them.

  1. There is an argument that "throwaway" keys will not work with paid relays. However, the majority of relays are still open and will automatically work with this NIP. The reason paid relays block throwaway keys is that they perform authentication during write rather than on connection. At least, this is what I observed with paid relays built on Nostream. I modified the code to authenticate on connection instead of write, and now it works with throwaway keys as well.
    Here’s a code snippet that connects to our DM relay using NIP-42 AUTH and then sends DM using throw away keys. https://gist.github.com/starbackr-dev/558c30c126bcb2e10d39e1dc9c58d8a6

As of today, There are 120 free relays Vs 65 paid relays. Using any of the free relays with NIP-59 will stop the metadata leakage and make it anonymous.

  1. Relay operators can still link throwaway keys to the original profile using IP addresses or by linking the authentication public key and the throwaway key. We can always enhance the NIP to prevent this by using routing relays/bots that act as intermediaries to pass the message, thereby addressing the concern. We can also create relays specifically designed for ingesting DMs and declare that they will not perform any data mining. In fact, we have already built one for Current App users.

This is also backward compatible and no user flow changes required. Clients query for both kind 4 and 1059 and unwraps kind 1059 into kind 4 automatically. When starting a new conversation they can choose a less(kind 4) or more secure (kind 1059) way to send DMs. Overtime we believe all Nostr clients will start to support kind 1059.

Couple of weeks ago we released a version on Current App with DM and push notifications. @v0l we are ready to start supporting and releasing a test version to see how the users react to it.

@Semisol - Your requested changes has been addressed. Can you review it one more time?

One reason why I enjoy developing on Nostr is its simplicity. We should not compromise that for complex solutions that no one will use in the end.

It's a simple, straightforward solution. Simple always wins... not SimpleX.

@paulmillr
Copy link
Contributor

Thanks for the analysis @starbackr-dev, good to have someone who's thinking about threat model.

The pull requests looks good to me. One small concern is that 1059 doesn't look cool. Maybe something more memorable and related to DMs?

@starbackr-dev
Copy link
Contributor

Agreed. Kind-1004 may be a good one to pick.

Kind 4 - legacy DM encryption with metadata leaks
Kind 1004 - Upgraded encryption with a fix to metadata leaks.

@v0l - your comments?

@paulmillr
Copy link
Contributor

Why not 444?

@starbackr-dev
Copy link
Contributor

Yes. 444 is even better..!

@paulmillr
Copy link
Contributor

In the meantime i've introduced NIP-44 that fixes broken cryptography of NIP-4 and introduces versioning:

#574

@arthurfranca
Copy link
Contributor

Why not 444

Because of NIP-16. I think kinds below 1000 are considered legacy, because it isn't clear how relays should handle them.

@arthurfranca
Copy link
Contributor

@starbackr-dev: [...] the majority of relays are still open and will automatically work with this NIP. The reason paid relays block throwaway keys is that they perform authentication during write rather than on connection. At least, this is what I observed with paid relays built on Nostream. [...]

Afaik they don't use NIP-42. Paid relays are just checking the pubkey of received events, so the side-effect is that one can only send events of allowed pubkeys. That's why NIP-59 won't work for now.

This probably has to do with the fact that NIP-42 flow is unpredictable in the sense that it is hard for the client to know when and if the relay 'AUTH' message will arrive. So just a few clients enable this, which makes relay don't require it.

Please support NIP-43 - Fast Authentication PR for a better alternative.

@earonesty
Copy link
Contributor

earonesty commented May 31, 2023

this is exactly how we are doing encrypted group chat invitations in arclib. we're also using a "fixed" nip4, which is basically the same as nip44. i see no problem with this proposal and would love it to be a standard. relay authentication can be trivially handled in many ways: - a special "auth" tag that the relay strips on send for example.

i would think that a simple json tag that the relay provider agrees to remove immediately is the best way to go for auth

it's easy to understand, and if they fail to remove it, they've only doxxed one ephemeral key... now you can ditch them and move to someone that honors it

@Egge21M
Copy link
Contributor

Egge21M commented May 31, 2023

this is exactly how we are doing encrypted group chat invitations in arclib. we're also using a "fixed" nip4, which is basically the same as nip44. i see no problem with this proposal and would love it to be a standard. relay authentication can be trivially handled in many ways: - a special "auth" tag that the relay strips on send for example.

i would think that a simple json tag that the relay provider agrees to remove immediately is the best way to go for auth

it's easy to understand, and if they fail to remove it, they've only doxxed one ephemeral key... now you can ditch them and move to someone that honors it

Interesting thought. Some trust in the relay is always required anyways unless you use different proxies for every connection to cloak your up address... so you envision an auth-tag that has the "main" pubkey as well as a signature on it?

@staab
Copy link
Member

staab commented Jun 1, 2023

I haven't been following the NIP 04 discussion, but I think general-purpose gift wrap is an amazing idea. Maybe not to fix DMs necessarily, but if we combine it with nsec bunker and NIP 51, we get role-based authentication for private groups for basically free. Either I'm nuts or this is a huge idea. Blogged about it here

@starbackr-dev
Copy link
Contributor

@staab Great idea. Can you point me to where I can read more on nsec bunker ?

@madelainemichele
Copy link

@staab Great idea. Can you point me to where I can read more on nsec bunker ?

Here is link to an article and utube video
https://www.nobsbitcoin.com/nsecbunker-announced/
nsecBunker setup demo https://youtu.be/2oojFSoQ-So

@earonesty
Copy link
Contributor

earonesty commented Jun 1, 2023

Interesting thought. Some trust in the relay is always required anyways unless you use different proxies for every connection to cloak your up address... so you envision an auth-tag that has the "main" pubkey as well as a signature on it?

the AUTH spec is already ephemeral, you don't need the tag. provider tosses away a fake AUTH event. so imo, no doxxing is needed. better than a tag, auth the connection in accordance with https://github.com/nostr-protocol/nips/blob/master/42.md

we could come up with a new auth tag per event. that could help with stuff like real onion routing through private relays. tradeoffs: bigger & routable vs. faster & stateful

59.md Outdated

This event kind can be used to wrap DM's or other events which are considered private.

This is similar in concept to Onion Routing.
Copy link
Contributor

@earonesty earonesty Jun 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the sender of the event would like to decrypt it at a later date, they can use their "tweaked" private key, rather than a random key when producing the event. the "tweak value" can use any outer-envelope data, depending on the needs of the user. tweaking secp256k1 private keys is a well-known mechanism, and should be considered if a client wants to be able to read these events later from a message store, for example.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any change in how key material is handled needs to be added to NIP-07 or else it's not possible for me to use, this is why ephemeral keys work for me because i create the secret material.

Copy link
Contributor

@earonesty earonesty Jun 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this would not be a change to how key material is handled. these keys are still ephemeral. its just that if you really wanted to decrypt it later, however annoying that may be. you, optionally, could.

doesn't have to be part of this NIP, but it might want to be mentioned that since the creator of the event controls the ephemeral key, there's no hard requirement that it not be a derived key or a weak key, and no guarantee that the recipient can be assured that the creator cannot decrypt it later, or that the key was made poorly

if you wanted to be able to guarantee the key is hard, you can derive the ephemeral key in such a way that the recipient can verify it

for example, you can use the inner event id as the private key

then you'd have to crack the dlp or reverse a hash to create a weak key - the recipient can be assured the ephemeral key is free of artifacts or structure

Copy link
Contributor

@arthurfranca arthurfranca Jun 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of adding a nonce key with a random string to the wrapped event JSON (not as a tag, but as an extra field)?

So for the DM example, you could send the gift wrapped DM with 2 1059 events from different random keys and 2 nonces, one to the recipient and another one to yourself (so you can keep chat history). With 2 nonces, the 2 gift wrap contents won't match, so no one can tell it is the same message.

This way you don't need to keep track of the wrapping key for decrypting chat history.

The down side is the created_ats will be close to each other and your copy won't have a p tag with the recipient so you cant fetch just the messages sent to a specific user. =(

Copy link
Member

@staab staab Jun 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is another way to solve this problem (not the decryption part, but the querying part).

What about optionally adding tags to the outer event, but encrypted? So ["p", pubkey] would be added to the wrapper as ["p", "encrypted pubkey"]. This would allow users in the know to still execute filters, supporting larger groups (though still not scaling as well as native nostr). A new k tag could be used to denote the kind of the inner event as well.

A 1k member group, each publishing 1k events over the course of a year would result in 1MM events, which would be far too many to process without filters.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@staab nice so the sender should encrypt the receiver pubkey using the shared secret

But from NIP-04, is secp256k1.getSharedSecret(SENDER_privkey, '02' + RECEIVER_pubkey).slice(1, 33) the same value as secp256k1.getSharedSecret(RECEIVER_privkey, '02' + SENDER_pubkey).slice(1, 33)? So the receiver can use it to also encrypt his own pubkey and query with { #p: [encrypted_receiver_pubkey] }?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No idea, encrypted tags would rely on signing with a shared secret. This PR uses ephemeral keys, but it would be easy to convert it to using a shared secret (since one is necessary for decryption anyway). Encrypted tags could then be encrypted symmetrically.

@earonesty earonesty mentioned this pull request Jun 1, 2023
59.md Outdated Show resolved Hide resolved
59.md Outdated
"pubkey": "63fe6318dc58583cfe16810f86dd09e18bfd76aabc24a0081ce2856f330504ed",
"content": "73ZaB2WzPlIwwC55YnhsFA==?iv=1R2WlyrAk9VQ1NdhNuWmsw==",
"kind": 4,
"created_at": 1682286690,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

created_at can be made optional to save space, verifier assumes created_at is the same, and verifies the sig as if it is present (not as if it is excluded!).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the additional complexity is worth the space saving.

@vitorpamplona
Copy link
Collaborator

We can create another name. The only point is that any event can be unsigned and put it into a warp to hide it's nostr properties from the public.

@earonesty
Copy link
Contributor

earonesty commented Jul 28, 2023 via email

@earonesty
Copy link
Contributor

earonesty commented Jul 28, 2023 via email

@mmalmi
Copy link

mmalmi commented Jul 28, 2023

you calculate the shared secret between sender and receiver

I agree. Would be cool if window.nostr had a method for getting hash(sharedSecret). Then you could calculate your secret chat id hash(sharedSecretHash + yourPub) and theirs hash(sharedSecretHash + theirPub).

That's actually what I used to do in the old Iris which used gundb, and it worked nicely.

@earonesty
Copy link
Contributor

yeah, we do this with the arcade app, and it allows blinded dm's .. we don't use gift wrap for that, because that leads to outbound subscription issues for state reloads.

@vitorpamplona
Copy link
Collaborator

outbound subscription issues for state reloads

Hum... what do you mean by that? I seem to be running fine.

@earonesty
Copy link
Contributor

earonesty commented Jul 28, 2023

outbound subscription issues for state reloads

Hum... what do you mean by that? I seem to be running fine.

you cant load the history of outbound messages when you log out and log back in... because with gift-wrap you have no idea who sent it. but if you use the hash(shared-secret + pub) as a private key... then you can easily subscribe to that author on the relay, and see all the fully blinded dm's. also you don't leak metadata if you use hash(shared-secret + pub) .... because you don't need any tags. you just always sub to the author.

@vitorpamplona
Copy link
Collaborator

you cant load the history of outbound messages when you log out and log back in... because with gift-wrap you have no idea who sent it.

ohh I see. Yes, I agree. That's why I am sending a GiftWrap to myself as well.

hash(shared-secret + pub) as a private key

How do you communicate the shared-secret to the other side?

because you don't need any tags

There should be at least one tag of the subscription id, no?

@earonesty
Copy link
Contributor

you cant load the history of outbound messages when you log out and log back in... because with gift-wrap you have no idea who sent it.

ohh I see. Yes, I agree. That's why I am sending a GiftWrap to myself as well.

hash(shared-secret + pub) as a private key

How do you communicate the shared-secret to the other side?

it's the standard shared secret (my priv * your pub)

because you don't need any tags

There should be at least one tag of the subscription id, no?

no need to have tags on blinded dm's. no receipient, no sender. everyone just subs to the join well-known shared-secret key

purely private dm

@vitorpamplona
Copy link
Collaborator

no need to have tags on blinded dm's. no receipient, no sender. everyone just subs to the join well-known shared-secret key

I am confused on this. Can you provide a Nostr filter example? Do they filter by author or by ptag?

@vitorpamplona
Copy link
Collaborator

it's the standard shared secret (my priv * your pub)

In this case, you won't receive messages from strangers, right? Because the app woundn't be able to subscribe to each other active pubkey out there.

@earonesty
Copy link
Contributor

correct, our system does not allow messages from strangers. also we allow private follows too (encrypted in a replaceable event)

here's the sub we use:

https://github.com/ArcadeLabsInc/arclib/blob/542ececf9ea03e9d375d2905820376e98ecba2aa/src/private.ts#L161

@vitorpamplona
Copy link
Collaborator

correct, our system does not allow messages from strangers

Yeah, that's a no go here.

https://github.com/ArcadeLabsInc/arclib/blob/542ececf9ea03e9d375d2905820376e98ecba2aa/src/private.ts#L161

Oh I see. So, you do have a "tag", but your tag is the pubkey. It's just that you are using that pubkey as a central point.

@earonesty
Copy link
Contributor

no I think that code is just to deal with various legacy stuff

basically you can tag or not tag depending on how much metadata you want to leak vs whether or not you want to allow strangers to message people. we chose more private but I imagine a good write-up of the protocol would allow for both

@vitorpamplona
Copy link
Collaborator

vitorpamplona commented Jul 28, 2023

no I think that code is just to deal with various legacy stuff

If two users are communicating with the same Private and Public Key, how do you know which one wrote each message without tags?

@vitorpamplona
Copy link
Collaborator

vitorpamplona commented Jul 28, 2023

ohh I see what you are doing. You have a custom payload where you place the pubkey of the writer together with the message in the .content of the event. You decrypt the message, get the payload, figure out who wrote it and then display the message accordingly.

So, in your scheme, you have to trust the counterparty, because the counterparty can add your pubkey in their message to pretend their message came from you.

Tricky bet.

@jb55
Copy link
Contributor

jb55 commented Aug 11, 2023

@earonesty

the sender can optionally use sender private * event id mod curve.n as the ephemeral key (derived key, not ephemeral). if this is done, the sender can "retain access" to the sent message and is capable of decrypting, deleting or otherwise operating on the outer message. this may not be strictly needed but could be useful if a sender wants to "retract" a statement in a group-chat... for example. I would propose adding this as an OPTION to the NIP-59 gift wrap.

I just re-read over this spec again and this is the only glaring thing that appears to be missing. Why no mention of deterministic gift wrap keys? Are clients just supposed to fire off random gift wraps with random keys and have no way of fetching them again? Maybe I'm missing something.

@vitorpamplona
Copy link
Collaborator

You make an extra one for yourself firing from yet another random key. It's actually really good. You can choose "Disappearing Messages" if you don't give yourself a gift for your own DMs, for instance. It happens to make a lot of sense.

In summary, you only need one filter p=your key to ALL gift wraps. Including those from and to yourself.

@jb55
Copy link
Contributor

jb55 commented Aug 11, 2023

nm I see it in there now, not sure how I missed it

@jb55
Copy link
Contributor

jb55 commented Aug 11, 2023

I see why I was confused, I was looking at the modified one by @vitorpamplona which doesn't mention deterministic ephemeral keys. who's the real gift wrap spec ?

https://github.com/nostr-protocol/nips/blob/1c9c9719e068b4dbd405133decb4bf1ec8d367fe/59.md

@vitorpamplona
Copy link
Collaborator

We have gift wraps now in 2-3 PRs, but I think they are mostly the same, just applied in different ways to add protections. The one I did is basically a double wrap of an unsigned event to make sure the wrapped event cannot leak and expose metadata.

Then @staab created a version for large groups with Group management events to try to reduce the amount of GiftWrapped messages in those large group implementations (1,000+ people).

@staab
Copy link
Member

staab commented Aug 11, 2023

Right, pull request #706 is my version. It distills double wrapping from Vitor's version down to its pure form (and has a distilled form of nip 44 as well). NIP 87 in the same PR is largely un-reviewed. If there's interest in getting 44 and 59 in, I can separate those into separate pull requests, or we can update the originals with my edits. I think the DM-specific stuff in the current versions should be superseded by Vitor's PR. Ultimately, I think we have four separate concepts here: versioned encryption, double-wrapping, small groups/dms, and large groups.

Edit: I also did a survey of DM proposals which you can read here. It doesn't include #706 because that was written afterward.

This was referenced Aug 11, 2023
@staab
Copy link
Member

staab commented Aug 11, 2023

I've created a new version of this PR at #716, @v0l please let me know if you'd like to close this PR or incorporate those changes into this one.

@v0l
Copy link
Member Author

v0l commented Aug 16, 2023

Replaced by #716

@v0l v0l closed this Aug 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.