NIP-112: Encrypted Group Events

[earonesty]

This NIP builds on the encryption from NIP-44, the gift-wrap of NIP-59 and the distributed chat channel management of NIP-28. It is similar to NIP38, with the exception that gift-wrap is used on every message with an ephemeral public key, and issues like metadata leakage and optional forward secrecy are addressed. Also lists of participants are not published.

It works by creating a shared secret that is used to encrypt the group or channel messages. Only the participants know that secret, so only such members can read/write to this group chat. By using NIP-59, it effectively hides metadata from external users (and from the relay).

All messages are posted using the "gift wrap" protocol, meaning no information about the sender is visible to non-participants.

Because all messages are gift-wrapped, you can send any "kind" of message to the recipients. The use case of "403 group chat" is one such use case.

[fyookball]

Is it correct that this doesn't hide who is posting to the group? If so , can we also use nip76 style encapsulation to hide the pubkeys of who is posting? As long as we're combining several nips, might as well add this valuable feature.

Also, it should be made explicit in the NIP how to specify (multiple) relays for the chat... This was not adequately documented in nip-28, seems like the correct way is multiple e-tags in the metadata.

[earonesty]

Is it correct that this doesn't hide who is posting to the group?

please look at draft nip59

Maybe I should reproduce it here

The sender always rolls a new public private key pair every time they post to the group. indeed every message is sent this way even the invitations.

there's an outer message (using an ephemeral public private keypair) and an inner message (which is signed by the actual sender)

Only members can verify the inner events and clients must validate signatures, since the relays cannot do it

I will make this more clear

Also, it should be made explicit in the NIP how to specify (multiple) relays for the chat... This was not adequately documented in nip-28, seems like the correct way is multiple e-tags in the metadata.

agreed I will make that modification that is the correct way

[fyookball]

So the gift wrap is with a temporary key? That's fine but where do the other chat participants get the key from? I would understand if the wrapping for each message is done with the same key as is used to get access to the encrypted room, but if a new key is used each time, then where do the others get the key from?

[earonesty]

So the gift wrap is with a temporary key? That's fine but where do the other chat participants get the key from?

• Yes, the gift-wrap sender's key is ephemeral. All messages in this protocol are gift-wrapped in this way. There are no tags or other metadata in the outer events. All information is inside the wrapped message.
• The recipient's key is the known public key of the chatroom. or, recipients of invitations. Invitations contain the private key of the chatroom. After being invited, members can decrypt all of the chatroom messages (up until the previous optional key rotation, if any.)
• The other participants receive invitations from the room creator (or other participants)
• All messages use ephemeral sender keys but well-known recipient keys. There's no purpose or value to obscuring recipient keys in any protocol.

[paulmillr]

keep in mind that Signal group chats are basically a bunch of 1-to-1 chats. There is no shared secret. I wonder which solution is better.

[fyookball]

@earonesty ok i get it. sorry for the confused question. thanks for explaining. I like this nip. Would implement. +1 on the relay thing.

[earonesty]

keep in mind that Signal group chats are basically a bunch of 1-to-1 chats. There is no shared secret. I wonder which solution is better.

agreed, it's all tradeoffs. a shared secret with frequent rotations has some weaknesses and some efficiencies.

pros:

• persistence for mobile ux, decent censorship resistance, and is far easier to understand and build on top of.
• a full encrypted group chat lib, replete with all the subscriptions, persistence layer, etc , in easy-to-understand code/sdk that anyone can make a cli, or an app, or a webapp on top of
• servers are "dumb" nostr relays, don't need to change anything to start using it today

cons:

• this proposal pushes a lot of work onto the clients (including key rotation and double-verifications), which could be expensive

it seems to be working so far. code isn't complete (indeed, i dont handle owner transfers properly, or even creation the way the nip says), but you can build the app right now off main and start chatting and the security properties are the same.

[staab]

This is a very well written nip, but it states:

This is the easiest way to allow the use of group chat with select group of people, while hiding metadata.

Which I don't think is true. I recently wrote a note about this topic, which you can read here. It shares the gift-wrap approach, but makes different tradeoffs.

The main trade-off of my idea is that it relies on an access-controlled nsec bunker rather than a private key shared directly over the network. I tend to think that's an improvement, because you get a lot more flexibility and control, not to mention consistency, without making clients do super complicated things to get the right channel metadata.

My main point here though is that my approach basically creates a generic cryptographic subset of the entire nostr protocol using gift wrap, rather than specifying each individual component independently. Private chat is an obvious start, but with this PR every time a feature is added to chat, two NIPs need to be updated. Likewise, every time we want encryption for some other part of the protocol (e.g. kind 1s), a new NIP needs to be added.

I would strongly prefer to see a new NIP that specifies a generic way to 1. define channels based on a shared public key, 2. communicate over gift-wrapped channels, and 3. specifies multiple ways to manage the key rotation (nsec bunker and DM'd private keys are both viable alternatives, as well as FROSTR potentially).

[earonesty]

I would strongly prefer to see a new NIP that specifies a generic way to 1. define channels based on a shared public key, 2. communicate over gift-wrapped channels, and 3. specifies multiple ways to manage the key rotation (nsec bunker and DM'd private keys are both viable alternatives, as well as FROSTR potentially).

genuinely confused. how does this not satisfy 1 and 2? "bunker" based approach can be used if you simply don't invite people. you can make an invitation to the bunker instead. probably it's not more secure though, also not decentralized. also requires an always-on, non-nostr server. the purpose of this protocol is to be a pure nostr encrypted chat that satisfies all the needs of people who want encrypted chat without having to change relay server requirements

[staab]

"bunker" based approach can be used if you simply don't invite people. you can make an invitation to the bunker instead.

Yes, I just think the authentication method should be decoupled from the data structure so you can more easily choose your solution.

probably it's not more secure though, also not decentralized.

Also, true, but more flexible and simpler. It doesn't make sense to me to re-implement centralized administration features on top of a decentralized, eventually consistent network.

My main point was the "a generic way" bit, which probably should have been its own bullet point. This NIP solves chat, but it doesn't solve private kind 1 for example, or anything else. I think we should start by creating a way to generically create "groups" within which a nostr sub-network can be run. That automatically solves groups, along with many other problems. Again, I know the nsec bunker vs in-protocol key management is contentious, specifying the auth strategy when creating the group would be a good way to decouple and keep possibilities open for future improvements. I will try to create a NIP later this week so I can explain myself more concretely.

[earonesty]

#468 does what you want. it does solve private kind 1.

you can use the groups in this NIP for for non-chat group applications. like diagon alley events.

it's just a group of people that know the same private key. you can send anything you want.

because #468 allows for any kind to be sent. all you have to do is get rid of the "send chat message" - and replace it with wrapped-kind-1 instead of wrapped-kind-403. see? then you can just say "wrapped-kind-X" is a generic "Send to group anything you want"

but @fiatjaf chimed in on this last week in another message. He thought "kind-1-delivered-to-a-group" was more confusing than just using a new number for group-events, and that it was better to simply invent a new kind for every "encrypted group thing" or "encrypted direct thing" and then reference the nonencrypted version if you want to stay in sync.

because you still want a different UX for "kind 1 delivered encrypted just to me" and "kind 1 delivered to an encrypted group"

[staab]

Awesome, I see, so apart from the nsec bunker vs in-protocol group management tradeoff (which we agree can coexist), we are on the same page. To make things clearer, I'd suggest removing the term "chat" entirely from this PR (except in the motivation/use cases section), and defining the term "channel" or "group" at the top of the NIP as something like the nostr subnet concept. I can then create my own PR for nsec-bunker-controlled channels at some future date.

[earonesty]

that sounds reasonable, thanks.

[staab]

Ok, now that we have that figured out, let me propose a slightly different design. Instead of following the NIP 40 channel definition model, what about overloading pubkeys? IOW if a pubkey represents an "entity" rather than a "person", you could create a "channel" by publishing a kind 0 (for public metadata), or a gift-wrapped kind 0 (for private metadata). Then all these 4xx events would be used to extend/revoke the ability for channel "members" to publish data on behalf of/to the group. This seems simpler, since you need a shared key anyway, you might as well refer to self rather than a sort of arbitrary event id.

The same NIP could also be used to implement key rotation for regular accounts. I don't think it would increase the attack surface area for rogue group members by widening the scope of published events to public, since they have the private key anyway in your scenario and could publish weird public claims on behalf of the whole group.

One other small thing, might 405/406 be expanded to support multiple admins?

[earonesty]

Ok, now that we have that figured out, let me propose a slightly different design. Instead of following the NIP 40 channel definition model, what about overloading pubkeys? IOW if a pubkey represents an "entity" rather than a "person", you could create a "channel" by publishing a kind 0 (for public metadata), or a gift-wrapped kind 0 (for private metadata). Then all these 4xx events would be used to extend/revoke the ability for channel "members" to publish data on behalf of/to the group. This seems simpler, since you need a shared key anyway, you might as well refer to self rather than a sort of arbitrary event id.

what key would you encrypt with? still a shared key? actually, that's how i built it in the first place, there is no "create" event in arclib - just invitations with metadata and a privkey. but there's no difference because any time you add/remove members, you need a new "entity" (kind 0), and you wind up pointing to a bunch of those to get a channel history. im ok with either, but....

storing encrypted metadata in kind0 is weird, imo. i think @fiatjaf wouldn't like it from other convos

also by wrapping everything and never using kind0, you get a bigger anonset. you wind up with a lot of symmetry in the protocol, lots of reusable code (everything is wrapped, the server knows nothing about the group)

One other small thing, might 405/406 be expanded to support multiple admins?

you can delegate multiple "p" tags in a delegation event. but then you have to worry about forks (if 2 admins revoke around the same time, you can fork the group). can add moderation too, by giving people the wrong privkey in an invite... but the right pubkey for the channel, and then reposting their rewrapped-events after moderation happens.

would like to point out this works very well, and people using it see snappy performance and a nice experience. working on push notifications... no way to know what event was pushed, but if you tell our service a pubkey, we'll tell you're phone

[staab]

Yeah, you would just do symmetric encryption using the group's key.

any time you add/remove members

I see what you mean. The approach I propose would make the nsec bunker approach simpler, but wouldn't help at all with the in-protocol approach since you have to rotate keys anyhow.

Thanks for working through my questions, I believe I can retrofit a custodial key approach onto either this nip or the gift wrap one directly if I end up going that direction.

[staab]

Some fairly minor suggestions to try to make things clearer. Other thoughts:

• What about creating two new key pairs? One for group moderators, and one for group members. Posting about the group would be done by keypair A, posting to the group would be done by keypair B. This would make it easier to ignore malicious events.

[fyookball]

What's the deal with event deletion in this context? I think the owner of a channel should be able to delete events. Why not extend nip-09 so that channel owner pubkey is additionally valid to delete any event in the channel, not just the author of the event.

[earonesty]

one cool thing i realized with this is that the relay url is also hidden

[earonesty]

What's the deal with event deletion in this context? I think the owner of a channel should be able to delete events. Why not extend nip-09 so that channel owner pubkey is additionally valid to delete any event in the channel, not just the author of the event.

if the author wants to be able to delete an event at the relay, then the author needs to use a "derived key" instead of an ephemeral key. we can write-up what this means, but consider something very simple:

ephemeral key = derived key = event hash * private key mod curve.n

this then allows the author to delete an event later (at the relay level!). the key is no longer "ephemeral"

i would be unwilling to put this in the spec without at least one other cryptographer chiming in

[staab]

Suggested change

	## Kind 1059: Encrypted Gift Wrap
	For all encrypted group events, the client first generates the approrpiate kind 4XX message, as below, and signs it with the user's identity key.
	Then they generate a new public/private keypair, and uses this to sign new kind 1059 message.
	The `content` of that message is a [NIP-44](https://github.com/nostr-protocol/nips/pull/574) encrypted JSON string.
	The content is encrypted using the public key of the destination (the chat room public key, or the invitee public key.
	All events called "wrapped-kind XX" refer to these kind 1059 gift wrapped events.
	For more information on gift-wrapping, see: [NIP-59](https://github.com/nostr-protocol/nips/pull/468)
	## Kind 1059: Encrypted Gift Wrap
	All events called "wrapped-kind XX" refer to kind `1059` gift wrapped events, using an ephemeral key for signing, and a known public key for the recipient (a chat room public key, or invitee public key depending on the event kind).
	For more information on gift-wrapping, see: [NIP-59](https://github.com/nostr-protocol/nips/pull/468)

[fyookball]

@earonesty I don't really understand your logic regarding deletion. Why should the author have to use a derived key or do anything special? I'm suggesting that the standard practice be that the room creator should have the power to delete events. If this is something that makes sense, then we don't necessarily need extra logic -- just specify in the nip that the creator of the room pubkey can delete events that are part of the channel.

[arthurfranca]

@fyookball The relay does not know that the inner event (gift-wrapped) is a chat message. So you can't edit NIP-09 (event deletion) the way you want.

There are only 21 solutions to this:

1. @earonesty way of deriving each time the gift-wrap key from the shared channel keyauthor key + inner event id
   2) Using my proposal of the ! tag by editing NIP-09 slightly

[fyookball]

@arthurfranca @earonesty i'm fine with the deletion being done client-side. As long as its nip-standardized it will work across any clients who implement. It doesn't have to be strict, but if it's there, it will improve the user experience.

[arthurfranca]

@earonesty taking advantage of your encryption knowledge, is there a way any group member could derive a pubkey from the channel owner's pubkey (+ shared group key? or kind 403 event id? or kind 403 event hash? or channel create event id? or channel create hash) in a way that the channel owner with its privkey could derive the corresponging privkey? Sorry that I have no idea on this.

If it was possible, the kind 403 wrapping events could have:

{
  ...,
  tags: [
    ["!", "<channel owner derived pubkey>"], <-- is this possible?
    ...
  ]
}

If possible, the ! tag derived value would hide the real channel owner pubkey while allowing the channel owner to delete kind 403 channel messages by signing kind 9 (updated with this relay rule) events using corresponding privkey.

Currently the channel owner lacks ability to delete kind 403 messages (on relays, not client-side).

[earonesty]

the only way a user can delete a message is if they prove they own the key that produced the message. this can be done deriving keys

[arthurfranca]

Today yes, but look at the ! tag proposal I linked on comment above. That's why I asked if such a keypair could be derived so to be used as ! tag.

[fyookball]

maybe there can be a boolean flag whether to allow owner to delete events. So you could have moderated channels like Telegram and open channels like Reddit without centralized moderation.

[rabble]

My reading says that there can be one and only one group owner?

[earonesty]

My reading says that there can be one and only one group owner?

that's the current construction. technically there's nothing stopping you from delegating to moderators or multiple owners with a group-update message. they're just tags in the message that have to chain back the the creator.

can just update the nip to include all that or release as-is, and wait for PR's from people who actually have experiences that require these changes.

[vishalxl]

The proposal does not seem to have a 'member's list'. One result of that would be that the members would not 'know', or the clients will not be able to find out, whether the messages that are being received for the group are from invited members, or whether some interloper is also sending messages ( by getting the shared secret somehow). I think for this kind of sanity check, a membership-list is desirable; NIP 38 has such a list, and I find that this kind of list will help people only interact with members who are invited, and will reduce bad-use cases of interlopers using group etc.

A 'members list' will also improve UI, in that the clients can show who the members of a group are.

So my suggestion would be that only the admin should send invites, and at that time also update the members list. So when user is added to group ( by admin), then the members list is updated in kind 402, etc.

[fyookball]

@vishalxl I don't like the membership-list idea because it's not going to be accurate. Being in the list is no guarantee those people have even viewed the channel once. It wouldn't be so bad if it was a one-time thing, but the overhead of continually updating a list with poor accuracy doesn't seem great.

[d-krause]

I see the following issues with this NIP Pull Request.

1. Channels do not need so many new event kinds. Fewer kinds make for simpler implementation.

   • Since events are replaceable, one does not need all of "400: create channel", "402: change channel", and "404: move channel" kinds.

   • Kind 400 could simply become "channel metadata". 402 & 404 can be handled by ONE kind with all the required channel metadata properties that verifies all of the properties of these "kinds" as simply properties of the channel metadata itself.

   • Imagine one channel metadata kind 400 that had properties like :

     {
        "name": "",
        "about": "",
        "picture": "",
        "owner": "",
        "ownerSig": "",
        "replacedBy": "", 
        "moderators": []
     }
     

   • Do we really need "405: delegate new owner" and _"406: accept owner delegation" kinds? Isn't enough to "move" the channel to a new channel using the "replacedBy" attribute (or "movedTo" ) in the channel metadata? Less moving parts means easier to implement and understand.

   • I introduced a "moderators" array above as food for thought only. This could be a simple list of pubkeys that can make moderator metadata. This metadata could be a new kind that limits posts (either by poster pubkey, or whatever rules you want to implement.)

2. The event kind 403 "send encrypted chat message" is not needed. An Encrypted chat message should simply be a giftwrapped kind of the kind of event desired. If it is a text message, then use kind 1, a reaction? then kind 7, an article? then kind 30023. Let the Gift wrap do the work it is intended to do. We don't need new kinds to do this.

3. All events should be deleteable by their creator. There is no reason why any replaceable event should not be deleteable. It is Kind 5 - just gift wrap it and post the event on the channel. Working Demo here https://nostr-nip76.web.app/private-channels

4. Each event should use a unique pubkey and separate unique encryption key.

   • By changing the author pubkey and encryption key on each event, we make it nearly impossible for outsiders to breach the entire channel if any message is compromised. We also make it exponentially more difficult for outsiders to make targeted eavesdrops on channels without having the channels keys.

   • I had previous pull requests for NIP 76 (here and here) that demonstrated with actual functioning code that this is not only very possible, but very powerful. While not all parts of NIP 76 may be worth keeping, I believe this aspect of ephemeral pubkeys and encryption keys is worthy of careful consideration.

   • The NIP 76 wording was a bit cumbersome, so please consider it in these simplest of terms:

   • A Private Channel in NIP-76 is created from 2 extended public keys - the signingKey and the encryptKey. An event on the channel chooses the event keys where

     • authorPubKey = (userPrivateKey + signingKey.ChainCode)/create_date,
     • encryptionkey = encryptKey/create_date.
     • Channel events are identified by a derived #e tag that is solely an unintelligent identifier.

   • An approach like this results in events that cannot be tracked by any third party, and even if one event is compromised, others will not be at risk. Working Demo here https://nostr-nip76.web.app/private-channels

[earonesty]

I see the following issues with this NIP Pull Request.

1. Channels do not need so many new event kinds. Fewer kinds make for simpler implementation.

agreed, simpler is better

• Since events are replaceable, one does not need all of "400: create channel", "402: change channel", and "404: move channel" kinds.
• Kind 400 could simply become "channel metadata". 402 & 404 can be handled by ONE kind with all the required channel metadata properties that verifies all of the properties of these "kinds" as simply properties of the channel metadata itself.
• Imagine one channel metadata kind 400 that had properties like :

  {
     "name": "",
     "about": "",
     "picture": "",
     "owner": "",
     "replacedBy": "", 
     "moderators": []
  }
  

agree. ownersig not needed (handled with giftwrap). not sure camel is consistent with other nips, but i agree.

• Do we really need "405: delegate new owner" and _"406: accept owner delegation" kinds? Isn't enough to "move" the channel to a new channel using the "replacedBy" attribute (or "movedTo" ) in the channel metadata? Less moving parts means easier to implement and understand.

yes, we can do that, replaced-by can just point to the new owner's channel

• I introduced a "moderators" array above as food for thought only. This could be a simple list of pubkeys that can make moderator metadata. This metadata could be a new kind that limits posts (either by poster pubkey, or whatever rules you want to implement.)

moderator is a nice feature. list of pubkeys

2. The event kind 403 "send encrypted chat message" is not needed. An Encrypted chat message should simply be a giftwrapped kind of the kind of event desired. If it is a text message, then use kind 1, a reaction? then kind 7, an article? then kind 30023. Let the Gift wrap do the work it is intended to do. We don't need new kinds to do this.

agreed. kind 1 is fine

3. All events should be deleteable by their creator. There is no reason why any replaceable event should not be deleteable. It is Kind 5 - just gift wrap it and post the event on the channel. Working Demo here nostr-nip76.web.app/private-channels

this is not possible. the relay cannot verify deletion without the sender remembering the key. you can have the client "agree to delete it", but you cannot have the relay delete it unless the sender uses a tweaked "sending key" and then can delete later. i am implementing this. sender will not use emphemeral key, an will instead use tweaked (private key * event id)... so we can delete for real.

4. Each event should use a unique pubkey and separate unique encryption key.

agreed. that's baked in to the gift-wrap. no need to change it.

• By changing the author pubkey and encryption key on each event, we make it nearly impossible for outsiders to breach the entire channel if any message is compromised. We also make it exponentially more difficult for outsiders to make targeted eavesdrops on channels without having the channels keys.

again, that's baked into gift wrap, all public keys are unique for each message.

[earonesty]

• Do we really need "405: delegate new owner" and _"406: accept owner delegation" kinds? Isn't enough to "move" the channel to a new channel using the "replacedBy" attribute (or "movedTo" ) in the channel metadata? Less moving parts means easier to implement and understand.

yes, we can do that, replaced-by can just point to the new owner's channel

but the new owner needs to say { "replaces": "original channel id" }

that way they are consenting up front

doing it without consent is a recipe for problems.

... but then it creates a weird UX. the new owners needs to create a group "as a replacement" first. then the new owner needs to tell the old owner about it. then the old owner can delegate.

this is reminding me of why i added those messages in the first place.

it's a better flow:

• invite new owner
• new owner sees the message
• new owner clicks "ok" to consent