DSNP spec #176853320

pages/DSNP/DSNP-Message-Serialization.md

@@ -0,0 +1,85 @@
+# DSNP Message Serialization
+## Specification Status
+
+| Version | Status |
+---------- | ---------
+| 0.1     | Proposed |
+
+## Purpose
+1. Describe the form and method of DSNP message serialization.
+1. Facilitate use of SDK
+
+## Assumptions
+* All assumptions from [DSNP Messages](/DSNP/DSNP-Messages)
+* Content may be compressed where feasible
+* Serialization is in JSON (highly subject to change)
+
+### Note on Serialization method
+At the time of this writing, we are using JSON serialization, however, the following other formats are being seriously considered:
+
+* compressed JSON
+* [RLP (Recursive Length Prefix)](https://eth.wiki/en/fundamentals/rlp)
+* [eth2 serialization](https://ethresear.ch/t/blob-serialisation/1705), which is a proposal that is planned to replace RLP.
+* [CBOR (Concise Binary Object Representation)](https://en.wikipedia.org/wiki/CBOR)
+
+## Serializing Identifiable Messages
+Identifiable messages are those with a DSNP Message Type other than `Private`.  They may still be private messages, but it's known what _type_ of message it is.
+
+1. Serialize the message, as with this GraphChange, which is a follow of address `0x3c0ffee5`:
+    ```json
+    {
+        "address": "0x3c0ffee5",
+        "actionType": 0
+    }
+    ```
+1. Post the log event with parameters `topic`, `dsnpType`, and `dsnpData`:
+    ```javascript
+    DSNPMessage("21734c56962501651bc24175682569d541893b6b320e149bacf1141de937dad7",
+                1,
+                '{"address":"0x3c0ffee5","actionType":0')
+    ```
+
+
+## Serializing Private Messages
+If it's not desired to provide the message type, use the Private message type value, and provide the actual message type in the data format. The types of messages that might use the `Private` type are:
+
+* GraphChange
+* Profile  
+* Inbox
+* Broadcast
+
+Rather than using one of the private/encrypted DSNP message types, instead wrap the serialized data and include the public message type, as in this uncompressed example for a Profile change message:
+
+```json
+{
+  "dsnpType": 3,
+  "dsnpData": {
+    "uri": "http://www.placekitten.com/400/600",
+    "messageID": "0x8c83cac8226fb6cbb05755139ee361b37554553e1851e0f2f3327ee97e26219f"
+  }
+}
+```
+
+The messageID here is the keccak-256 hash of what is supposed to be at `uri`, for validation purposes. 
+
+This JSON object would then be compressed and encrypted, then added to a Private type DSNP message. The hash supplied for the actual DSNP message is the hash of the encrypted compressed data.
+
+The DSNP message object would look something like this:
+```json
+{
+  "dsnpType": 0,
+  "messageID": "0x4dda635953920eb0b1e3725a084fce713458bcfd9d75bf43bfbb96443680628c",
+  "dsnpData": "GyA5NTRM34AqyNc/bK9YAoLGBVFMac4FR92xtxyK1SacVg6lICgZ82uSXAYcHHrkfDN+douVwQDVtAkbABHg0g=="
+}
+``` 
+
+The messageID in the Private message is the hash of the dsnpData field.
+
+Thus the decrypted message data is nothing more than a Profile message wrapped in an object, plus a message type.
+
+**NOTE:** Dead drop messages (Drop), cannot be made private this way. The dead drop ID, which is a necessary shared secret for decryption, would not be available.
+
+### Hash validity
+Where ever there is a messageID, as stated earlier it means a keccak-256 hash of the content at the referenced URI. This hash cannot be checked at the contract level for validity, in the sense of being actually the hash of the uri content, however it is not allowed to be either zero length or 256 zeroes. 
+
+The one exception would be if is decided that Replies are separate types from other Broadcast messages; in such case an "original post" is one that has a "zero hash" for its inReplyTo field.

pages/DSNP/DSNP-Message-Types.md

@@ -0,0 +1,40 @@
+# DSNP Message Types
+## Specification Status
+
+| Version | Status |
+---------- | ---------
+| 0.1     | Proposed |
+
+## Purpose
+1. List valid message types and their [DSNP Message](/DSNP/DSNP-Messages) type assigned enumeration
+1. Facilitate use of SDK 
+1. Facilitate interpretation of on-chain data and message results
+1. Estimate data size
+
+
+## Assumptions
+* All assumptions from [DSNP Messages](/DSNP/DSNP-Messages)
+* Message types are preset, but the DSNP Message contract will be upgradeable to allow more types.
+* Message type values outside the contract presets are invalid
+
+| name     | description | value |
+|-------    |-------------| ----|
+| Private | message and message type are private; message data is encrypted | 0 |
+| GraphChange | follow or unfollow an account | 1 |
+| Broadcast | a public post or reply | 2 |
+| Profile | a Profile change | 3 |
+| KeyList | KeyList rotation | 4 |
+| PrivateGraphKeylist | PrivateGraph keylist rotation | 5 |
+| EncryptionKeyList | Encryption keyList rotation | 6 |
+| Reaction | a public visual reply to a Broadcast | 7 |
+| PrivateGraphChange | an encrypted follow or unfollow | 8
+| Drop | a dead drop message | 9
+| EncryptedInbox | an encrypted direct message | 10
+| PrivateBroadcast | an encrypted broadcast | 11
+
+## Private Messages
+If it's not desired to specify a message type, use the `Private` value as the message type.
+
+Note that a Drop, i.e. a dead drop message cannot be made private with this message, since the dead drop ID, which is a necessary shared secret for decryption, would not be available.
+
+See [DSNP Message Serialization](/DSNP/DSNP-Message-Serialization) for more detail

pages/DSNP/DSNP-Messages.md

@@ -0,0 +1,190 @@
+# DSNP Messages
+## Specification Status
+
+| Version | Status |
+---------- | ---------
+| 0.1     | Proposed |
+
+## Purpose
+1. Describe the form and content of DSNP Messages posted to the blockchain used for all Liberty Platform activities. Only some of these activities will have the full message posted to chain. Examples:
+    * Public messages (profile changes, public posts, reactions)
+    * Direct messages between accounts (dead drop, inbox)
+    * Key changes (private graph keylist, encryption keylist, public key rotation)
+    * Graph changes (follow/unfollow)
+1. Specify an on-chain message format
+1. Provide data size estimations
+1. Facilitate use of SDK and interpretation of on-chain data
+
+## Assumptions
+* Chain messages are on Ethereum
+* Message data is posted via [Ethereum log events](https://medium.com/mycrypto/understanding-event-logs-on-the-ethereum-blockchain-f4ae7ba50378)
+* Signature algorithm is [secp256k1](https://en.bitcoin.it/wiki/Secp256k1). This allows the use `ecreover` 
+  to get public keys. A public key also need not be included in a log event for ease of validation.
+* content hashes are created via the same [keccak-256 hashing algorithm](https://en.wikipedia.org/wiki/SHA-3) used by Ethereum.  
+
+## DSNP Message Formats
+We have seriously considered two possibilities, a [variable message format](#Variable-Message-Format), and a [unified message format](#unified-message-format).  Others are listed at the end of this page.
+
+###  Variable Message Format
+This format is the current preference.
+
+* All actions are posted to chain with some or all pertinent information about the action
+* Different information is posted depending on the action.
+* **Advantages**
+    * can index all actions without requesting off-chain data
+    * can validate most actions without requesting off-chain data
+    * can archive some actions without requesting off-chain data
+* **Disadvantages**
+    * more data (likely more costly up front) than a simple URI
+
+**Log message format ** 
+
+This is what would be posted as a Log event in Ethereum:
+
+| field | description | type |
+|-------|-------------|------|
+| topic | Ethereum log topic | bytes|
+| fromAddress | social identity |bytes |
+| dsnpType | DSNP message type |number/enum |
+| dsnpData | serialized, possibly compressed message data| bytes |
+
+#### Some other options:
+* Emit no topic, have a single contract that subscribers watch for events from.  Subscribers can perform filtering based on the `dsnpTopic` field.
+* The topic is the dsnpType (possibly not an enum).  Subscribers could listen for desired topics.
+
+### DSNP Messages
+These messages would be serialized, compressed where feasible, and emitted in the log event as the `DSNPData` field.
+
+For details on how messages are serialized, see [DSNP Message Serialization](/DSNP/DSNP-Message-Serialization)
+
+#### Broadcast
+a public post (was Announcement)
+
+| field     | description | type |
+|-------    |-------------| ----|
+| inReplyTo | messageID replied to | bytes32
+| messageID | keccak-256 hash of content stored at uri |  bytes32
+| uri       | content uri | bytes
+
+
+**NOTE** If origin broadcasts and replies stay the same type, inReplyTo is allowed to be blank.  If replies are separated into their own type, the inReplyTo field here will be dropped.
+
+#### Drop
+a dead drop message
+
+| field | description | type
+|-------|-------------| ---|
+| ddid | dead drop id |  bytes
+| uri  | content uri  |  bytes
+| messageID | keccak-256 hash of content |  bytes32
+
+#### GraphChange 
+a public follow/unfollow
+
+| field | description | type
+|-------|-------------| ---|
+|address  | social identity|  bytes
+|actionType | follow/unfollow| number/enum
+
+#### KeyList, PrivateGraphKeyList, EncryptionKeyList
+
+a keylist rotation
+
+| field | description | type
+|-------|-------------| ---|
+|keylist | new list of valid keys | [bytes]
+
+#### Inbox
+a direct message
+
+| field | description | type
+|-------|-------------| ---|
+|messageID | keccak-256 hash of content | bytes32
+|uri  | content uri  | bytes
+
+#### EncryptedInbox
+an encrypted direct message.  This describes the format once decrypted.  Possibly combine both of these and expect that all Inbox messages are encrypted.
+
+| field | description | type
+|-------|-------------| ---|
+|messageID | keccak-256 hash of content | bytes32
+|uri  | content uri  | bytes
+
+#### Reaction
+a visual reply to a post
+
+| field | description | type
+|-------|-------------| ---|
+|emoji | the encoded reaction  | number
+|inReplyTo | messageID the reaction is for |  bytes32
+
+### Possible Message Types
+
+#### Profile
+a profile update such as name or icon change
+
+| field | description | type
+|-------|-------------| ---|
+|uri    | uri for the profile data  |bytes
+| messageID |  keccak-256 hash of content at uri | bytes32
+
+
+#### Private
+An encrypted message of unknown type. See [DSNP Message Types: Private Messages](/DSNP/DSNP-Message-Types#private-messages) for details.
+
+| field | description | type
+|-------|-------------| ---|
+| data | encrypted graph change data | bytes
+| messageID | keccak-256 hash of unencrypted content | bytes32
+
+#### PrivateBroadcast
+An encrypted Broadcast decipherable by specific accounts . This describes the format once decrypted.
+
+| field     | description | type |
+|-------    |-------------| ----|
+| inReplyTo | messageID replied to | bytes32
+| messageID      | keccak-256 hash of content stored at URI |  bytes32
+| uri       | content uri | bytes
+
+#### Reply
+This is a message type that would allow a Broadcast to drop the `inReplyTo` field.  In such case a Reply is exactly like Broadcast above, but `inReplyTo` is not allowed to be blank.
+
+
+### Unified Message Format
+This is currently not the recommended solution, but is presented as a comparison.
+
+* All actions are posted to the chain with some pertinent information about the action
+* The same information is posted regardless of action
+* The rest of the information for the action is stored off chain.
+* **Advantages**
+    * less data on chain --> lower up-front costs and lower node storage requirements
+    * privacy management is easier -- for example, "right to be forgotten" is easier to comply with since 
+        data is external, off-chain.
+* **Disadvantages**
+    * indexing requires retrieving content @ uri
+    * archiving requires retrieving more content @ uri
+    * validation requires retrieving more content @ uri
+    * cost savings may be minimal
+
+| field | description | type
+|-------|-------------| ---|
+| topic | Ethereum log topic |  bytes
+| action type | the type of action | bytes
+| fromAddress | social identity | bytes
+| uri | uri of stored action information | bytes
+
+### All data on chain
+One possibility is not to have any data stored off-chain; instead, even the ActivityPub content would be posted to chain. 
+The disadvantages far outweigh the advantages:
+
+* **Advantages**
+    * Validation, indexing, discovery are much easier
+* **Disadvantages**
+    * This amount of data would rapidly slow down the network
+    * The posting of illegal content could potentially shut down the network
+    * Garbage collection, validation, privacy concerns, and dealing with illegal content become interdependent, and would pose conflicting interests.
+    * Would still need archive to store at least some content
+    * Unknown, likely chilling effect on incentive models
+
+### No data except hashes on chain
+Only hashes of the events are stored on chain; everything else is interpreted via API(s)

sidebar.json

@@ -2,6 +2,35 @@
     "path": "pages",
     "name": "pages",
     "children": [
+        {
+            "path": "pages/DSNP",
+            "name": "DSNP",
+            "children": [
+                {
+                    "path": "pages/DSNP/DSNP-Message-Serialization.md",
+                    "name": "DSNP-Message-Serialization.md",
+                    "meta": {},
+                    "title": "DSNP Message Serialization",
+                    "url": "/DSNP/DSNP-Message-Serialization"
+                },
+                {
+                    "path": "pages/DSNP/DSNP-Message-Types.md",
+                    "name": "DSNP-Message-Types.md",
+                    "meta": {},
+                    "title": "DSNP Message Types",
+                    "url": "/DSNP/DSNP-Message-Types"
+                },
+                {
+                    "path": "pages/DSNP/DSNP-Messages.md",
+                    "name": "DSNP-Messages.md",
+                    "meta": {},
+                    "title": "DSNP Messages",
+                    "url": "/DSNP/DSNP-Messages"
+                }
+            ],
+            "size": null,
+            "type": "directory"
+        },
         {
             "path": "pages/folder",
             "name": "folder",