Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[examples/eth-dm] Do not use JSON over the wire #214

Closed
D4nte opened this issue Jun 23, 2021 · 1 comment · Fixed by #228
Closed

[examples/eth-dm] Do not use JSON over the wire #214

D4nte opened this issue Jun 23, 2021 · 1 comment · Fixed by #228

Comments

@D4nte
Copy link
Contributor

D4nte commented Jun 23, 2021

Problem

JSON is not data efficient, see https://forum.vac.dev/t/eth-dm-learnings-and-what-is-next/84/2

Solution

Use compressed JSON instead. See #214 (comment)
Use protobuf.

@D4nte
Copy link
Contributor Author

D4nte commented Jun 30, 2021

There does not see to be a widely adopted standard for JSON compression.

There are a number of library that attempt to compress the JSON format to reduce data duplication:

Others propose to use a different format coupled with a generic compression algorithm:

Because there is no standard I can refer too, I do not think it would be a good idea to used compressed JSON.

However, the fact that JSON is a not a size efficient format, it would still be good to replace it with something else. Alternatives are:

In this occurrence, it may be best to just role with protobuf. We can review at a later stage if there are other format worth introducing.

@D4nte D4nte changed the title [examples/eth-dm] Use compressed JSON instead of JSON over the wire [examples/eth-dm] Do not use JSON over the wire Jul 1, 2021
D4nte added a commit that referenced this issue Jul 15, 2021
### Added
- `WakuRelay.deleteObserver` to allow removal of observers, useful when
  a React component add observers when mounting and needs to delete it
  when unmounting.
- Keep alive feature that pings host regularly, reducing the chance of
  connections being dropped due to idle.
  Can be disabled or default frequency (10s) can be changed when calling
  `Waku.create`.
- New `lib/utils` module for easy, dependency-less hex/bytes
  conversions.
- New `peers` and `randomPeer` methods on `WakuStore` and
  `WakuLightPush` to have a better idea of available peers;
  Note that it does not check whether Waku node is currently connected
  to said peers.
- Enable passing decryption private keys to `WakuStore.queryHistory`.
- Test: Introduce testing in browser environment (Chrome) using Karma.
- Add support for Waku Message version 1: Asymmetric encryption,
  symmetric encryption, and signature of the data.

### Changed
- **Breaking**: Auto select peer if none provided for store and light
  push protocols.
- Upgrade to `libp2p@0.31.7` and `libp2p-gossipsub@0.10.0` to avoid
  `TextEncoder` errors in ReactJS tests.
- Disable keep alive by default as latest nim-waku release does not
  support ping protocol.
- **Breaking**: Optional parameters for `WakuMessage.fromBytes` and
  `WakuMessage.fromUtf8String` are now passed in a single `Options`
   object.
- **Breaking**: `WakuMessage` static functions are now async to allow
  for encryption and decryption.
- **Breaking**: `WakuMessage` constructor is now private, `from*` and
  `decode*` function should be used.
- `WakuMessage` version 1 is partially supported, enabling asymmetrical
  encryption and signature of messages;
  this can be done by passing keys to `WakuMessage.from*` and
  `WakuMessage.decode*` methods.
- Examples (eth-dm): Use Waku Message version 1 encryption scheme
  instead of `eth-crypto`.
- Examples (eth-dm): Use Protobuf for direct messages instead of JSON
  ([#214](#214)).

### Fixed
- Disable `keepAlive` if set to `0`.
@D4nte D4nte mentioned this issue Jul 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant