feat(ETH transport & heartbeats): various enhancements/features #2058

onur-ozkan · 2024-01-23T20:04:43Z

ETH websocket transport
- komodo-defi-proxy signed messages implementation
- lock & wait free request pipelines for response quickness
- expirable hashmap implementation (so request contexts can be cleaned in REQUEST_TIMEOUT_AS_SEC)
Refactor web3 contexts, connection rotation and remove singular web3 field from ETH coin
~~[ ] Implementing and using fn with_socket for ETH COIN_BALANCE event~~ (3c2313a)
Heartbeats implementation for streaming channels.
Refactored node rotation (moved from transport to protocol level, so it can be rotated across different transports)
RPC abstraction for ETH

Signed-off-by: onur-ozkan <work@onurozkan.dev>

… into eth-ws-and-heartbeats

Signed-off-by: onur-ozkan <work@onurozkan.dev>

mariocynicys

Thanks!

Q: To get my thoughts straight, the websocket_transport.rs implementation is thread safe because we get a new id (hashmap key) each time thus we will never access (read/write) the same key from two threads at the same time?

mm2src/coins/eth/web3_transport/metamask_transport.rs

mm2src/coins/eth/v2_activation.rs

mm2src/coins/eth.rs

mariocynicys · 2024-02-22T11:12:02Z

mm2src/coins/eth/eth_rpc.rs

+                Web3Transport::Metamask(metamask) => metamask.execute(method, params.clone()),
+            };
+
+            match execute_fut.timeout(Duration::from_secs(15)).await {


Could you define this 15 literal as a const instead? Also, isn't 15 a very long time :o

Made the Duration constant altogether and used in EthCoin::get_live_client and EthCoin::try_rpc_send.

mariocynicys · 2024-02-22T12:08:25Z

mm2src/coins/eth/web3_transport/websocket_transport.rs

+                                let response_map = unsafe { &mut *self.responses.0 };
+                                let _ = response_map.insert(request_id, res_bytes);
+
+                                notifier.send(()).expect("receiver channel must be alive");


Won't this ever panic?

No. Caller is awaiting for the response through this channel. If it ever drop before that, we should panic as something def. going wrong.

ahaa I was thinking this was a notifier for external communication (not us).

mm2src/coins/eth/web3_transport/websocket_transport.rs

shamardy

🔥

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan · 2024-02-22T20:21:37Z

Q: To get my thoughts straight, the websocket_transport.rs implementation is thread safe because we get a new id (hashmap key) each time thus we will never access (read/write) the same key from two threads at the same time?

That's part of it. Each request utilizes an atomic ID and only makes modifications using that ID. Additionally, we are waiting for notification from another thread to read and remove the key, ensuring that we don't attempt to read and delete it while another thread is attempting to write to it. This sequential computing is ensured via the thread channel notification.

laruh

LGTM!

mariocynicys

This sequential computing is ensured via the thread channel notification.

I get the point in that we process all the responses in a single worker thread, but don't we ever access the map from another thread (even with a different key)?

e.g. are we always sure ...

mm2src/coins/eth/web3_transport/websocket_transport.rs

Signed-off-by: onur-ozkan <work@onurozkan.dev>

mariocynicys

LGTM, Thanks!

mm2src/coins/eth/web3_transport/websocket_transport.rs

Signed-off-by: onur-ozkan <work@onurozkan.dev>

* dev: feat(indexeddb): advanced cursor filtering impl (KomodoPlatform#2066) update dockerhub destination repository (KomodoPlatform#2082) feat(event streaming): configurable worker path, use SharedWorker (KomodoPlatform#2080) fix(hd_tests): fix test_hd_utxo_tx_history unit test (KomodoPlatform#2078) feat(network): improve efficiency of known peers handling (KomodoPlatform#2074) feat(nft): enable eth with non fungible tokens (KomodoPlatform#2049) feat(ETH transport & heartbeats): various enhancements/features (KomodoPlatform#2058)

onur-ozkan added enhancement New feature or request in progress Changes will be made from the author labels Jan 23, 2024

onur-ozkan force-pushed the eth-ws-and-heartbeats branch from fbf0dad to 4663949 Compare January 23, 2024 20:07

onur-ozkan changed the title ~~feat(transport, streaming channels): various features on ETH and streaming channels~~ feat(ETH transport & heartbeats): various enhancements/features Jan 23, 2024

onur-ozkan force-pushed the eth-ws-and-heartbeats branch from 08c44d5 to e39bede Compare January 24, 2024 12:49

shamardy mentioned this pull request Jan 25, 2024

feat(ETH): balance event streaming for ETH #2041

Merged

onur-ozkan added 4 commits January 26, 2024 16:10

implement bare types for websocket transport

f02fd9b

Signed-off-by: onur-ozkan <work@onurozkan.dev>

ws transport support and removal of eth::web3 field

7977a72

Signed-off-by: onur-ozkan <work@onurozkan.dev>

implement fn prepare for websocket_transport

2734cd1

Signed-off-by: onur-ozkan <work@onurozkan.dev>

save dev state

6973845

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan force-pushed the eth-ws-and-heartbeats branch from ba35834 to 6973845 Compare January 26, 2024 13:11

onur-ozkan added 8 commits January 29, 2024 14:58

partial design of handling request and socket events

37324bc

Signed-off-by: onur-ozkan <work@onurozkan.dev>

fix WASM errors

1de6a3f

Signed-off-by: onur-ozkan <work@onurozkan.dev>

implement keepalive for ws connections

b969898

Signed-off-by: onur-ozkan <work@onurozkan.dev>

save dev state

b6f7c6d

Signed-off-by: onur-ozkan <work@onurozkan.dev>

add TODO

4d02fb0

Signed-off-by: onur-ozkan <work@onurozkan.dev>

make the websocket transport working

d06c195

Signed-off-by: onur-ozkan <work@onurozkan.dev>

improve websocket_transport performance by a lot

059971e

Signed-off-by: onur-ozkan <work@onurozkan.dev>

add some doc-comments

d770af9

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan force-pushed the eth-ws-and-heartbeats branch from 59b78de to d770af9 Compare January 31, 2024 13:36

onur-ozkan added 10 commits January 31, 2024 16:52

make the connection loop continious

f89191e

Signed-off-by: onur-ozkan <work@onurozkan.dev>

implement memory deallication on map pointer

d336dbf

Signed-off-by: onur-ozkan <work@onurozkan.dev>

add todo note

73d71fb

Signed-off-by: onur-ozkan <work@onurozkan.dev>

implement ExpirableEntry common module

1e0cba4

Signed-off-by: onur-ozkan <work@onurozkan.dev>

fixed-TODO: clear disconnected/outdated channels

95152b5

Signed-off-by: onur-ozkan <work@onurozkan.dev>

fix comma

ca790a4

Signed-off-by: onur-ozkan <work@onurozkan.dev>

add client rotation on eth

648832c

Signed-off-by: onur-ozkan <work@onurozkan.dev>

add TODO task

b045e8d

Signed-off-by: onur-ozkan <work@onurozkan.dev>

unify with_polling and with_socket

3c2313a

Signed-off-by: onur-ozkan <work@onurozkan.dev>

use event handlers in websocket transport

277aa75

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan added 3 commits February 21, 2024 20:13

update outdated logs

6b7267c

Signed-off-by: onur-ozkan <work@onurozkan.dev>

Merge branch 'dev' of github.com:KomodoPlatform/komodo-defi-framework…

7107275

… into eth-ws-and-heartbeats

fix WASM error

df48248

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan force-pushed the eth-ws-and-heartbeats branch from b2409d2 to df48248 Compare February 21, 2024 17:55

shamardy mentioned this pull request Feb 22, 2024

Share EVM Web3 Instances for Platform Coin and Tokens to Reduce Open Websocket Connections #2071

Open

mariocynicys reviewed Feb 22, 2024

View reviewed changes

shamardy previously approved these changes Feb 22, 2024

View reviewed changes

apply various improvements

665dc2f

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan dismissed shamardy’s stale review via 665dc2f February 22, 2024 17:10

laruh previously approved these changes Feb 23, 2024

View reviewed changes

mariocynicys reviewed Feb 23, 2024

View reviewed changes

mm2src/coins/eth/web3_transport/websocket_transport.rs Outdated Show resolved Hide resolved

mm2src/coins/eth/web3_transport/websocket_transport.rs Outdated Show resolved Hide resolved

send responses through notifier channel

9de616c

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan dismissed laruh’s stale review via 9de616c February 26, 2024 10:10

onur-ozkan linked an issue Feb 26, 2024 that may be closed by this pull request

heartbeats for event stream channels #2042

Closed

mariocynicys previously approved these changes Feb 26, 2024

View reviewed changes

mm2src/coins/eth/web3_transport/websocket_transport.rs Outdated Show resolved Hide resolved

onur-ozkan dismissed mariocynicys’s stale review via 8200379 February 26, 2024 11:41

clear outdated entries before sending response

537a020

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan force-pushed the eth-ws-and-heartbeats branch from 8200379 to 537a020 Compare February 26, 2024 12:47

shamardy approved these changes Feb 26, 2024

View reviewed changes

shamardy merged commit 6aa5d66 into dev Feb 26, 2024
24 of 30 checks passed

shamardy deleted the eth-ws-and-heartbeats branch February 26, 2024 13:51

shamardy added 2.1.0-beta QA sec-review labels Feb 26, 2024

shamardy mentioned this pull request Feb 27, 2024

feat(nft): enable eth with non fungible tokens #2049

Merged

shamardy mentioned this pull request Mar 25, 2024

fix(eth): error handling in RPCs #2090

Merged

shamardy mentioned this pull request Jul 24, 2024

Add docs for Heartbeat events KomodoPlatform/komodo-docs-mdx#298

Closed

shamardy mentioned this pull request Aug 1, 2024

chore(release): v2.1.0-beta #2043

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(ETH transport & heartbeats): various enhancements/features #2058

feat(ETH transport & heartbeats): various enhancements/features #2058

onur-ozkan commented Jan 23, 2024 •

edited

Loading

mariocynicys left a comment

mariocynicys Feb 22, 2024

onur-ozkan Feb 22, 2024 •

edited

Loading

mariocynicys Feb 22, 2024

onur-ozkan Feb 22, 2024

mariocynicys Feb 23, 2024

shamardy left a comment

onur-ozkan commented Feb 22, 2024

laruh left a comment

mariocynicys left a comment

mariocynicys left a comment

feat(ETH transport & heartbeats): various enhancements/features #2058

feat(ETH transport & heartbeats): various enhancements/features #2058

Conversation

onur-ozkan commented Jan 23, 2024 • edited Loading

mariocynicys left a comment

Choose a reason for hiding this comment

mariocynicys Feb 22, 2024

Choose a reason for hiding this comment

onur-ozkan Feb 22, 2024 • edited Loading

Choose a reason for hiding this comment

mariocynicys Feb 22, 2024

Choose a reason for hiding this comment

onur-ozkan Feb 22, 2024

Choose a reason for hiding this comment

mariocynicys Feb 23, 2024

Choose a reason for hiding this comment

shamardy left a comment

Choose a reason for hiding this comment

onur-ozkan commented Feb 22, 2024

laruh left a comment

Choose a reason for hiding this comment

mariocynicys left a comment

Choose a reason for hiding this comment

mariocynicys left a comment

Choose a reason for hiding this comment

onur-ozkan commented Jan 23, 2024 •

edited

Loading

onur-ozkan Feb 22, 2024 •

edited

Loading