feat(event streaming): configurable worker path, use SharedWorker #2080

shamardy · 2024-03-05T21:19:25Z

This PR allows any worker path to be used instead of hardcoded worker.js file. The worker path can be specified using worker_path in event_stream_configuration, if not specified worker.js will be used as default. Example:

"event_stream_configuration": {
    "access_control_allow_origin": "*",
    "active_events": {
      "NETWORK": { "stream_interval_seconds": 1.5 }
    },
    "worker_path": "index.js" // This can be added in wasm only configuration
}

mm2src/mm2_event_stream/src/lib.rs

onur-ozkan · 2024-03-05T21:25:17Z

mm2src/mm2_net/src/wasm_event_stream.rs

@@ -17,7 +19,8 @@ pub async fn handle_worker_stream(ctx: MmArc) {
            "message": event.message(),
        });

-        let worker = web_sys::Worker::new("worker.js").expect("Missing worker.js");
+        let worker_path = config.worker_path.as_deref().unwrap_or(DEFAULT_WORKER_PATH);
+        let worker = web_sys::Worker::new(worker_path).expect("Missing worker.js");


It might not be "worker.js".

Was already fixing this. Done :)

we might have another worker.js that serves a diff purpose In the future so why not make it balance_streaming_worker.js as default ?

Also, worker.js is a very basic worker that's not doing much and we might have a more complex worker in the future

I left it as worker.js for backward compatibility with this https://github.com/KomodoPlatform/react-komodefi-wasm/blob/streaming/worker.js ref #1978 (comment)

I can make it balance_streaming_worker.js if agreed by @onur-ozkan and they would have to fix it in the react-komodefi-wasm branch

Also, worker.js is a very basic worker that's doing much and we might have a more complex worker in the future

Can you please elaborate on this @borngraced ?

I believe we should get some feedback from the frontend devs. They may prefer to handle all the events from single worker file by filtering the incoming data.

It's still a single worker file for event streaming

If they requested multiple workers in the future we can do it in another PR.

the current worker script used for balance streaming is only logging the messages to the console..

It's just for demonstration, it doesn't have to be that way obviously.

You can handle all kinds of operations from a single worker file by filtering the incoming payload. As I mentioned, this preference should be defined by the frontend developers not by us as this affects their project structure directly.

It's still a single worker file for event streaming

I am aware of it, I meant event streaming + different tasks on workers

I am aware of it, I meant event streaming + different tasks on workers

In this case they can change the config I guess.

shamardy · 2024-03-05T22:01:33Z

@smk762 @gcharang after this is merged to dev, worker.js here https://github.com/KomodoPlatform/react-komodefi-wasm/blob/streaming/worker.js should be renamed to event_streaming_worker.js

…e it once

…rializing `EventStreamConfiguration`

shamardy · 2024-03-07T11:59:33Z

@smk762 @gcharang I changed the worker type to be a shared worker, so the script in event_streaming_worker.js should be changed to the following

// use chrome://inspect in Chrome to debug the worker
// use about:debugging in Firefox to debug the worker
self.onconnect = function(event) {
    var port = event.ports[0];

    console.log('A new connection to the shared worker has been successfully established.');

    port.onmessage = function(e) {
        console.log(e.data);
    };
};

Shared workers debugging can be accessed from chrome://inspect in chrome or about:debugging in Firefox, from there you can navigate to shared workers and choose event_streaming_worker.js and see the console logs.

onur-ozkan

🔥

borngraced

according to this doc, and usage of shared worker..do you think we need it here just stick to dedicated worker without much stuffs going on in this code.

borngraced

LGTM 🔥

shamardy · 2024-03-07T14:03:27Z

according to this doc, and usage of shared worker..do you think we need it here just stick to dedicated worker without much stuffs going on in this code.

it was requested by @naezith since normal worker doesn't fit his needs in the flutter codebase. Plus, I think it's good to have event streaming worker shared and accessed from multiple contexts.

* dev: feat(indexeddb): advanced cursor filtering impl (KomodoPlatform#2066) update dockerhub destination repository (KomodoPlatform#2082) feat(event streaming): configurable worker path, use SharedWorker (KomodoPlatform#2080) fix(hd_tests): fix test_hd_utxo_tx_history unit test (KomodoPlatform#2078) feat(network): improve efficiency of known peers handling (KomodoPlatform#2074) feat(nft): enable eth with non fungible tokens (KomodoPlatform#2049) feat(ETH transport & heartbeats): various enhancements/features (KomodoPlatform#2058)

shamardy added 3 commits March 5, 2024 21:55

Allow worker script URL to be specified in configuration

e97d49c

fix wasm lint

44052d4

make worker_path part of EventStreamConfiguration

41c8a83

shamardy added the under review label Mar 5, 2024

shamardy requested a review from onur-ozkan March 5, 2024 21:19

onur-ozkan reviewed Mar 5, 2024

View reviewed changes

shamardy added 5 commits March 5, 2024 23:26

fix expect message to use work_path

f9ca486

Refactor worker_path to use PathBuf

0682dd1

fix wasm lint

0476609

add doc comment for worker_path

50e8f74

Rename DEFAULT_WORKER_PATH to event_streaming_worker.js

4955bf4

smk762 mentioned this pull request Mar 6, 2024

Rename worker.js (next API release) KomodoPlatform/react-komodefi-wasm#12

Open

borngraced previously approved these changes Mar 6, 2024

View reviewed changes

shamardy added in progress Changes will be made from the author and removed under review labels Mar 6, 2024

Use SharedWorker instead of Worker in WASM event stream and initializ…

d6120f3

…e it once

shamardy dismissed borngraced’s stale review via d6120f3 March 6, 2024 19:38

shamardy added 3 commits March 6, 2024 22:48

use default_worker_path if worker_path was not provided when dese…

a161ada

…rializing `EventStreamConfiguration`

update browser compatibility link

9bb1e8a

add browser compatibility link for SharedWorker

5f015f6

shamardy added under review and removed in progress Changes will be made from the author labels Mar 7, 2024

shamardy requested review from onur-ozkan and borngraced March 7, 2024 11:56

onur-ozkan approved these changes Mar 7, 2024

View reviewed changes

onur-ozkan added the enhancement New feature or request label Mar 7, 2024

borngraced reviewed Mar 7, 2024

View reviewed changes

borngraced approved these changes Mar 7, 2024

View reviewed changes

shamardy changed the title ~~feat(event streaming): make worker path configurable~~ feat(event streaming): make worker path configurable and use shared worker Mar 7, 2024

shamardy changed the title ~~feat(event streaming): make worker path configurable and use shared worker~~ feat(event streaming): make worker path configurable and use SharedWorker Mar 7, 2024

shamardy changed the title ~~feat(event streaming): make worker path configurable and use SharedWorker~~ feat(event streaming): configurable worker path configurable, use SharedWorker Mar 7, 2024

shamardy changed the title ~~feat(event streaming): configurable worker path configurable, use SharedWorker~~ feat(event streaming): configurable worker path, use SharedWorker Mar 7, 2024

shamardy merged commit 10fe7fa into dev Mar 7, 2024
29 of 37 checks passed

shamardy deleted the fix-config-worker-file branch March 7, 2024 14:08

shamardy mentioned this pull request Aug 1, 2024

chore(release): v2.1.0-beta #2043

Merged

smk762 mentioned this pull request Aug 1, 2024

Event stream config KomodoPlatform/komodo-docs-mdx#307

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(event streaming): configurable worker path, use SharedWorker #2080

feat(event streaming): configurable worker path, use SharedWorker #2080

shamardy commented Mar 5, 2024 •

edited

Loading

onur-ozkan Mar 5, 2024

shamardy Mar 5, 2024

borngraced Mar 5, 2024 •

edited

Loading

shamardy Mar 5, 2024 •

edited

Loading

shamardy Mar 5, 2024

shamardy Mar 5, 2024

shamardy Mar 5, 2024

onur-ozkan Mar 5, 2024

onur-ozkan Mar 5, 2024

shamardy Mar 5, 2024

shamardy commented Mar 5, 2024 •

edited

Loading

shamardy commented Mar 7, 2024

onur-ozkan left a comment

borngraced left a comment

borngraced left a comment

shamardy commented Mar 7, 2024 •

edited

Loading

feat(event streaming): configurable worker path, use SharedWorker #2080

feat(event streaming): configurable worker path, use SharedWorker #2080

Conversation

shamardy commented Mar 5, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

borngraced Mar 5, 2024 • edited Loading

Choose a reason for hiding this comment

shamardy Mar 5, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shamardy commented Mar 5, 2024 • edited Loading

shamardy commented Mar 7, 2024

onur-ozkan left a comment

Choose a reason for hiding this comment

borngraced left a comment

Choose a reason for hiding this comment

borngraced left a comment

Choose a reason for hiding this comment

shamardy commented Mar 7, 2024 • edited Loading

shamardy commented Mar 5, 2024 •

edited

Loading

borngraced Mar 5, 2024 •

edited

Loading

shamardy Mar 5, 2024 •

edited

Loading

shamardy commented Mar 5, 2024 •

edited

Loading

shamardy commented Mar 7, 2024 •

edited

Loading