-
Notifications
You must be signed in to change notification settings - Fork 2.6k
Pass an executor through the Configuration #4688
Conversation
client/service/src/lib.rs
Outdated
err | ||
); | ||
this.to_poll.push(Box::pin(err.into_future().compat().map(drop))); | ||
if let Some(tasks_executor) = this.tasks_executor.as_ref() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we still need this logic? The builder creates a thread builder anyway and browser gives us a spawner as well. This sounds like we can remove the option here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the user leaves None
in the Configuration
, we try to create a ThreadPool
and this is the path that is reached if spawning the threads failed.
In practice, without this code we will get a panic in the browser if the user forgets to tweak the configuration. I guess it's indeed debatable whether it's worth the additional code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should go the way of checking at Service
creation that a task_executor
is provided and otherwise an error is returned. I know that is not the best solution, but IMHO the best we can do for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed with @bkchr.
I'm trying of create a version of Polkadot using this branch to see if it indeed fixes the performance issue. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM apart from that one discussion.
We deployed this PR to both a Kusama validator and a Kusama sentry node, and CPU usage seems to be mostly the same if not lower than version 0.7.18 (that includes the revert to libp2p 0.13). |
@tomaka service tests needs to be updated. |
Hopefully fixes the performance issues that Polkadot was having.
One issue that we have in master right now is that all the tasks go through this path:
substrate/client/service/src/lib.rs
Line 334 in a90c72e
substrate/client/service/src/lib.rs
Lines 339 to 341 in a90c72e
In other words, everything is single-threaded (except the networking).
In this PR, we now pass a "tasks executor" through the
Configuration
struct.It is set up to be a tokio runtime, or the
spawn_local
function ofwasm_bindgen_futures
,or a threads pool by default, or local polling if all else failsEDIT: removed fallbacks after review.Important: this executor is stored within the
Service
, and all the components of Substrate should then use theService
(throughspawn_task
orspawn_task_handle
orspawn_essential_task
) to create tasks.