Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

remote-externalities: parallelize value download #12044

Closed
kianenigma opened this issue Aug 16, 2022 · 1 comment · Fixed by #12537
Closed

remote-externalities: parallelize value download #12044

kianenigma opened this issue Aug 16, 2022 · 1 comment · Fixed by #12537
Labels
I9-optimisation An enhancement to provide better overall performance in terms of time-to-completion for a task. Z2-medium Can be fixed by a coder with good Rust knowledge but little knowledge of the codebase.

Comments

@kianenigma
Copy link
Contributor

The way this tool works now is that we first download all of they keys, in a paginated way. Then, we start requesting the values of the corresponding keys in batches. My guess is that this process can still be optimized by parallelizing the requests.
@kianenigma kianenigma added I9-optimisation An enhancement to provide better overall performance in terms of time-to-completion for a task. Z2-medium Can be fixed by a coder with good Rust knowledge but little knowledge of the codebase. labels Aug 16, 2022
@kianenigma kianenigma moved this to 📕 Backlog in (Nominated) Proof of Stake Aug 16, 2022
@kianenigma kianenigma moved this from 📕 Backlog to ⌛️ Sometime-soon in (Nominated) Proof of Stake Aug 16, 2022
@ggwpez
Copy link
Member

ggwpez commented Aug 17, 2022

You can also play with the PAGE and BATCH_SIZE parameters to see if there is a sweet-spot.

@kianenigma kianenigma mentioned this issue Oct 20, 2022
Merged
2 tasks
Repository owner moved this from ⌛️ Sometime-soon to ✅ Done in (Nominated) Proof of Stake Dec 14, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
I9-optimisation An enhancement to provide better overall performance in terms of time-to-completion for a task. Z2-medium Can be fixed by a coder with good Rust knowledge but little knowledge of the codebase.
Projects
(Nominated) Proof of Stake
Status: ✅ Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Try-runtime Revamp and Facelift paritytech/substrate
2 participants
@kianenigma @ggwpez