Increase LOTUS_FVM_CONCURRENCY
#11817
Labels
Milestone
Comments
Stebalien
added a commit
to filecoin-project/filecoin-ffi
that referenced
this issue
Apr 3, 2024
We used to allocate 1024 instances per "concurrency" but we now only allocate 1024 plus 20 per "concurrency". So, I'm increasing the default and maximum concurrency factors to take advantage of this. Importantly: 1. Max concurrency 128 -> 4096. `1024 + 4096*20 < 128 * 1024`. 2. Default concurrency 4 -> 150. `1024 + 150*20 < 4 * 1024`. Motivation: I'm hoping that the default (150) is sufficient (more than the previous max...) so we can just get rid of this knob. fixes filecoin-project/lotus#11817
Stebalien
added a commit
to filecoin-project/filecoin-ffi
that referenced
this issue
Apr 6, 2024
We previously used the FVM's `ThreadedExecutor` to execute messages on separate threads because the FVM requires 64MiB of stack space. 1. The FVM v3 supported for 8 concurrent threads. 2. The FVM v4 supports up to the number of CPU threads available. Unfortunately, neither version was influenced by the `LOTUS_FVM_CONCURRENCY` environment variable. This patch fixes this by: 1. Moving the thread-pool to the FFI itself (sharing it between FVM versions). 2. Setting the thread-pool size equal to `LOTUS_FVM_CONCURRENCY`. It also defaults `LOTUS_FVM_CONCURRENCY` to the number of available CPU threads instead of the previous 4. fixes filecoin-project/lotus#11817
Stebalien
added a commit
to filecoin-project/filecoin-ffi
that referenced
this issue
Apr 6, 2024
We previously used the FVM's `ThreadedExecutor` to execute messages on separate threads because the FVM requires 64MiB of stack space. 1. The FVM v3 supported for 8 concurrent threads. 2. The FVM v4 supports up to the number of CPU threads available. Unfortunately, neither version was influenced by the `LOTUS_FVM_CONCURRENCY` environment variable. This patch fixes this by: 1. Moving the thread-pool to the FFI itself (sharing it between FVM versions). 2. Setting the thread-pool size equal to `LOTUS_FVM_CONCURRENCY`. It also defaults `LOTUS_FVM_CONCURRENCY` to the number of available CPU threads instead of the previous 4. NOTE: I've also tried increasing the stack size instead of using threads, but Go _does not_ like it when other foreign mess with the stack size of _its_ threads (but it has no problem if we create our own threads). fixes filecoin-project/lotus#11817
Stebalien
added a commit
to filecoin-project/filecoin-ffi
that referenced
this issue
Apr 6, 2024
We previously used the FVM's `ThreadedExecutor` to execute messages on separate threads because the FVM requires 64MiB of stack space. 1. The FVM v3 supported for 8 concurrent threads. 2. The FVM v4 supports up to the number of CPU threads available. Unfortunately, neither version was influenced by the `LOTUS_FVM_CONCURRENCY` environment variable. This patch fixes this by: 1. Moving the thread-pool to the FFI itself (sharing it between FVM versions). 2. Setting the thread-pool size equal to `LOTUS_FVM_CONCURRENCY`. It also defaults `LOTUS_FVM_CONCURRENCY` to the number of available CPU threads instead of the previous 4. NOTE: I've also tried increasing the stack size instead of using threads, but Go _does not_ like it when other foreign mess with the stack size of _its_ threads (but it has no problem if we create our own threads). fixes filecoin-project/lotus#11817
8 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We set this pretty low (4) because we used to reserve
concurrency*1024instances up-front. We now only allocate1024 + concurrency * 20, so we should be able to increase the default significantly.The text was updated successfully, but these errors were encountered: