-
Notifications
You must be signed in to change notification settings - Fork 410
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to set a memory limit for a sandboxed processor? #1555
Comments
Currently they are implemented as spawn processes, so there is no such thing as memory configuration AFAIK. |
Thanks for the answer. What is the advantage of using a spawned process (in comparison to worker threads)? Also - when running a node.js app, there's an option to set the I believe setting this param for the spawned process (it is spawned via |
Ok, if I understand properly, they are implemented as forks, (not as spawned processes) -https://github.com/taskforcesh/bullmq/blob/master/src/classes/child-pool.ts#L114 Which is even better, because I believe that each fork has a new V8 instance created = better isolation from the "host" envrionment. And if that's really the case - it should be possible to set the memory limit per forked process - with sth like:
https://stackoverflow.com/a/37596117 Would you consider adding the max available memory setting in the Worker configuration options? |
Yeah well, a fork is also a spawn, the only difference is that it creates a new V8 for you, but it is still a new process so there is no practical difference (with spawn you will need to call the node cmd line tool instead). |
For me personally - the simplest solution (assuming, that this is more of an "advanced" feature) - with sth like this: new Worker(updateQueueName, updateProcessor, {
concurrency: workersConfig.update,
metrics: {
maxDataPoints: MetricsTime.ONE_WEEK,
},
sandboxExecArgv: ['--max-old-space-size=128', '--foo=bar']
}); would be awesome! The |
Hey, any updates? :-) |
Hey - just asking while reviewing our internal issues ;-) - is there any timeline for implementing this feature? |
Hey, do you plan to implement this feature? EDIT: I see, that support for worker_threads has been recently added (0820985) - does this new implementation support https://nodejs.org/api/worker_threads.html#workerresourcelimits ? |
It does not seem to be a lot of interest in this feature. However if someone creates a PR with a clean and simple implementation I do not have anything against merging it. |
i need this feature too |
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [bullmq](https://bullmq.io/) ([source](https://github.com/taskforcesh/bullmq)) | dependencies | minor | [`5.13.2` -> `5.15.0`](https://renovatebot.com/diffs/npm/bullmq/5.13.2/5.15.0) | --- ### Release Notes <details> <summary>taskforcesh/bullmq (bullmq)</summary> ### [`v5.15.0`](https://github.com/taskforcesh/bullmq/releases/tag/v5.15.0) [Compare Source](taskforcesh/bullmq@v5.14.0...v5.15.0) ##### Features - **worker-fork:** allow passing fork options ([#​2795](taskforcesh/bullmq#2795)) ([f7a4292](taskforcesh/bullmq@f7a4292)) ### [`v5.14.0`](https://github.com/taskforcesh/bullmq/releases/tag/v5.14.0) [Compare Source](taskforcesh/bullmq@v5.13.2...v5.14.0) ##### Features - **worker-thread:** allow passing Worker options ([#​2791](taskforcesh/bullmq#2791)) ref [#​1555](taskforcesh/bullmq#1555) ([6a1f7a9](taskforcesh/bullmq@6a1f7a9)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOC4xMDEuMSIsInVwZGF0ZWRJblZlciI6IjM4LjEwNi4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiXX0=--> Reviewed-on: https://git.tristess.app/alexandresoro/ouca/pulls/178 Reviewed-by: Alexandre Soro <code@soro.dev> Co-authored-by: renovate <renovate@git.tristess.app> Co-committed-by: renovate <renovate@git.tristess.app>
I assume that Sandboxed Processors work underneath as a separate node.js Workers.
How can I pass the memory configuration to the Sandboxed processor?
E.g. the
resourceLimits. maxOldGenerationSizeMb
from the original Worker apihttps://nodejs.org/api/worker_threads.html#worker_threads_new_worker_filename_options
The text was updated successfully, but these errors were encountered: