Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Worker implementation selection #63

Closed
yurynix opened this issue Jun 13, 2023 · 3 comments · Fixed by #64
Closed

Worker implementation selection #63

yurynix opened this issue Jun 13, 2023 · 3 comments · Fixed by #64

Comments

@yurynix
Copy link
Contributor

yurynix commented Jun 13, 2023

Hi 👋
Thank you for the great library! 🙏

When moving from 1.x to 2.x, the implementation of the workers was switched from process (child_process) to threads (worker_threads), which is great for many workloads, however, for some workloads, processes are preferred.

Specifically what bothers me is workloads that do process.chdir() which is not supported inside worker_threads.

Would you be open to a new option in WorkerNodesOptions -> .workerType which can get thread (default) or process,
that will control the underlying implementation?

I don't think it's a big code change, however one downside is that WorkerNodesOptions.resourceLimits is only going to be supported in worker_threads.

What do you think? 🙃

@yurynix
Copy link
Contributor Author

yurynix commented Jun 15, 2023

@bgalek what are your thoughts on this? 🙃

@bgalek
Copy link
Member

bgalek commented Jun 15, 2023

@yurynix if we leave worker_threads as a default I'm ok with providing option to change thread to process ;)

@yurynix
Copy link
Contributor Author

yurynix commented Jun 15, 2023

Cool, so I'll draft a PR next week 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants