[Worker] heap out of memory

[sapics]

• Node.js Version: Build from main with gcc-10
• OS: Ubuntu 20.04
• Scope (install, code, runtime, meta, other?): code

I have got error occasionally [ERR_WORKER_OUT_OF_MEMORY]: Worker terminated due to reaching memory limit: JS heap out of memory.

I am using nodejs with 125 worker in [64 core, 128 thread, 256GB memory] machine, and my program eat a lot of memory.
Thus, each worker can use only 2GB memory, at a maximum.
To decrease the number of errors, I set the following configuration.

export CFLAGS="-march=native"
export CXXFLAGS="-march=native"
export CPPFLAGS="-march=native"
./configure --experimental-enable-pointer-compression --enable-lto

and run nodejs with node --max-old-space-size=2040.

From the documentation, I guess that --max-old-space-size=2040 is inherited to worker.

execArgv <string[]> List of node CLI options passed to the worker.
V8 options (such as --max-old-space-size) and options that affect
the process (such as --title) are not supported.
If set, this will be provided as process.execArgv inside the worker.
By default, options will be inherited from the parent thread.

Question:

• Are there any recommended configuration option to add or remove for decreasing memory usage or running faster?
• I have not set worker.resourceLimits yet. Should I set resourceLimits.maxOldGenerationSizeMb instead of --max-old-space-size?

[addaleax]

• Are there any recommended configuration option to add or remove for decreasing memory usage or running faster?

You may be running into the problem described in nodejs/node#32265 (comment) – i.e. you’re hitting the memory map limit in Linux. Increasing it might help?

• I have not set worker.resourceLimits yet. Should I set resourceLimits.maxOldGenerationSizeMb instead of --max-old-space-size?

That’s up to you, they both achieve the same thing.

[sapics]

Thank you @addaleax! I check Linux memory map limit.

That’s up to you, they both achieve the same thing.

Oh, I undestand 👍

[sapics]

I check settings which relates vm.max_map_count

# sysctl vm.max_map_count
vm.max_map_count = 65530
# sysctl kernel.threads-max
kernel.threads-max = 2060362
# sysctl kernel.pid_max
kernel.pid_max = 4194304

I guess that vm.max_map_count should be twice of kernel.threads-max, thus, I set as follows.

# sysctl -w vm.max_map_count=4120724

My error is occasionally about 2-3 times a month, thus, I will check this setting about a month.
Thanks again @addaleax !

[sapics]

Thanks to the suggestions, the error is gone! 👍