Support percentage-based --max-old-space-size option for dynamic memory allocation

[Asaf-Federman]

What is the problem this feature will solve?

Currently, --max-old-space-size requires a fixed memory value in MB. However, in containerized environments (Docker, Kubernetes, etc.), available memory can vary. A percentage-based option would allow Node.js to dynamically adjust its memory usage based on the environment, reducing the risk of OOMKills while optimizing performance.

Why is this needed?

• Improves compatibility with containerized environments like Kubernetes, where memory limits are dynamic.
• Prevents over-allocating memory and OOMKills when limits change.
• Simplifies memory management without requiring external scripts to calculate heap size dynamically.

What is the feature you are proposing to solve the problem?

Introduce a new syntax for --max-old-space-size, allowing it to accept percentages of total system memory (e.g., --max-old-space-size=80%).

For example:
node --max-old-space-size=80% app.js

If the system has 4GB RAM, this would set 3.2GB (80%) as the old space size.
If the system has 1GB RAM, it would set 800MB (80%).

What alternatives have you considered?

• Users must manually set a fixed value.
• Some use external scripts (os.totalmem()) to compute the size dynamically.

[bnoordhuis]

--max-old-space-size is a V8 flag, not a Node.js flag. It's technically possible to intercept and rewrite V8 flags but we've never done that before and personally I see no reason to start now.

Also, percentages seems like a bad idea in general. What happens when I start two node processes with --max-old-space-size=80%? Boom, is what.

Not a hypothetical. The flag is allowed in NODE_OPTIONS. Child processes inherit it.

[Asaf-Federman]

Hi bnoordhuis,

I appreciate your points about the potential complexities of using --max-old-space-size with percentages, especially when dealing with multiple Node.js processes. However, I believe there are compelling reasons to consider allowing this functionality.

One significant advantage of supporting percentage values is the potential for dynamic resource allocation, similar to what's seen with Vertical Pod Autoscaler (VPA) in Kubernetes. By utilizing percentages, we can enable applications to adapt their memory usage based on the available resources at runtime. This flexibility could lead to more efficient memory management, especially in containerized environments where resource limits can change.

Moreover, allowing percentage-based configurations could help developers optimize their applications without the need to hardcode specific values, which might not suit varying production environments. It empowers users to maintain better control over their applications' memory usage, potentially improving performance and resource utilization.

Thanks for your input!

[michaelper22]

Hi bnoordhuis,

While the implications of more config permutations cannot be ignored, percentage-based is a pattern reliably utilized in production environments in other platforms, notably Java with -XX:MaxRAMPercentage=nn%. Allowing percentages would let our JS developers unify these configs across the enterprise.

Are there popular frameworks/libaries for Node which fork processes in their standard functionality?

[bnoordhuis]

Are there popular frameworks/libaries for Node which fork processes in their standard functionality?

Yes, plenty. The built-in cluster module, for starters.

[Asaf-Federman]

I'd like to emphasize that even with fixed values, we can still encounter similar problems.

When we set a fixed value for --max-old-space-size, if both the parent and child processes are given a substantial allocation, the combined memory usage can easily exceed the available system memory, leading to crashes or performance issues.

Moreover, this feature shouldn't introduce any regressions or breaking changes; hence, any existing codebase won't break

[bnoordhuis]

I'd like to emphasize that even with fixed values, we can still encounter similar problems.

Yes, obviously.

To set expectations: don't expect anyone to work on this. Open a pull request yourself if you want to see it happen. Keep in mind there's no guarantee it's going to get merged.

In your pull request you're going to have to answer the question what 80% really means. 80% of total system memory? What if the process is in a cgroup? Does it make sense to limit yourself to 80% of the cgroup's memory limit? If yes, why? If not, why not?

[Asaf-Federman]

I'd like to emphasize that even with fixed values, we can still encounter similar problems.

Yes, obviously.

To set expectations: don't expect anyone to work on this. Open a pull request yourself if you want to see it happen. Keep in mind there's no guarantee it's going to get merged.

In your pull request you're going to have to answer the question what 80% really means. 80% of total system memory? What if the process is in a cgroup? Does it make sense to limit yourself to 80% of the cgroup's memory limit? If yes, why? If not, why not?

Thanks, @bnoordhuis , for your feedback!

One last question, if I may: would it be more fitting to try to contribute this effort to the V8 engine instead of Node.js?

[bnoordhuis]

V8 is the better place, the question is if they'd accept it.

[Asaf-Federman]

I reached out to the V8 community regarding this feature, and they indicated that it would be more appropriate to contribute it to the Node.js project. You can find the relevant discussion here.

In light of this feedback, I intend to proceed with contributing this feature to the Node.js project. Any assistance or guidance on this would be greatly appreciated!

[mlc-mlapis]

@bnoordhuis @Asaf-Federman You are right. The containerized environments and their dynamic ability to scale the amount of memory resources for each container are a long-term, underutilized potential, especially if those environments can scale the allocated memory in minutes and charge based on exact consumption. It is a growing, serious disadvantage of the Node.js platform.

[Asaf-Federman]

I'm still working on the percentage implementation for --max-old-space-size. A functional version is now complete, but it requires code cleanup, proper testing, etc before a pull request can be submitted.