Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improvement]: Flink optimizer configure managed memory explicitly in both the doc and the startup script #1907

Closed
2 of 3 tasks
Tracked by #1930
wangtaohz opened this issue Sep 1, 2023 · 0 comments

Comments

@wangtaohz
Copy link
Contributor

Search before asking

  • I have searched in the issues and found no similar issues.

What would you like to be improved?

The current Flink optimizer mainly utilizes the heap memory of Task Manager, but Flink allocates around 50% of memory to managed memory by default, resulting in less heap memory available and increasing the risk of OOM issues for the Task Manager.

How should we improve?

To reduce the risk of OOM of Task Manager caused by unreasonable managed memory configuration, the configuration taskmanager.memory.managed.size should be explicitly specified in the doc and script of starting Flink optimizer.

And taskmanager.memory.managed.size=32mb may be a reasonable value.

The doc is here: https://github.com/NetEase/amoro/blob/master/docs/admin-guides/managing-optimizers.md#deploy-external-optimizer.

The startup script is here: com.netease.arctic.optimizer.FlinkOptimizerContainer#buildOptimizerStartupArgsString

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Subtasks

No response

Code of Conduct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant