Skip to content

A third party tool to simulate the calculation result of Flink's memory configuration. Valid for Flink-1.10 and Flink-1.11.

License

Notifications You must be signed in to change notification settings

Mrart/flink-memory-calculator

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 

Repository files navigation

flink-memory-calculator

A third party tool to simulate the calculation result of Flink's memory configuration.

  • calculator-1.10.sh for Flink-1.10.
  • calculator-1.11.sh for Flink-1.11.

Usage: Add the calculator-{1.10 or 1.11}.sh to the FLINK_DIST/bin. You should set all configurations in your FLINK_CONF_DIR and then execute bin/calculator-{1.10 or 1.11}.sh. You could also append dynamic config options to it.

If you want to use "-tm", please repalce it with "-Dtaskmanager.memory.process.size=". This config would be deprecated eventually.

If successed, the result format should be:

JVM Parameters:
    -Xmx536870902
    -Xms536870902
    -XX:MaxDirectMemorySize=268435458
    -XX:MaxMetaspaceSize=100663296

TaskManager Dynamic Configs:
    taskmanager.memory.framework.off-heap.size=134217728b
    taskmanager.memory.network.max=134217730b
    taskmanager.memory.network.min=134217730b
    taskmanager.memory.framework.heap.size=134217728b
    taskmanager.memory.managed.size=536870920b
    taskmanager.cpu.cores=1.0
    taskmanager.memory.task.heap.size=402653174b
    taskmanager.memory.task.off-heap.size=0b

We also provide human-readable mode, you could switch to this mode by executing bin/calculator-{1.10 or 1.11}.sh -h. In this mode, all the memory size would be converted to a human-readable string. Note that there could be precision loss in converting. For the above sample, it would be:

JVM Parameters:
    -Xmx511mb
    -Xms511mb
    -XX:MaxDirectMemorySize=256mb
    -XX:MaxMetaspaceSize=96mb

TaskManager Dynamic Configs:
    taskmanager.memory.framework.off-heap.size=128mb
    taskmanager.memory.network.max=128mb
    taskmanager.memory.network.min=128mb
    taskmanager.memory.framework.heap.size=128mb
    taskmanager.memory.managed.size=512mb
    taskmanager.cpu.cores=1.0
    taskmanager.memory.task.heap.size=383mb
    taskmanager.memory.task.off-heap.size=0b

About

A third party tool to simulate the calculation result of Flink's memory configuration. Valid for Flink-1.10 and Flink-1.11.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Shell 100.0%