Auto adjust RAM

[andsens]

Instead of hardcoding Xms and Xmx to 1024M you could do some autodetection like this

# Total memory
memory_kb=`awk '/MemTotal/{print $2}' /proc/meminfo`

# Calculate minimum memory
memory_min=$(($memory_kb/100*30))
# Calculate maximum memory
memory_max=$(($memory_kb/100*80))

java -jar -Xms${memory_min}K -Xmx${memory_max}K ....

It's good practice not to set min and max to the same, this way more than one minecraft server can run. The can change their size in RAM depending on how many players there are in each world.
If only one world is played it can use all the RAM it wants.

The minimum should be adjusted to the number of worlds running. With 4 worlds for example, it should be 20% (=80% of RAM).
With 6 worlds and 20% the server would start to swap, so it should be 13%.

[marcuswhybrow]

In all my reading I have been given the opposite impression: that "it is good practice to in fact set the min and max to be the same". I have thought on the matter of Mms and Mmx values before, so forgive this mental dump:

In my experience there are some reasons to advise against differing Mms (minimum) and Mmx (maximum), although they are derived from reading on the subject and not actual testing:

• The minecraft.net download page and the Minecraft wiki suggest using the same amount for both values. The wiki cites that, quiet understandably, that setting the initial ram to be less than the max "will result in a little slowdown whenever the allocation is done".
• This StackOverflow answer gives a nice explanation of Xms and Xmx and suggests not using the minimum flag when you have a dedicated service, unless you have a good reason. (Maybe we do.)
• In my understanding of garbage collection theory the JVM, if allowed a smaller initial memory setting than the maximum allowed, would allocate more memory when the heap becomes too small (the obvious part), but when memory is deallocated, after a GC, I cannot find any documentation that suggests the JVM will relinquish RAM back to the system for use, in our case, by other Minecraft JVM's. This situation, if correct, would still require admins to calculate what RAM they have and what total RAM each server should get as a result, and just add the extra annoyance of Minecraft server's lagging every time they allocate more memory.

---

Other comments:

• You suggest defaulting to 80% of maximum RAM for a new server, possibly leaving 20% for system stuff and maybe in-ram worlds. Since each JVM will happily increase to this size, will it not cause paging as soon as a second server is introduced as a combined 160% of RAM will be in use straight away.
• I assume your referral to "worlds" in your calculations means separate JVM server instances?
• I think (I would like to know what you think) this would result in every server constantly lagging with memory allocation problems, and also, in combination, utilising more RAM than is physically available: causing paging lag.

I don't think MSM can take the RAM allocation responsibly away from the administrator, they will know their system best including what other servers are running utilising what RAM already, what each server requires resource-wise, and whether or not they expect/accept paging in operation.

---

I have written, it suddenly occurs to me, enough for one comment. Let me know what you think.

[andsens]

With worlds I mean separate JVM server instances, yes.
I don't think we have to worry about the allocation lags too much, in the grand scheme of things I suspect this lag is negligible. After all: It's ram allocation, you're not writing anything to it and we are not talking about the harddisk here. Also, it is only done when the instance needs more space, this should happen a bit at startup but only seldom later on.

Of course setting the max of one instance to 80% and then starting a second world is a problem. This problem goes away once you restart all the instances, I don't really know how this could be fixed otherwise

With regard to taking the choice away from the admin:
This is a manager, it can be argued that management of memory allocation can be expected (convention over configuration, allow adjustments).

Pagefaults should not be avoided at all costs of course, it's an easy way to move an unused instance out of memory and allows used instances to utilize the RAM better.

By the way: I have created a plugin for my ec2 bootstrapping script that installs msm. I have tested it on an ec2 micro instance with 650MB RAM, I set Xmx to 350M and the server runs just fine. Mind you, this was only with one client.

[andsens]

Actually, what if Xms and Xmx were not set at all? Let java do the calculations: http://docs.oracle.com/javase/1.5.0/docs/guide/vm/gc-ergonomics.html
Allocation lag can be avoided by checking what an instance consumes just after it is started. Set Xms to that.
This way the JVM will not have to allocate RAM piecemeal wise.

[marcuswhybrow]

I'm a believer in sensible defaults with easy overrides: so I think it would be a good idea to not set RAM at all like you suggest and allow admins to set hard values if they think it best.

MSM could even check the instance RAM usage automatically after startup and suggest an Xms value for future use. Or even set that as the server specific Xms value after the first run, so by default you get quick startup allocation after the first time.

With regard to the Xmx upper limit: will it not default to 1GB instead of having no upper limit if unspecified (based on that oracle link). If that is the case I think the best default option is to specify it as a percentage of total RAM and allow instances to page if they use that much.

[marcuswhybrow]

Also that plugin for ec2 looks good. I've never used EC2 myself but I read through it and I get the gist.

[JonathanStade]

An important consideration for implementing this will be a design decision of whether msm is going to assume that the primary purpose of a server it is installed on is minecraft. I dropped msm onto my mail/dns/LAMP server, and I took a careful look at the memory settings to make sure java wasn't going to starve the other services of RAM: if it had arbitrarily decided set Xms & Xmx to 2G, it would have made a bit of a mess of the mysql tuning.

[marcuswhybrow]

This is something I was wary of also, and a case for leaving all RAM settings up the admin. I think it MSM should probably have an over eager default for the common user, for have a strong note about it in setup urging experienced users to set there own RAM allocations.

I know if I read that I would set RAM usage myself.