Windows service uses incorrect value for MaxDirectMemorySize

[DaveCTurner]

A user on the forums reported an issue with their 7.3.0 nodes hitting java.lang.OutOfMemoryError: Direct buffer memory frequently. The nodes were being run with -XX:MaxDirectMemorySize=536870912 despite a 6GB heap. It turned out the heap size was being set twice:

-XX:MaxDirectMemorySize=536870912 would be consistent with JvmErgonomics incorrectly picking up that the heap is 1GB rather than 6GB. I don't have a way to reproduce this to investigate further but this feels like a bug somewhere along the way.

[russcam]

Thanks for raising @DaveCTurner.

I think this issue should be opened on https://github.com/elastic/elasticsearch, as it looks like it relates to running the zip distribution as a Windows Service; looking at the screenshot. is looks to be using Apache Commons Daemon, which is what the zip distribution uses to run Elasticsearch as a Windows Service.

The MSI installation overwrites the heap values in jvm.options, so the default values are overridden.

[DaveCTurner]

Thanks @russcam, I've moved this to the main Elasticsearch repo.

[elasticmachine]

Pinging @elastic/es-core-infra (:Core/Infra/Packaging)

[williamrandolph]

It looks like the setup here is controlled by an invocation in bin\elasticsearch-service.bat. We're parsing Xmx and Xms values out of the java options string and feeding them into the Commons Daemon service setup, but we're not then removing those values from the options string, leading to the redundancy and potential for drift we see above.

Conceptually, the fix is simple — remove the Xmx/Xms settings from the options string after parsing them — but the work will be figuring out how to make that happen in a Windows batch script.