Impossible to run Spring Boot applications with default configuration and <512M

[sdeleuze]

The fact that it is not possible to run a Spring Boot hello world (MVC or WebFlux) with less than 512M and that most non-trivial applications requires at least 1G on Cloud Foundry is pretty frustrating for users, and give the false impression that Spring Boot can't run with less than 1G of memory.

We are doing some work with @dsyer, Boot team and others to optimize Spring Framework and Spring Boot to generate less GC pressure and have lower memory consumption, but I tend to think we could also improve Cloud Foundry to handle that in a better way.

I have raised cloudfoundry/java-buildpack-memory-calculator#24 about the memory calculation rule.

Another point: if the number of class is computed by counting all the .class files in the application including its dependencies, it is not a reliable source of information for Spring applications that has not a very fine grained JAR granularity and will load effectively just a small ratio of the classes available on Spring JARs.

The problem is even more visible with the optimizations we have shipped in Spring Boot 2.1, and the optimization we are currently working on for Spring Boot 2.2.

My gut feeling is that typical users knows how much Xmx memory is required locally by their application, and we should maybe use that information by default.

[making]

It is possible to run with less than 512M by configuring the memory calculator.
Following manifest works for me (spring boot + webflux).

applications:
- name: myapp
  path: target/myapp-0.0.1-SNAPSHOT.jar
  memory: 256m
  env:
    JAVA_OPTS: '-XX:ReservedCodeCacheSize=32M -XX:MaxDirectMemorySize=32M'
    JBP_CONFIG_OPEN_JDK_JRE: '[memory_calculator: {stack_threads: 30}, jre: {version: 11.+}]'

A vanilla webflux app actually runs with only 60MB ram.

I'm a big fan of memory calculator since most developers tend to care about only heap size and get an unexpected OOME (e.g. Metaspace) but I agree that we could improve it.
Default thread size (300?) in the calculator would be too large for at least webflux.

For beginners, it's pretty hard to find the way to customize the memory size.
Configuration examples by usecase in README would be very helpful.

[sdeleuze]

@making I have updated the title of this issue to make it more clear that it is about the default configuration being sub-optimal for (at least) Boot applications. It is of course totally possible to run Boot applications with 256M and custom configuration, but the default memory configuration seems to me very very far from accurate and that's where I would like we make some progress, because it impacts a lot of users.

The number of threads you mention for WebFlux application is indeed a very interesting point, the build pack could provide additional added value by detecting what kind of app it is. It may be tricky since some MVC application can use Reactive WebClient, but I am sure we can do something more clever and accurate.

Your custom memory configuration highlights IMO that something is wrong in the automatic configuration mechanism. As explained in cloudfoundry/java-buildpack-memory-calculator#24, if I specify explicitly the number of classes (between 8000 and 10000) effectively used by Boot apps, the parameter generated is -XX:ReservedCodeCacheSize=240M, where you specify -XX:ReservedCodeCacheSize=32M. This difference is really huge, could we make a more educated guess?

Also 8000 or 10000 are number of classed effectively used by Boot apps. If the build pack is computing that from the number of classes in the app + dependencies, I tend to think the number of classes specified to the memory calculator will be artificially high (I still need to check the value currently guessed but the build pack) due to the nature of Spring Framework JARs.

[sdeleuze]

I am also wondering if we leverage the new container memory options that are available in latest Java 8 and in Java 11. They are designed for Docker but I guess we could benefit as well of this in Cloud Foundry. Is Java runtime aware that we are running in containers despite CF not using Docker? Do we leverage options like -XX:InitialRAMPercentage, -XX:MaxRAMPercentage or -XX:MinRAMPercentage?

[youngm]

It is a tricky problem @sdeleuze. Does the java-buildpack use java defaults, that we assume some thought went into java setting, allowing applications to function, by default, more like they do when not running in CF? Or does the java-buildpack change defaults to make initial experience better but then leave users searching for answers when the application doesn't run as expected not in CF?

Today the java-buildpack has chosen to use java and tomcat defaults trusting in those values as a typical application today. I personally like that approach more than changing java defaults that may affect the application in ways not very visible/explicit to the user.

As far as the new properties go, my understanding of the RAMPercentage properties is that they simply automatically set heap values based on a percentage of ram available. No thoughts for non heap memory requirements like code cache size, thread stack size, metaspace, GC memory overhead, etc. That work is still left as an exercise for the user to guess at how much non heap memory their application needs. I look forward to a day when java can simply manage all the memory pools properly to keep an application within the memory constraints of a container. But I suspect we are still a long ways away from that.

[youngm]

As a side note this is the JAVA_OPTS and jre config I set for my applications that I need to be stable with low memory and speed isn't a factor.

• I set MaxMetaSize to a discovered through profiling.
  JAVA_OPTS

-Xss256K -Xms1M -XX:+UseSerialGC -Djava.compiler=none -XX:ReservedCodeCacheSize=2496k -XX:MaxDirectMemorySize=1M

jre config

  stack_threads: 20

[nebhale]

Let's go chronologically here:

1. if the number of class is computed by counting all the .class files in the application including its dependencies, it is not a reliable source of information for Spring applications that has not a very fine grained JAR granularity and will load effectively just a small ratio of the classes available on Spring JARs.

We determine the total number of classes available, but then crucially use a load factor of 35% of the available classes. We did an extensive analysis of different kinds of applications (including lots of Spring applications) and it's amazing how close this 35% number ended up being. We tweaked it a couple of times over the years and it's been stable and accurate for quite a while. If you know that it's being conservative and you load many fewer classes, you can override it, but given that each class is modeled at 5.7Kb it's going to have to be way off before the size changes meaningfully.

2. A vanilla webflux app actually runs with only 60MB ram.

Runs yes, but not well. The ReservedCodeCache is the space allocated by the JVM to the results of JIT compilation. All evidence that we've seen is that reducing the size of that memory space in Java 8 and later massively compromise the performance of the application (like orders of magnitude) because the JIT is instantly evicting previously compiled/optimized code and it's that code is never making it into the later compilation tiers. If you reduce that number significantly, you're effectively paying the CPU penalty of JIT-ing, but getting none of the benefits of ever running the optimized code.

3. Default thread size (300?) in the calculator would be too large for at least webflux.

Yep, totally true. Same is true about the default 10M of direct memory. However, given that the number of applications that are using reactive programming on Cloud Foundry is a rounding error, the default stays and overrides are available to users.

4. ...the parameter generated is -XX:ReservedCodeCacheSize=240M, where you specify -XX:ReservedCodeCacheSize=32M. This difference is really huge, could we make a more educated guess?

This is already the educated guess in that it's the JVM's default. We made a very conscious decision (-Xss=1M being another place) where we will choose the JVM's default (if it exists) even if it is obviously over-provisioned.

5. I am also wondering if we leverage the new container memory options that are available in latest Java 8 and in Java 11.

The new container memory options are no good. In addition to being questionably implemented, the assumptions they encode are fundamentally flawed. The problem is in the use of a percentage of available memory. Given a constant application (same number of classes, threads, etc.) but a variable size container the amount of non-heap memory is fixed. Metaspace, thread stacks, reserved code cache, etc. are all constant based on a given application. Therefore when a container is enlarged every single new byte should go to heap. By choosing a percentage, other than a single combination of application and container size, you're either over allocating heap, or over allocating the constant non-heap. Using those features makes you susceptible to both exceeding a container's memory limit and wasting unused space in a container unless they are changed in lock-step with the container size changes. The memory calculator on the other hand, handles this correctly, keeping non-heap constant and allocating heap with whatever is left as the container size changes. Check out the first section of the Buildpack 4.0 Release Announcement for more details on this whole problem.

[sdeleuze]

Thanks for the detailed feedback @nebhale, I really appreciate. I think you have answered to most of my questions or proposals.

Do you think we could give some guidance for async and reactive application in stack-threads documentation since this kind of application is likely to become more common in the future?

On my side, I plan also to experiment on some class filtering mechanism on Spring Fu side that could maybe lead to a more accurate estimation of the memory required when we use only a very small subset of Spring.

[nebhale]

Do you think we could give some guidance for async and reactive application in stack-threads documentation since this kind of application is likely to become more common in the future?

Sounds good. Please open an issue for that.