Document Docker Engine resource requirements for running tests

[dreis2211]

Hi,

I wanted to take a look at improving build times of BootBuildImageIntegrationTests, but noticed that 20% of them randomly fail on my local machine with messages like the following:

org.gradle.testkit.runner.UnexpectedBuildFailure: Unexpected build execution failure in /var/folders/zl/7zkcchrx0xb2t32qxx5wgh240000gp/T/gradle-4546028020545539739 with arguments [-PbootVersion=TEST-SNAPSHOT, --stacktrace, bootBuildImage, --pullPolicy=IF_NOT_PRESENT, --warning-mode, all]

Output:
> Task :compileJava
> Task :processResources
> Task :classes
> Task :bootWarMainClassName
> Task :bootWar

> Task :bootBuildImage
Building image 'docker.io/library/gradle-4546028020545539739:latest'

 > Executing lifecycle version v0.10.2
 > Using build cache volume 'pack-cache-6509a184f4ea.build'

 > Running creator
    [creator]     ===> DETECTING
    [creator]     5 of 18 buildpacks participating
    [creator]     paketo-buildpacks/ca-certificates   2.1.0
    [creator]     paketo-buildpacks/bellsoft-liberica 7.1.0
    [creator]     paketo-buildpacks/executable-jar    5.0.0
    [creator]     paketo-buildpacks/dist-zip          4.0.0
    [creator]     paketo-buildpacks/spring-boot       4.1.0
    [creator]     ===> ANALYZING
    [creator]     Previous image with name "docker.io/library/gradle-4546028020545539739:latest" not found
    [creator]     ===> RESTORING
    [creator]     ===> BUILDING
    [creator]     
    [creator]     Paketo CA Certificates Buildpack 2.1.0
    [creator]       https://github.com/paketo-buildpacks/ca-certificates
    [creator]       Launch Helper: Contributing to layer
    [creator]         Creating /layers/paketo-buildpacks_ca-certificates/helper/exec.d/ca-certificates-helper
    [creator]     
    [creator]     Paketo BellSoft Liberica Buildpack 7.1.0
    [creator]       https://github.com/paketo-buildpacks/bellsoft-liberica
    [creator]       Build Configuration:
    [creator]         $BP_JVM_VERSION              11.*            the Java version
    [creator]       Launch Configuration:
    [creator]         $BPL_JVM_HEAD_ROOM           0               the headroom in memory calculation
    [creator]         $BPL_JVM_LOADED_CLASS_COUNT  35% of classes  the number of loaded classes in memory calculation
    [creator]         $BPL_JVM_THREAD_COUNT        250             the number of threads in memory calculation
    [creator]         $JAVA_TOOL_OPTIONS                           the JVM launch flags
    [creator]       BellSoft Liberica JRE 11.0.10: Contributing to layer
    [creator]         Downloading from https://github.com/bell-sw/Liberica/releases/download/11.0.10+9/bellsoft-jre11.0.10+9-linux-amd64.tar.gz
    [creator]         Verifying checksum
    [creator]         Expanding to /layers/paketo-buildpacks_bellsoft-liberica/jre
    [creator]         Adding 129 container CA certificates to JVM truststore
    [creator]         Writing env.launch/BPI_APPLICATION_PATH.default
    [creator]         Writing env.launch/BPI_JVM_CACERTS.default
    [creator]         Writing env.launch/BPI_JVM_CLASS_COUNT.default
    [creator]         Writing env.launch/BPI_JVM_SECURITY_PROVIDERS.default
    [creator]         Writing env.launch/JAVA_HOME.default
    [creator]         Writing env.launch/MALLOC_ARENA_MAX.default
    [creator]       Launch Helper: Contributing to layer
    [creator]         Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/active-processor-count
    [creator]         Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/java-opts
    [creator]         Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/link-local-dns
    [creator]         Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/memory-calculator
    [creator]         Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/openssl-certificate-loader
    [creator]         Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/security-providers-configurer
    [creator]         Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/security-providers-classpath-9
    [creator]       JVMKill Agent 1.16.0: Contributing to layer
    [creator]         Downloading from https://github.com/cloudfoundry/jvmkill/releases/download/v1.16.0.RELEASE/jvmkill-1.16.0-RELEASE.so
    [creator]         Verifying checksum
    [creator]         Copying to /layers/paketo-buildpacks_bellsoft-liberica/jvmkill
    [creator]         Writing env.launch/JAVA_TOOL_OPTIONS.append
    [creator]         Writing env.launch/JAVA_TOOL_OPTIONS.delim
    [creator]       Java Security Properties: Contributing to layer
    [creator]         Writing env.launch/JAVA_SECURITY_PROPERTIES.default
    [creator]         Writing env.launch/JAVA_TOOL_OPTIONS.append
    [creator]         Writing env.launch/JAVA_TOOL_OPTIONS.delim
    [creator]     
    [creator]     Paketo Executable JAR Buildpack 5.0.0
    [creator]       https://github.com/paketo-buildpacks/executable-jar
    [creator]       Class Path: Contributing to layer
    [creator]         Writing env/CLASSPATH.delim
    [creator]         Writing env/CLASSPATH.prepend
    [creator]       Process types:
    [creator]         executable-jar: java org.springframework.boot.loader.WarLauncher (direct)
    [creator]         task:           java org.springframework.boot.loader.WarLauncher (direct)
    [creator]         web:            java org.springframework.boot.loader.WarLauncher (direct)
    [creator]     
    [creator]     Paketo Spring Boot Buildpack 4.1.0
    [creator]       https://github.com/paketo-buildpacks/spring-boot
    [creator]       Creating slices from layers index
    [creator]         dependencies
    [creator]         spring-boot-loader
    [creator]         snapshot-dependencies
    [creator]         application
    [creator]       Launch Helper: Contributing to layer
    [creator]         Creating /layers/paketo-buildpacks_spring-boot/helper/exec.d/spring-cloud-bindings
    [creator]       Spring Cloud Bindings 1.7.0: Contributing to layer
    [creator]         Downloading from https://repo.spring.io/release/org/springframework/cloud/spring-cloud-bindings/1.7.0/spring-cloud-bindings-1.7.0.jar
    [creator]         Verifying checksum
    [creator]         Copying to /layers/paketo-buildpacks_spring-boot/spring-cloud-bindings
    [creator]       Web Application Type: Contributing to layer
    [creator]         Non-web application detected
    [creator]         Writing env.launch/BPL_JVM_THREAD_COUNT.default
    [creator]       4 application slices
    [creator]       Image labels:
    [creator]         org.springframework.boot.version
    [creator]     ===> EXPORTING
    [creator]     Adding layer 'paketo-buildpacks/ca-certificates:helper'
    [creator]     ERROR: failed to export: caching layer (sha256:84bf98cca9651949da8909a7f9cdefaa71648359ef97fb2de93af40019bf948e): write /launch-cache/staging/sha256:84bf98cca9651949da8909a7f9cdefaa71648359ef97fb2de93af40019bf948e.tar: no space left on device

> Task :bootBuildImage FAILED

I wonder if we can improve that somewhat or how I need to tweak my Docker setup (naively giving it more space, didn't fix things)

As my original intention was to improve the time of those tests (around 30 minutes on my machine), I'd appreciate any thoughts on that, too.

Cheers,
Christoph

[wilkinsona]

Judging by no space left on device, I think you need to give Docker some more disk space. Could you please give that a go and let us know if things stabilise?

[dreis2211]

@wilkinsona Ha - just edited the text with that. Did that already, but maybe not enough.

[wilkinsona]

FWIW, my Docker install's got 128GB of disk. I had a clear out recently so it's currently only using 11.9GB. I'm sure it doesn't need over 100GB of head room, but it might be worth giving it even more space. How much free space does your install have?

[dreis2211]

It had 10GB left, but I just gave it another 20GB. I'll report back in 30 minutes

[dreis2211]

Yeah, 20GB more worked. Feel free to close the ticket or repurpose to a documentation ticket

[scottfrederick]

I wanted to take a look at improving build times of BootBuildImageIntegrationTests

As my original intention was to improve the time of those tests (around 30 minutes on my machine), I'd appreciate any thoughts on that, too.

@dreis2211 I've been working on some new integration tests to verify the contracts between the Spring Boot image-building plugins and the Paketo buildpacks (for example the features documented by Paketo as enabled by Spring Boot). It's a work-in-progress right now on a branch here. The idea is that these tests would be run from different triggers than the current builds that run on every commit, such as triggering by new Paketo builder images in Docker Hub and Boot snapshots being published.

As part of this, we've started discussing changing the approach used by the existing Gradle BootBuildImageIntegrationTests and Maven BuildImageTests, which do take a long time to run. We might change the tests run on each commit to stub out a layer underneath the plugins instead of invoking the full Docker and CNB Builder paths, and leave the full-path testing to the new integration tests. We can share more information once we have time to discuss as a team again.

[scottfrederick]

Created #25824 for further feedback.

[dreis2211]

@scottfrederick I actually thought about extracting that from the Gradle plugin tests, but in a first try I ended up with quite some copy-paste as well. I'm somewhat glad that you ended up with that, too ;-)

My initial thought before that was to come up with a new annotation @EnabledOnCI that is only running things when on CI and not put the burden on local machines. Maybe that's a pragmatic step in between.

In general, I'm curious what you can come up with. Thanks for sharing your thoughts. :)

[scottfrederick]

We can add tips for Docker resource configuration to the Working with the Code section of CONTRIBUTING.adoc.

[scottfrederick]

We'll add the Docker configuration tips to the Working with the Code wiki page.

[scottfrederick]

Documentation added to https://github.com/spring-projects/spring-boot/wiki/Working-with-the-Code#tests-requiring-docker.

I cleaned all containers, images, and volumes from my local Docker and ran a full build with tests. Docker reported using just under 20GB of the disk image space, so hopefully the documented recommendation holds up for others.