buildDockerImageTask runs in every invocation of :distribution:docker:integTest

[jkakavas]

This happens even though nothing has changed in between, which I would assume make gradle reuse existing images.

Reproduces for instance, by running:

./gradlew ':distribution:docker:integTest' --tests "org.elasticsearch.docker.test.DockerYmlTestSuiteIT.test {p0=/11_nodes/Additional disk information}"

consecutively

[elasticmachine]

Pinging @elastic/es-core-infra (:Core/Infra/Build)

[pugnascotia]

Also very irritating when iterating on Docker packaging tests and running them locally.

[mark-vieira]

Indeed, we should make these tasks support incremental build.

[pugnascotia]

@jkakavas this should be fixed in master. Would you mind verifying?

[pugnascotia]

Actually, Mark found that we still need some improvements here.

[mark-vieira]

This was fixed with #49613.

Here's an example build with all the build tasks UP-TO-DATE: https://gradle-enterprise.elastic.co/s/fb36ht6rfwtty

[pugnascotia]

@mark-vieira there seems to be a regression here - https://gradle-enterprise.elastic.co/s/y447z33bdzgrg

What happened was:

1. I make a change to the Docker-related files
2. Ran the DockerTests, which rebuilt the image and ran the test
3. I then immediately re-ran one of the tests, which rebuilt the image

[rjernst]

Looks like the ML jar changed:
https://gradle-enterprise.elastic.co/s/y447z33bdzgrg/timeline?end=1579690100773&start=1579690100029#cpltkof7et2ea

[rjernst]

@pugnascotia Do you have the build scan from the build just before the one you linked?

[pugnascotia]

@mark-vieira I did two test runs back-to-back in my IDE:

First: https://gradle-enterprise.elastic.co/s/4ddr4hi2dlpa2
Second: https://gradle-enterprise.elastic.co/s/ovtybfoiywgjo

You can see in the second that it rebuilt the linux tar and docker image, when I had changed nothing.

[pugnascotia]

Another run from the command line, instead of IntelliJ, and just building the Docker image. Again, back-to-back runs, on master this time.

1. https://gradle-enterprise.elastic.co/s/j2elqcpc7rm6s
2. https://gradle-enterprise.elastic.co/s/2woutebnogeay

[rjernst]

@pugnascotia Can you please add -Dscan.capture-task-input-files to your command line and run the two again?

[pugnascotia]

1. https://gradle-enterprise.elastic.co/s/ntzgihd5t2cyk
2. https://gradle-enterprise.elastic.co/s/tfsmubr5ossv6

[mark-vieira]

@pugnascotia I am unable to reproduce this locally for the life of me. Is this continuing to happen for you? @rjernst were you able to reproduce this locally?

If this is still happening do you mind sharing the contents of x-pack/plugin/ml/build/tmp/jar/MANIFEST.MF before each build run?

[pugnascotia]

Something's changed on master, because I just did two back-to-back builds and the second run was fine. I even ran the tests and saw that the first time it exported the Docker image as expected, and the second time it didn't.

So we're all good here as far as I can tell.

[mark-vieira]

Thanks, Rory.