Using Docker Stages to support no-argument docker bulding #6764
Conversation
|
Now to begin the process of making the travis tests work on this branch.... |
|
Nifty, I learned about docker system prune... |
jamesemery
force-pushed
the
je_makeTheDockerBuildContained
branch
from
374b0cd to
dadc7e3
Compare
|
@lbergelson this is finally working on travis! can you take a look and let me know your thoughts? |
.travis.yml
Outdated
| @@ -158,15 +158,22 @@ script: | |||
| fi; | |||
| sudo docker images; | |||
| echo ${TEST_TYPE}; | |||
| sudo mkdir -p build/reports/; | |||
| sudo chmod -R a+w build/reports/; | |||
| sudo rm -r ./build; | |||
There was a problem hiding this comment.
Why was this necessary? Just precautionary?
| sudo rm -r ./build; | ||
| ./gradlew clean shadowTestClassJar shadowTestJar; | ||
| mkdir ./testJars; | ||
| echo $( find ./build/libs/ -name "gatk*test.jar"); |
There was a problem hiding this comment.
Could you add some comments here explaining what this is for?
| @@ -16,8 +16,8 @@ | |||
| // Asserting that Picard programs are able to pipe their output without errors | |||
| public void testPipeForPicardTools() { | |||
| File output = createTempFile("testOutput",".bam"); | |||
| BaseTest.runProcess(ProcessController.getThreadLocal(), new String[]{"/bin/sh","-c"," ./gatk SortSam -I "+INPUT_BAM+" -O /dev/stdout -SO coordinate " + | |||
| "| ./gatk SetNmMdAndUqTags -I /dev/stdin -O "+ output.getAbsolutePath()+ " --CREATE_INDEX true -R "+b37_reference_20_21});//.split(" ")); | |||
| BaseTest.runProcess(ProcessController.getThreadLocal(), new String[]{"/bin/sh","-c"," ${GATK_LAUNCH_SCRIPT} SortSam -I "+INPUT_BAM+" -O /dev/stdout -SO coordinate " + | |||
There was a problem hiding this comment.
We should be careful about this substitution. Can you have it default to ./gatk if it doesn't exist?
|
|
||
| RUN export GRADLE_OPTS="-Xmx4048m -Dorg.gradle.daemon=false" && /gatk/gradlew clean collectBundleIntoDir shadowTestClassJar shadowTestJar -Drelease=$RELEASE | ||
| RUN cp -r $( find /gatk/build -name "*bundle-files-collected" )/ /gatk/unzippedJar/ | ||
| RUN unzip -o -j $( find /gatk/unzippedJar -name "gatkPython*.zip" ) -d /gatk/unzippedJar/scripts |
There was a problem hiding this comment.
comments about the test jar would be useful to explain that it's necessary plumbing for tests but not for the build itself
|
@lbergelson can you re-review? i have responded to your comments |
|
@jamesemery Did you forget to push some changes? I don't see any of the comments I asked for or the changes to what happens when the GATK_LAUNCH_SCRIPT is null. |
There was a problem hiding this comment.
@jamesemery Looks good to me. 👍 when travis gives it the all clear
)" This reverts commit d8588cc.
Running just "docker build ." in a gatk clone works with this branch. Unfortunately unless you then run a prune command you will end up with a 9 GB "waste" docker image from the first stage leftover. The build.docker script handles this situation by deleting the intermediate stage using tags...
Fixes #6747