Add interlocks for move_to_xyz scripts. #1319

fnothaft · 2016-12-18T21:41:21Z

Resolves #1317. Depends on #1318.

CC @shaneknapp this should prevent the problem you saw this week from happening again.

AmplabJenkins · 2016-12-18T21:51:31Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1682/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1319/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains da0fd0a # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1319/merge^{commit} # timeout=10Checking out Revision da0fd0a (origin/pr/1319/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f da0fd0aee7456485f314d9dc6630d568335aac8cFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

heuermh · 2016-12-19T03:05:49Z

scripts/move_to_spark_1.sh

+
+set +x
+
+grep -q "spark2" pom.xml


Not entirely sure I understand your objection here. We run the same command in both move_to_spark_1 and move_to_spark_2 but we check for different exit codes. In the Spark 1 script, we check for exit code 1 which indicates that "spark2" was not found in pom.xml, while in the Spark 2 script, we check for exit code 0, which indicates that "spark2" was found in pom.xml.

Ah right, I see that now

Yeah, I agree that it is non-obvious though. Let me touch it up with some inline docs to draw attention to what's going on here.

AmplabJenkins · 2016-12-19T05:51:14Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1683/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1319/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains c314a49ddbae9b191a96cc5318190dc4f61e626b # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1319/merge^{commit} # timeout=10Checking out Revision c314a49ddbae9b191a96cc5318190dc4f61e626b (origin/pr/1319/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f c314a49ddbae9b191a96cc5318190dc4f61e626bFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2016-12-19T07:36:30Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1684/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1319/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains df9842722ceff5d115425205b3e4cd7ba76addd7 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1319/merge^{commit} # timeout=10Checking out Revision df9842722ceff5d115425205b3e4cd7ba76addd7 (origin/pr/1319/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f df9842722ceff5d115425205b3e4cd7ba76addd7First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

shaneknapp

some of these look like testing commits... please let me know when the test code is removed so i can give a better review. :)

shaneknapp · 2016-12-19T18:41:36Z

scripts/move_to_spark_1.sh

@@ -2,6 +2,15 @@

 set +x

+grep -q "spark2" pom.xml
+if [[ $? == 1 ]];
+then


i think it'd be better, and more consistent w/the other script(s), if you grep for spark1, check for a 0 exit code, and then exit.

The Spark 1.x artifact names don't contain "spark1". We append "spark2" to the artifact names when building for Spark 2.

shaneknapp · 2016-12-19T18:42:08Z

scripts/move_to_spark_1.sh

+    echo "Cowardly refusing to move to Spark 1..."
+
+    exit 1
+fi


do you want to fail out of the build if it's already properly set up?

Yeah, mostly as a heads up.

shaneknapp · 2016-12-19T18:43:42Z

scripts/move_to_scala_2.10.sh

+find . -name "pom.xml" -exec sed -e "s/2.11.8/2.10.6/g" \
+    -e "s/2.11/2.10/g" -i.2.10.bak \
+    -e "/no Scala/ s/Scala 2.10/Scala 2.11/g" -i.2.10.bak \
+    '{}' \;
 find . -name "*.2.10.bak" -exec rm {} \;


generally, it might be good to put a -f flag on these files to ensure they get deleted. completely optional, tho

+1, will do

rm -f? (if you don't want to bother adding -f to all of your rm calls this time around, then please feel free to punt)

shaneknapp · 2016-12-19T19:30:23Z

another thought: since the sed/etc scripts are quite small, it might be more readable if everything is located in one script w/function calls.

AmplabJenkins · 2016-12-21T23:04:39Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1686/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1319/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 1b7df1d72133ed6b6768344d679be4d135626c20 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1319/merge^{commit} # timeout=10Checking out Revision 1b7df1d72133ed6b6768344d679be4d135626c20 (origin/pr/1319/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 1b7df1d72133ed6b6768344d679be4d135626c20First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

fnothaft · 2016-12-22T09:26:05Z

Jenkins, retest this please.

AmplabJenkins · 2016-12-22T09:37:56Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1689/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1319/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 1b7df1d72133ed6b6768344d679be4d135626c20 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1319/merge^{commit} # timeout=10Checking out Revision 1b7df1d72133ed6b6768344d679be4d135626c20 (origin/pr/1319/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 1b7df1d72133ed6b6768344d679be4d135626c20First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

fnothaft · 2016-12-23T00:38:25Z

Jenkins, test this please.

AmplabJenkins · 2016-12-23T01:10:26Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1693/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1319/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 1b7df1d72133ed6b6768344d679be4d135626c20 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1319/merge^{commit} # timeout=10Checking out Revision 1b7df1d72133ed6b6768344d679be4d135626c20 (origin/pr/1319/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 1b7df1d72133ed6b6768344d679be4d135626c20First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2016-12-23T05:18:01Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1694/
Test PASSed.

fnothaft · 2016-12-23T05:20:13Z

Squashed down, this should be good to go.

AmplabJenkins · 2016-12-23T05:43:52Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1695/
Test PASSed.

fnothaft · 2016-12-23T18:13:42Z

@heuermh do you want this in 0.21.0, or shall we bump it? I have it tagged in the release and I think it is good to go, if you can merge.

heuermh · 2016-12-23T18:16:18Z

If you feel good about it (I haven't been looking closely at any of the jenkins failures), then I'm fine with it going in 0.21.0. @shaneknapp can we get a +1 from you?

fnothaft · 2016-12-23T18:17:38Z

If you feel good about it (I haven't been looking closely at any of the jenkins failures), then I'm fine with it going in 0.21.0.

Yeah, I think it is good to go. The jenkins failures were due to not cleaning up the POMs after modification and failing the git status --porcelain test, not test failures. Those are resolved now.

shaneknapp · 2016-12-23T18:32:48Z

taking a quick look now...

shaneknapp

the profusion of these shell scripts to hack the build environment is fragile... as a longer-term goal i think it'd be wise to refactor (or even replace them w/something like python+jinja templates).

shaneknapp · 2016-12-23T18:33:27Z

scripts/jenkins-test

+./scripts/move_to_spark_1.sh
+if [[ $? == 0 ]];
+then
+    echo "Moving to Spark 1 did not cowardly fail!"


i really don't like these messages. please clean them up.

OOC, what is it that you don't like about them? I'm willing to clean them up, but I don't have enough info here to know what direction to clean them in.

"did not cowardly fail" is not as clear as, for example, "completed updating pom.xml for spark1"

Oh, sure! Let me get those updated.

shaneknapp · 2016-12-23T18:33:45Z

scripts/jenkins-test

+./scripts/move_to_scala_2.10.sh
+if [[ $? == 0 ]];
+then
+    echo "Moving to Scala 2.10 did not cowardly fail!"


same, re: message

shaneknapp · 2016-12-23T18:33:59Z

scripts/jenkins-test

+    ./scripts/move_to_spark_2.sh
+    if [[ $? == 0 ]];
+    then
+        echo "Moving to Spark 2 twice did not cowardly fail!"


same, re: message

shaneknapp · 2016-12-23T18:34:05Z

scripts/jenkins-test

+    ./scripts/move_to_scala_2.11.sh
+    if [[ $? == 0 ]];
+    then
+        echo "Moving to Scala 2.11 twice did not cowardly fail!"


same, re: message

shaneknapp · 2016-12-23T18:34:49Z

scripts/move_to_spark_2.sh

+if [[ $? == 0 ]];
+then
+    echo "POM is already set up for Spark 2."
+    echo "Cowardly refusing to move to Spark 2..."


same, re: message

shaneknapp · 2016-12-23T18:37:15Z

scripts/move_to_scala_2.10.sh

+find . -name "pom.xml" -exec sed -e "s/2.11.8/2.10.6/g" \
+    -e "s/2.11/2.10/g" -i.2.10.bak \
+    -e "/no Scala/ s/Scala 2.10/Scala 2.11/g" -i.2.10.bak \
+    '{}' \;
 find . -name "*.2.10.bak" -exec rm {} \;


rm -f? (if you don't want to bother adding -f to all of your rm calls this time around, then please feel free to punt)

shaneknapp · 2016-12-23T18:37:56Z

scripts/move_to_scala_2.11.sh

+if [[ $? == 0 ]];
+then
+    echo "Scala version is already set to 2.11."
+    echo "Cowardly refusing to move to Scala 2.11..."


same, re: message

shaneknapp · 2016-12-23T18:37:58Z

scripts/move_to_scala_2.10.sh

+if [[ $? == 0 ]];
+then
+    echo "Scala version is already set to 2.10."
+    echo "Cowardly refusing to move to Scala 2.10..."


same, re: message

fnothaft · 2016-12-23T18:45:48Z

the profusion of these shell scripts to hack the build environment seems to me to be fragile... as a longer-term goal i think it'd be wise to refactor (or even replace them w/something like python+jinja).

Yeah, @ryan-williams has been looking at ways to do this with sbt. I don't like sbt at all, but I would consider moving if it eliminates this problem.

shaneknapp · 2016-12-23T18:50:25Z

i have... mixed feelings about SBT as well.

shaneknapp · 2016-12-23T18:57:37Z

btw, i'm about to leave for the afternoon (bowling w/the inlaws+kids), so don't hesitate on merging if everything passes. :)

fnothaft · 2016-12-23T18:57:56Z

Yeah, would be good to discuss more @shaneknapp. I think @heuermh and I are pretty much of the mindset that while Maven is not a beautiful tool, it is a solid tool. I agree with @ryan-williams that the build hacking is crufty, but I've found SBT to be wanting in the past and have migrated several projects off of SBT in the past 4 years. From what I hear, SBT has been improving a lot though.

Also, I've just updated the error messages. Let me know if they look better, and have a great time bowling!

AmplabJenkins · 2016-12-23T19:13:42Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1703/
Test PASSed.

AmplabJenkins · 2016-12-23T19:34:10Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1704/
Test PASSed.

shaneknapp · 2016-12-23T22:26:42Z

scripts/move_to_scala_2.10.sh

+if [[ $? == 0 ]];
+then
+    echo "Scala version is already set to 2.10 (Scala artifacts have _2.10 version suffix in artifact name)."
+    echo "Cowardly refusing to move to Scala 2.10 a second time..."


missed one....

I think I misunderstood you the first time around. I thought that what you were saying was that the previous messages weren't sufficiently descriptive, so I added to them in bdc90a8. E.g., here we went from:

echo "Scala version is already set to 2.10." echo "Cowardly refusing to move to Scala 2.10."

to:

echo "Scala version is already set to 2.10 (Scala artifacts have _2.10 version suffix in artifact name)." echo "Cowardly refusing to move to Scala 2.10 a second time..."

Since these messages are still not looking good to you, I think I must've misunderstood your critique. If you've got a minute, perhaps it'd be useful if you gave an example of what you think a good message would be?

shaneknapp · 2016-12-23T22:26:53Z

scripts/move_to_scala_2.11.sh

+if [[ $? == 0 ]];
+then
+    echo "Scala version is already set to 2.11 (Scala artifacts have _2.11 version suffix in artifact name)."
+    echo "Cowardly refusing to move to Scala 2.11 a second time..."


and another...

shaneknapp · 2016-12-23T22:27:27Z

scripts/move_to_spark_1.sh

+if [[ $? == 1 ]];
+then
+    echo "POM is already set up for Spark 1 (Spark 1/2 artifacts are missing -spark2 suffix in artifact names)."
+    echo "Cowardly refusing to move to Spark 1 a second time..."


...and another.

shaneknapp · 2016-12-23T22:27:40Z

scripts/move_to_spark_2.sh

+if [[ $? == 0 ]];
+then
+    echo "POM is already set up for Spark 2 (Spark 1/2 artifacts have -spark2 suffix in artifact names)."
+    echo "Cowardly refusing to move to Spark 2 a second time..."


shaneknapp · 2016-12-23T22:32:25Z

scripts/jenkins-test

+# this next line is supposed to fail
+set +e
+
+echo "Rewriting POM.xml files to Scala 2.10 and Spark 1 should error..."


remove this comment (or add more comments when running the subsequent scripts)

fnothaft · 2016-12-29T06:30:34Z

Heads up to @shaneknapp: @heuermh and I spoke today, and this commit is blocking a release that we're trying to cut ASAP. We're going to move forward on merging this PR (once I do a bit of cleanup) and I will circle back with you after new years to sort out the error message problems.

Resolves bigdatagenomics#1317.

fnothaft · 2016-12-29T06:32:27Z

@heuermh this is good to go from my side.

AmplabJenkins · 2016-12-29T06:55:49Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1710/
Test PASSed.

shaneknapp · 2016-12-29T15:52:16Z

LGTM. Merge away, and we can discuss next week.

…

On Dec 29, 2016 1:55 AM, "UCB AMPLab" ***@***.***> wrote: Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1710/ Test PASSed. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1319 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABiDrCpN8QFFhb6VKZhU4GxN55wvhwhAks5rM1l3gaJpZM4LQPe0> .

heuermh · 2016-12-29T16:08:55Z

Thank you, @fnothaft @shaneknapp!

fnothaft · 2016-12-29T16:21:19Z

Thanks @heuermh and @shaneknapp. I've created #1331 to follow up.

heuermh requested changes Dec 19, 2016

View reviewed changes

fnothaft mentioned this pull request Dec 19, 2016

[ADAM-1307] Add script for moving to Spark 1. #1318

Merged

heuermh approved these changes Dec 19, 2016

View reviewed changes

fnothaft force-pushed the issues/1317-interlock-move-scripts branch from e7a3ca4 to 4e70a98 Compare December 19, 2016 05:38

shaneknapp reviewed Dec 19, 2016

View reviewed changes

heuermh mentioned this pull request Dec 19, 2016

Add move_to_spark_1.sh script and update README to mention #1307

Closed

fnothaft force-pushed the issues/1317-interlock-move-scripts branch from ffb547c to 5c236b4 Compare December 23, 2016 05:20

fnothaft added this to the 0.21.0 milestone Dec 23, 2016

shaneknapp suggested changes Dec 23, 2016

View reviewed changes

fnothaft force-pushed the issues/1317-interlock-move-scripts branch from 3555e4a to 70fe5da Compare December 23, 2016 18:49

shaneknapp reviewed Dec 23, 2016

View reviewed changes

fnothaft mentioned this pull request Dec 27, 2016

[ADAM-1307] move_to_spark_2 fails after moving to scala 2.11. #1329

Merged

fnothaft force-pushed the issues/1317-interlock-move-scripts branch from bdc90a8 to 32749d9 Compare December 29, 2016 06:30

[ADAM-1317] Add interlocks for move_to_xyz scripts.

d774073

Resolves bigdatagenomics#1317.

fnothaft force-pushed the issues/1317-interlock-move-scripts branch from 32749d9 to d774073 Compare December 29, 2016 06:32

heuermh merged commit 0ad63f1 into bigdatagenomics:master Dec 29, 2016

fnothaft mentioned this pull request Dec 29, 2016

Follow up on error messages in build scripts #1331

Closed

Add interlocks for move_to_xyz scripts. #1319

Add interlocks for move_to_xyz scripts. #1319

Conversation

fnothaft commented Dec 18, 2016

AmplabJenkins commented Dec 18, 2016

Build result: FAILURE

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AmplabJenkins commented Dec 19, 2016

Build result: FAILURE

AmplabJenkins commented Dec 19, 2016

Build result: FAILURE

shaneknapp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shaneknapp Dec 23, 2016 • edited Loading

Choose a reason for hiding this comment

shaneknapp commented Dec 19, 2016

AmplabJenkins commented Dec 21, 2016

Build result: FAILURE

fnothaft commented Dec 22, 2016

AmplabJenkins commented Dec 22, 2016

Build result: FAILURE

fnothaft commented Dec 23, 2016

AmplabJenkins commented Dec 23, 2016

Build result: FAILURE

AmplabJenkins commented Dec 23, 2016

fnothaft commented Dec 23, 2016

AmplabJenkins commented Dec 23, 2016

fnothaft commented Dec 23, 2016 • edited Loading

heuermh commented Dec 23, 2016

fnothaft commented Dec 23, 2016

shaneknapp commented Dec 23, 2016

shaneknapp left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shaneknapp Dec 23, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fnothaft commented Dec 23, 2016

shaneknapp commented Dec 23, 2016

shaneknapp commented Dec 23, 2016

fnothaft commented Dec 23, 2016

AmplabJenkins commented Dec 23, 2016

AmplabJenkins commented Dec 23, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fnothaft commented Dec 29, 2016

fnothaft commented Dec 29, 2016

AmplabJenkins commented Dec 29, 2016

shaneknapp commented Dec 29, 2016 via email

heuermh commented Dec 29, 2016

fnothaft commented Dec 29, 2016

shaneknapp Dec 23, 2016 •

edited

Loading

fnothaft commented Dec 23, 2016 •

edited

Loading

shaneknapp left a comment •

edited

Loading

shaneknapp Dec 23, 2016 •

edited

Loading